Four questions worth asking before you buy any AI product

Platform · PolyQ Team

Four questions worth asking before you buy any AI product

Most AI evaluations focus on the demo. The things that determine whether a deployment survives contact with your business are usually not in the demo at all. Here are the four we would ask.

We build AI products, and we also sit on the other side of the table evaluating them. The questions that separate a system you can run for years from one you quietly turn off after six months are consistent, and they are rarely the ones that come up in a first meeting.

  • Where does the answer come from? A system grounded in your documents, policies, and records behaves differently from one drawing on a model's general knowledge. Ask to see the source of a specific answer, not just the answer.
  • Who can change its behavior, and is that change recorded? If the instructions driving the system can be edited without review or history, you have no way to explain its behavior later. Versioning and approval are not enterprise nice-to-haves; they are how you stay accountable.
  • Where does the data live, and for how long? Retention, residency, and deletion should be configurable and enforced by the product, not promised in a contract. Ask what is stored, where, and what happens when you ask for it to be gone.
  • What happens when it is wrong? Every AI system is wrong sometimes. The question is whether a correction is a support ticket or a first-class feature that improves the system permanently.
  • Who configures it after the sale? If every change requires the vendor or an engineering backlog, the system will drift out of date. The people who own the process should be able to adjust it.

Good AI products are not the ones that demo best. They are the ones whose behavior you can explain, adjust, and account for a year after you bought them.

Bring these questions to us. We would rather answer them early than discover a mismatch after deployment.

More from Thinking