How to judge AI vendor claims before you sign
By Rodney Noriega, Founder, Universal Enablement · Published Oct 12, 2026
Short answer
Treat every AI claim as a prediction until you've tested it on your own work. Ask what it gets wrong, what happens to your data, and which AI model it's actually built on. Then ask for a small pilot with results you can measure. A good vendor welcomes that. A weak one pushes for a big commitment instead.
AI has a long record of confident promises. The 1956 project that named the field expected major progress in one summer. Experts have been wrong in both directions, and many loud voices have money riding on you believing them.
A useful habit: picture the best and worst reason a vendor could have for a claim. Then ask which one better fits what they say and do. It's not cynicism. It's a quick way to see where to dig.
Many AI products are built on a model from OpenAI, Anthropic, Google or Meta, with their own features on top. That's fine, but you should know which one, because it affects quality, privacy and cost.
Six questions to ask:
- Can we test this on our own work before we commit?
- What does it get wrong, and how often?
- Which AI model is it built on, and what happens if that changes?
- Where is our data stored, who can see it, and is it used to train the model?
- How does it support our obligations under BC PIPA and, where it applies, PIPEDA?
- What happens to our workflow if you raise prices or shut down?
If the answers are vague, the product probably is too.
Related
Start with one workflow. In 30 minutes we'll look at where your team's time goes and tell you honestly whether AI can help.
Book a free 30-minute workflow review