The Most Useful Thing Our AI Assistant Does Is Refuse to Answer

An AI now answers questions about your business whether you are involved or not. So we built one for ours, and gave it permission to stop.

We have argued on this blog that customer discovery no longer runs through a single search box. Assistants read, summarize and recommend on your behalf, and they do it from whatever material they can find about you. If that is true for your business, it is true for ours. So we built the first line of RepOtz support as an AI assistant. Her name is Sofia, she works in English, French and Spanish, and she answers at 3 in the morning.

What follows is not a launch note. It is the set of rules we imposed on her, the number we got when we measured her, and the failure we have not fixed yet, because those three things together are the only honest way to talk about putting AI in front of customers.

She only knows what we wrote down

The convenient path was to drop our content into a live chat vendor’s AI feature and let it index the site. We did not take it. That moves our knowledge out of our repository, where it is versioned, reviewed and traceable, and into someone else’s black box.

Instead the knowledge base stays as versioned documentation, and every change produces two things from the same commit: the public documentation humans read, and the corpus Sofia reads. One source, one truth, updated at the same instant. She cannot be a version behind the product, and she cannot tell a customer a different story than our own documentation tells.

The other half of that rule matters as much. Every chapter carries a disclosure level, and only public chapters are exported to her index. Confidential material is not withheld from her by a polite instruction in a prompt. It is absent. “Never reveal this” has never been a security boundary, and any vendor who offers you that as one is selling you a feeling.

On a fact, she cites. Otherwise she says so

Pricing, plan limits, a billing procedure: she answers from retrieved passages and shows the sources under her reply. When she finds nothing, the instruction is not to do her best. It is to say she does not know, and to file the question for a human.

One implementation detail carries most of the trust here. We never ask the model which sources it used, because a model asked that question will happily produce a plausible one. We already know which passages we injected, so the citations are rebuilt on the server from that list. A fabricated source is not discouraged, it is structurally impossible.

That is the trade we made deliberately: an assistant that admits ignorance ten times is worth more than one that invents a price once. An invented price is a commercial dispute.

The bug worth telling you about was scope, not accuracy

Our first version was cold and pushed everything back toward a subscription. The cause was not the wording, it was the design. We had grounded every single turn in retrieved product documentation. So “hello”, “who am I talking to?” and “what is online reputation anyway?” retrieved nothing, hit the fallback, and became support tickets. The assistant read like a brochure and the follow-up queue filled with noise.

The fix was to give her the three registers a good salesperson already uses. Ordinary conversation, answered directly, with no lookup at all. General domain knowledge, where she shares what is known about customer reviews and local visibility, stated plainly as general advice and never as a product promise. And verifiable product facts, where she is bound to her sources or says nothing. She can now talk about online reputation with someone who has never heard of RepOtz, without steering them toward a plan.

If you are about to put AI on your own content, this is the part nobody warns you about. Grounding everything sounds safe. It quietly turns your assistant into a machine that refuses small talk and escalates it.

What the measurement says

We did not want to judge this on impressions. There is a set of 30 real support questions in English, French and Spanish, replayed through the live pipeline, run like a test suite. The last pass:

• 27 correct decisions out of 30
• 4.2 seconds on average for a complete answer
• $0.18 for all 30 conversations, under one cent each

And the 3 errors are all the same case: an out-of-scope question she answered instead of escalating. That is precisely the guarantee we care about most, so it is the next fix rather than a footnote. An automated support line should be judged on its ability to stop, not on its ability to talk, and we would rather publish that number than hide it.

The standard this sets, including for your business

Under a cent per conversation, on infrastructure we already had, with no third-party subscription added. That economics is real, and it is not the interesting part. Two things matter more, and both apply to you as much as to us.

First, every question she could not answer becomes a line in our documentation backlog. She does not write the documentation. She writes the list of what remains to be documented, from what customers actually ask. The front line improves itself, week after week, and the same loop works on your own reviews and your own frequently asked questions.

Second, the standard. AI systems are already summarizing your business to people who have not met you, and increasingly you will be offered AI tools that speak on your behalf. Ask of them what we asked of ours. Where does it get its facts. Will it show them. What does it do when it does not know. Can it repeat something confidential because someone phrased a question cleverly. A tool that cannot answer those four questions should not be answering your customers.

Judge for yourself

No account, no form, no sales demo:

1. Go to repotz.com
2. Click the green bubble in the bottom right corner
3. Ask her real questions, in English, French or Spanish, about RepOtz or simply about customer reviews and your online reputation

If she does not know, she will tell you. That is more or less the whole point. And if you catch her answering something she should have refused, tell us. That is the report we want most.