Taking an AI project from demo to production
The prototype impresses everyone in the meeting. Three months later, it still isn't live. Here are the four obstacles behind most of these stalls.
Building a convincing AI demo now takes a few hours. Putting it in the hands of your staff or your customers takes weeks. That gap is not about technology: it comes from everything a demo is allowed to ignore.
1. The demo is never wrong
A prototype is shown on three hand-picked examples. In production, it receives questions that are badly worded, off topic, or about documents that don't exist. A language model always answers, even when it doesn't have the information, and its invented answer sounds just as confident as its correct one.
The remedy comes down to two measures: restrict the model to the passages actually retrieved from your documents, and give it explicit permission to say it doesn't know. The second point is often forgotten because it makes the demo less impressive.
2. Nobody measures quality
Without a reference set of questions, every change to the system is a gamble. You adjust a setting, the answer to the question you are testing improves, and you have no idea what got worse elsewhere.
- Put together around thirty representative questions with their expected answers, written by the people who do the work.
- Rerun them after every change and compare the results.
- Tell two kinds of failure apart: the wrong answer and the missing answer. The first is serious; the second is acceptable.
- Keep a record of the scores over time; it is the only proof of progress.
3. The bill is a variable, not a constant
Every request costs in proportion to the amount of text sent and generated. A prototype used by three people costs a few euros a month. The same system opened up to five hundred users, with long documents, moves into a different order of magnitude.
The levers exist and are well known: caching the parts of the context that don't change, selecting the passages sent more carefully, using a lighter model for simple steps, and capping usage per user per day. But they have to be planned before launch.
4. Sensitive data leaves your perimeter
Before sending documents to an external provider, check the terms of the selected plan: possible use for training, retention and processing location. Define which documents may be sent and who may access them. Choosing a business plan does not remove the need for these checks.
An AI project without a written processing note is not ready for production, however technically advanced it may be.
Where to start
Pick a single use case, the one where the time lost is easiest to measure. Put a figure on the expected gain in hours per week. Build it, evaluate it, and open it to five people for a month. If the gain holds up, widen access. If you can't measure the gain, you have picked the wrong use case.
AI integration for your application
Have a prototype to take into production? See the scope, deliverables and pricing of the AI integration service.
Is this question coming up in your business?
Describe your situation. We reply within one working day.
Reply within one working day. You can also write to contact@chenit.eu or call 0456 71 39 98.