What an AI pilot is and what you learn with your own cases

Seeing an application run on your own cases is not the same as reading a well-documented, well-structured specification. An AI pilot is exactly that: the base of the application to be built, tested on what really reaches your company, without replacing the way you work yet. The difference shows when something nobody foresaw comes up.

The short answer: an AI pilot, or proof of concept, is a bounded version of the application tested on your company's real cases, without going into production, to validate the product before developing it. It sharpens the requirements: things you thought you needed turn out not to be needed, and new ones nobody had foreseen appear. If it does not convince, the project stops there.

How does a pilot differ from a prepared demo and from production?

All three show up on a screen and serve different purposes:

A pilot is not a small version of the final application already in use. Nor is it a demo to confirm what we already thought: its value is that it shows what the specification could not know.

What do you discover when you test on real cases?

A specification written before seeing anything work describes how we believe we work. The pilot checks it against how we really work. In Tecnea pilots it usually happens that:

• Some things we thought necessary turn out not to be. A step, a field, a report or a connection to another system that looked essential and, once seen on real cases, adds nothing. Each one that is dropped makes the development cheaper.
• New things appear. Cases that were not on the list, documents that arrive another way, rules applied without being written anywhere. It is better to find them now than halfway through development.
• You see where the work really is. Often the part that takes the most time is not the one pointed out at the start.
• You decide what a person reviews. With real cases you see what the AI handles well, where it hesitates and what is worth having someone confirm, and by what criterion.
• Scope and price of what comes next are sharpened. With what has been learned, the next phase is closed with less uncertainty.

Seven ways to get the most out of a pilot

1. Choose a process that really hurts and name who answers for it. A process with a clear start and end and enough volume. If it does not waste time or money every week, what you learn will not be worth it.
2. Decide beforehand what you want to learn and which figure would make you continue. How much the cost per case has to fall, or how much the number of finished cases per week has to rise. Without a figure fixed beforehand, you decide with what you hope to see (how to measure it).
3. Bring real cases, with their exceptions. The pilot is tested on a copy of your documents and data, not on a clean sample. Ask beforehand who holds them and who grants access: it is the most common cause of delay.
4. Gather cases with a known answer. As a guideline, thirty to a hundred real cases, mixing the easy with the hard, with the correct answer written down without looking at what the AI says. That way you measure whether it gets things right without relying on anyone's impression.
5. Have it tested by whoever uses the process every day. IT provides access and security, but the judgement on whether the result is useful belongs to the person who uses it, who can say what is superfluous and what is missing. Human oversight has levels, and it is worth testing the one you will need.
6. Look after personal data in the pilot too. You work on a copy, and if there is personal or sensitive data you decide beforehand which is pseudonymised (replaced by identifiers so the model does not see who is who), where it is processed and with which suppliers there is a contract. More in permissions for an AI agent.
7. Close with a decision and a list. A decision: build what you learned, adjust the pilot or stop. And a list of what goes in, what comes out and what is postponed, which is the starting material for the development.

Common mistakes

• Expecting from a pilot what belongs to production. It is not integrated into all your systems nor designed for the whole team's daily use: that comes with the deployment.
• Testing only on the cases that go well. The ones left out are exactly the ones that consume hours today, and the ones that teach the most.
• Wanting everything right the first time. Seeing where it fails is part of what you are buying: it shows what to review, what to reinforce and what to leave out.
• Not writing down what was learned. If what comes out of the pilot does not turn into changes to the requirements list, you have made an expensive demo.
• Having nobody from the business involved. Without a person who uses the process, the pilot is judged on opinions.

What happens after the pilot

The pilot ends with a decision, and there are three possible outcomes: move to development with the scope already sharpened, adjust the approach and repeat what is needed, or stop. At Tecnea the project goes in phases with a fixed price for each: the first 30-minute conversation is free, the pilot (proof of concept) takes two to four weeks and starts at €850, and only afterwards is the scope of the development fixed (what each amount includes and how long each phase takes, both in Spanish). To choose who to do it with, the criteria for evaluating an AI supplier include what should be put in writing.

Frequently asked questions

What is an AI pilot?

It is a bounded version of the application to be built, tested on your company's real cases and without going into production. It is also called a proof of concept. It validates the product and sharpens the requirements before investing in the full development.

How does it differ from a demo?

A demo usually uses clean documents and the case that goes well. A pilot uses your real cases, the hard ones too, and so it shows what the specification could not know: what is superfluous, what is missing and where the work really is.

How long does a pilot take and how much does it cost?

At Tecnea, between two and four weeks and from €850, with the first conversation free. If it drags on much longer, it is usually because the scope is too large or the data was not ready. The amounts for each phase are in how much an AI application costs (in Spanish).

What data is it done with?

With a copy of real documents and data from your company, including the exceptions. Personal or sensitive data is handled with care from the start: you decide which is pseudonymised and where it is processed.

What happens if the pilot does not convince?

The project stops and the full development has not been bought. It is a valid and cheap result: you find out before investing. You look at what failed (the data, the test cases, the idea) and decide whether to adjust or stop.

Who should take part inside the company?

The person who answers for the process and whoever uses it every day, with IT's support for access and security. To see what happens if the AI gets it wrong (in Spanish), they should be involved from the start.

If you have a process that could be a pilot, tell us which one.

This article is informative. Tecnea develops custom applications with AI and runs paid pilots, so we have an interest in you trying one and, if it pays off, commissioning the development. What is described is how we work, it does not come from an external study, and each company will have its own questions depending on its process and sector.

Sources

1. Regulation (EU) 2016/679, General Data Protection Regulation: eur-lex.europa.eu
2. Regulation (EU) 2024/1689, Artificial Intelligence Act: eur-lex.europa.eu
3. Tecnea, how long an AI project takes and success stories, both in Spanish.