What a Four-Week AI Proof of Concept Should Prove Before You Scale It

A proof of concept that impresses in a demo can still be the wrong thing to scale. The model handles the sample cases, the room is convinced, and a few months later the project has quietly stopped. Gartner reported in January 2026 that at least half of generative AI projects had been abandoned after proof of concept by the end of 2025, citing poor data quality, inadequate risk controls, escalating costs or unclear business value.

None of those reasons is about whether the model can do the task. That is usually the easy part. A good proof of concept is built to answer the harder questions while the stakes are still small. If you are still choosing which process to test, start with five questions that pick the right first automation.

A demo is not a proof

A demo shows that something is possible. A proof shows that it works on your data, at your volume, for the people who will use it. The difference matters because the model is only a small part of a working system. In a well-known paper on machine learning in production, Google engineers showed that the model code is a small fraction of a real-world system, and that the infrastructure around it is vast and complex.

So a four-week test should do more than ask whether the model can do the job. It should run the narrowest real version of the whole path: real input arriving, the model doing its step, the result landing where people work, and a person able to correct it.

Write the pass mark before week one

A pilot without an agreed definition of success tends to drag on, because every result is open to interpretation. Agree three things before anything is built.

• A test set of real cases. Collect historical examples with known correct outcomes, in the mix you actually receive, including the awkward ones. Anthropic's guidance on evaluations recommends test sets that mirror the real distribution of the task and include edge cases.
• A baseline. Measure how the work is done today: time per item, error rate, turnaround. Google's Rules of Machine Learning put metrics first and advise tracking as much as possible in the current system before formalising what the new one will do.
• A threshold for each measure. Write down the result that would justify scaling and the result that would make you stop. Make each one specific and measurable: "the right category on routine requests, and no confident errors on refunds" rather than "works well".

Five things four weeks should prove

Each row below matches a reason pilots fail later. Use the table as the agenda for the final review.

On data: if the test cases contain personal data, the GDPR applies to the pilot just as it does to production. Use only the data the test needs (the data minimisation principle in Article 5) and confirm how it may be used before week one.

Cost deserves its own check. Gartner notes that projects which look viable in a proof of concept can become budget problems in production, because small per-use costs multiply across users and use cases. Measure cost per item during the pilot, and read the real cost of building an AI system for what else to budget.

A four-week plan that leaves room for the answer

Four weeks is enough when the scope is one process and one decision. A shape that works:

1. Week 1: agree and collect. Fix the pass mark, assemble the test set, get access to the real data and name the people who will try the system.
2. Week 2: build the thin path. Connect input, model and output end to end for the main case only. Run the full test set at the end of the week, even if the results are poor.
3. Week 3: put it in front of users. Let the team use it on live work alongside the normal process, so nothing depends on it yet. Log every correction and measure cost per item.
4. Week 4: fix, rerun, decide. Address the most common error types, run the test set again and write a one-page decision note that sets the numbers against the pass mark.

Running the same test set in week two and week four is the point. It turns progress into a number and keeps the final review from becoming a debate about impressions. The NIST AI Risk Management Framework says AI systems should be tested before deployment and regularly while in operation, and the test set you build now can become the regression test for the live system.

Scale, adjust or stop

At the end of week four there are three honest outcomes.

• Scale when every row of the table has evidence, the pass mark is met and a named person in the business owns the result. The next phase is engineering work: integration, monitoring, access control and support.
• Adjust when one or two rows are weak and the fix is clear, such as a missing data source. Agree a short extension with a fixed end date and the same pass mark. Adjusting should not become a permanent state.
• Stop when quality is far from the mark, or when the value disappears once running cost and review time are counted. A pilot that ends this way after four weeks has done its job: it spared you a much larger project.

In our projects, the pilots that move smoothly into production are the ones where the decision rules were written down first. To set a realistic pass mark, estimate what the process is worth today with the ROI calculator, or browse common workflows by business area for candidates worth testing.