The average organization scraps 46% of its AI proof-of-concept projects before they ever reach production. That number comes from S&P Global’s Voice of the Enterprise: AI & Machine Learning, Use Cases 2025, and it was the first slide of substance in our session at AWS Summit Zurich on September 2. Four days later I’m still thinking about it, because the room’s reaction told me everything: nobody flinched. Every architect in there has a graveyard.
Some of those cancellations are healthy. An experiment that fails is an experiment doing its job. But the ones that hurt are the projects where the demo worked, leadership was excited, and the budget was approved, and then nothing shipped.
From a talk about the problem to a talk about the fix
Six months ago I stood on a stage at Mobile World Congress and argued that healthcare AI has a velocity gap: AI evolves at venture-capital speed, regulated healthcare adopts at regulatory speed, and the mismatch is a solvable engineering problem rather than a permanent condition.
That talk described a problem. This one, session AIM302, “From Demo to Deployment: Solving Agentic AI’s Toughest Challenges,” was about what we actually built.
I shared the stage with Alberto Antoñanzas, Senior Software Architect in the same Architecture and Software Engineering team at RIS Roche Diagnostics, and Pinaki Banerjee, Senior Solutions Architect for Healthcare and Life Sciences at AWS. Pinaki opened with the framework. Alberto and I brought the receipts.
Backstage, mics on, thirty seconds of pure nerves before AIM302. One of us was better dressed for it than the other two.
The chasm has exactly four sides
The framing Pinaki used, and the one I now use internally, is that the prototype-to-production gap is not one problem. It’s four, and teams that cross the chasm address all of them:
| Pillar | The question it answers |
|---|---|
| Operational Excellence | Can we see what’s happening and control what we’re spending, at scale? |
| Data & Context | Does the agent have the right information at the right time? |
| Reliability | Does it behave consistently, and can we prove it? |
| Trust | Is user data protected, and does the agent stay inside its boundaries? |
On the left of that chasm: your prototype. Runs on your laptop, one user, usually works, cost doesn’t matter yet. On the right: thousands of users, unpredictable inputs, and “usually works” is not a standard anyone will sign off on.
Guess which pillar teams skip.
Most agent failures are data failures
If you take one thing from AIM302, take this. When an agent answers wrong, the instinct is to blame the model, so people go shopping for a better one. But look at the actual failure modes: the wrong documents were retrieved, the relevant document was never indexed, the context was stale, memory didn’t survive the session, or the agent had the data and had no idea what it meant in your business.
The model is rarely the bottleneck. If you fix only one thing before going to production, fix your data pipeline.
That line got the most nods of anything on our slides. It is also the least glamorous advice in agentic AI, which is probably why it keeps needing to be said.
Agents are not a hammer
The second idea I’d tattoo on a whiteboard: use code when possible.
I’ve watched teams build an agent that reasons its way through “take Q3 revenue, subtract Q2, divide by Q2.” It works. It’s also slow, expensive, and occasionally wrong, which is a strange set of properties for arithmetic.
Decompose one agent task and you can see the seam. Understanding the request, deciding what to do next, and explaining the result: that’s reasoning, give it to the agent. Fetching, calculating, validating against business rules: that’s a Python function that runs in milliseconds, costs nothing, and returns the same answer every time.
Let agents orchestrate code rather than replace it. In a regulated environment this is not just an efficiency argument, it’s an auditability argument. A deterministic function has a testable contract. A prompt has a vibe.
What we built: the AI-SDLC Orchestrator
The Roche half of the talk was our platform, an end-to-end AI-first software development lifecycle built for regulated product development. It consolidates research and concept, requirements and specs and UX, implementation and verification and validation, and shipping into one governed environment. The design draws on the AI-Driven Development Lifecycle (AI-DLC) method definition by Raja SP at AWS.
The shape of it, in numbers we put on screen: 9 agents, 20 skills, 80 agent tools, 100% Git-based, 1 plugin for everything.
The agent catalogue is organized by intent and outcome rather than by technology, orchestrated with LangGraph:
- Researcher supports product conceptualization
- Business Owner maps product goals into structured requirements
- Technical Expert translates requirements into technical specifications
- UX / UI translates requirements into compliant user journeys
- Developer generates test-driven code from specs
- Risk Manager scans specifications and generates the risk assessment
- Quality Manager validates regulatory compliance
- The Librarian and The Orchestrator run across every phase, indexing enterprise knowledge and carrying context between transitions
Spec-to-code runs on Claude Code on Amazon Bedrock, gated by test-driven development, a quality review, a spec compliance review, and then human review plus CI. In a regulated environment the human gate is not optional, and I have no interest in pretending otherwise.
Two design decisions matter more than the agent list.
Process as code. Our regulatory process lives in a single .yml file: rules, templates, quality schemas, and agent skills, maintained by the quality and regulatory functions themselves. Publish a new process version, products accept it, agents bind to the new deliverables automatically. One source of truth, centrally governed, propagated the moment it lands.
The LLM Wiki. Corporate knowledge in a large organization is scattered by default. We consolidate product knowledge behind multi-source enterprise connectors, and the Librarian agent syncs it, indexes it, and, the part I care about most, flags drift between newly ingested files and existing specifications before a single line of code gets written. A side effect nobody expected to love as much as they do: slide decks generated on demand from live specs, traced back to the actual business and technical specifications. No copy-paste, no stale numbers.
The constraints are real, and I said so on stage. ISO/IEC 42001 is now part of the conversation. So are codebases older than ten years, which is where the actual business runs. For scale context, Roche Diagnostics digital solutions run more than 20 products, over 10,000 live software instances worldwide, supporting more than 50 million in-vitro diagnostic tubes per day.
The slide I was slightly nervous about
Our architecture slide contains an admission, and I think it’s the most useful thirty seconds of the whole session.
Today the agents do not run on Amazon Bedrock AgentCore Runtime. They run LangGraph on Amazon EKS, alongside the API, the Git file server, and the docs processor. Why? Because we already had an approved Kubernetes platform and, more importantly, an approved compliance story wrapped around it. In a regulated setting the pre-existing audit trail is worth more than the newer primitive.
We are now migrating to AgentCore Runtime and taking the stack with us. What makes that a low-risk move is the thing Pinaki had shown an hour earlier: Amazon CloudWatch generative AI observability instruments agents on either side of that line, hosted on AgentCore Runtime or on EC2, EKS, on-prem, another cloud, as long as the telemetry is OpenTelemetry. So we keep the full agent trace before, during, and after the migration.
That’s the whole argument for open telemetry standards in one sentence, and it took us a production platform to feel it properly. Instrument to a standard and your migrations stop being blind.
For the record on maturity, because I don’t like previews described as products: AWS Agent Registry went GA on August 6, 2026. AgentCore Evaluations has been GA since March 2026. Policy and Evaluations are new. Insights is still preview.
What I’d tell you to do on Monday
Crawl, walk, run, then fly. Not as a motivational poster, as a sequencing constraint.
- One use case. One agent. Prove it works.
- Add observability and tools. Deploy to ten users. Learn what actually breaks, which will not be what you predicted.
- Scale to hundreds with platform standards, so the second team doesn’t rebuild the first team’s plumbing.
And before any of that, spend the first week on your data pipeline rather than your model choice.
AWS made the case that you don’t have to solve the foundations alone, and they have numbers for it: their Generative AI Innovation Center moved 65% of customer initiatives from pilot to production last year, which they put at roughly twice the industry baseline, some in as little as 45 days. Those are AWS’s figures, not mine. What I can say from our side is narrower and maybe more useful: the pillar we underestimated was Data and Context, and the decision that has aged best is putting the regulatory process in a file that the quality team owns.
Six months ago in Barcelona I had a diagnosis. In Zurich I had an architecture diagram with an embarrassing honest box on it. The next talk is the one where the box is gone, and I already know the migration will teach me something I’ll have to admit on stage again.
What’s the pillar your team keeps skipping? My money is on observability, because it never blocks a demo and always blocks a deployment. Tell me I’m wrong on X or LinkedIn, especially if you’ve shipped agents into a regulated environment and did it differently.
Keep reading: the predecessor to this piece is why healthcare AI has a velocity gap and how we’re closing it, and if you want the developer-level view of working this way, see why I’m faster with AI code tools even though the studies say I shouldn’t be. More in the AI category.
This post is a retrospective of session AIM302 at AWS Summit Zurich, September 2, 2026, and covers only material presented on stage. Views are my own.