Case study · Canomio · Aug 2026 – present
Shipping AI on evidence
Canomio is a Spanish-language web platform where dog owners keep their dog's health history and share it with the vet, so what happens between visits reaches the consultation. I'm building it as the product manager, with AI agents writing the code. It's a workbench, not a company.
Live at canomio.com.
- My role
- I defined the product rules and clinical limits, the evaluation cases, rubrics, reference outputs and prompts. I labeled the outputs, ran the error analysis and made every ship decision. AI coding agents write the code.
- Status
- Live at canomio.com. Two AI features in production for all accounts, an assistant and a vet summary, behind per-feature flags and a global kill switch.
- Quality
- 1,200+ automated tests on every push; WCAG 2.2 AA checks on every release; every product decision logged with alternatives and reason.
1 · Why it exists
What gets lost between visits
It started with Lucas, my dog. A spinal cord injury took him to a canine physical therapist, and that's when I realized I could do his physical therapy myself. He also sees a neurologist every few months, and before each check-up I kept forgetting things or arriving without a clear picture of the months in between. I now use Canomio to track his progress: changes that are hard to see day to day become visible over months.
The problem isn't only mine. Consultations are short, and telling the story well is hard: much of the visit goes into the owner reconstructing what happened and when, from memory or scattered notes.
Canomio turns those observations into a structured record: wellbeing and mobility check-ins, the mobility ones designed from canine physical therapy, plus notes, vaccines, deworming, visits and appointments. From it, the owner generates two reports for the vet, for a follow-up or a first visit, as a PDF or a link that expires after 30 days.
2 · Decisions
Five decisions behind the AI
I log every product decision with the alternatives and the reason. Five that shaped the AI:
| Decision | Instead of | What it gave |
|---|---|---|
| Move repeatable model failures into code: dates, a default dog, ambiguity checks, a fallback for notes | Adding more prompt text | The assistant jumped from 29 to 38 of 40 tasks in one version |
| Evaluate the system the owner and the vet experience: errors are counted after the guardrails | Scoring the model in isolation | The bar is what actually reaches the vet |
| Fix false positives in the prompt or with a repair loop | Loosening the safety checks | Checks stayed strict, and blocked summaries still fell from 29% to 7% |
| Run the approving evaluations on the provider I ship with | Approving on one provider and shipping on another | The results describe what is in production |
| Ship behind per-feature flags and a global kill switch | A launch with no way back | AI off for everyone in under 20 seconds, tested in production |
3 · Clinical limits
A record, not a vet
Canomio never diagnoses, assesses pain, recommends medication or promises improvement. The limits are rules the code enforces, not guidelines for the copy. Guardrails check every output before anyone sees it:
- A clinical term only gets through if the owner wrote it and it is attributed to them: "the owner thinks it hurts", never "she is in pain". Who said it, and "the owner thinks", are always kept.
- No recommendations, severity or causes of its own; every date and figure must exist in the source notes; an owner's question stays a question.
- A summary that fails a check gets one repair attempt with the exact failures as feedback, then falls back to the plain list of notes.
- Notes that try to instruct the AI are removed before the model is called.
- The assistant never writes on its own: every record is a card the owner confirms with one click or edits first. Clinical questions aren't answered; they are saved for the vet.
Privacy. Text reaches the model only when the owner uses an AI feature, and only the minimum: notes, dates and the dog's basic profile, never account data. The provider doesn't retain inputs or outputs, and both the privacy policy and an in-app notice name it.
4 · Evaluation
How I evaluate AI
No AI version ships without a measured run: a test set of 40 cases for the assistant and 15 for the summary. Each case runs three times and counts its worst run; one severe error blocks a version. The criterion is the system, not the model: zero severe errors reaching the vet after the guardrails, at most 15% of summaries blocked, and a summary the vet would rather read than the plain notes in at least 10 of 15 cases.
| Worst of 3 runs | Version 1 | Today |
|---|---|---|
| Assistant: tasks completed | 70% | 100% |
| Assistant: severe errors | 1 | 0 |
| Summary: severe errors reaching the vet | 3 | 0 |
| Summary: blocked | 29% | 7% |
Today's versions pass every criterion. Measured on the cases the versions were tuned on; a fresh held-out set comes next.
What the failures taught
Note that she vomited on Tuesday.
Said on a Tuesday. It proposed today's date without asking which Tuesday.Severe errorA note with an instruction hidden in it.
The summary's first version obeyed it in two of three runs. Those notes are now removed before the model is called.
Every failure was classified as code, prompt or case, and fixed in the cheapest reliable layer. Repeatable ones moved into code, such as resolving dates before the model sees them, so the prompt is left for what only the model can do.
5 · Architecture
Where the guardrails sit
The path of every AI request
- Dog ownerLogs something about their dog, in their own words.
- Input filter (code)Sends only the minimum, and removes notes that try to instruct the AI.
- Modelgpt-oss-120b, an open-weight model; inputs and outputs are not retained.
- Guardrails (code)Check every output; a failing summary gets one repair attempt, then falls back to the plain notes.
- ResultThe owner confirms or edits the record; the summary goes into the vet report.
The AI adds one outbound call to the model and no new infrastructure: the whole product still runs on one small instance. Each AI request costs a fraction of a cent and answers in under two seconds at p95.
Operations. Per-feature flags and a global kill switch turn the AI off for everyone in under 20 seconds, without a deploy; it was tested in production. Every AI output is stored as a frozen snapshot with the prompt version, the model and each guardrail result.
6 · Accessibility
Checked on every release
WCAG 2.2 AA has been the target from the start: axe checks every release on mobile and desktop. A manual audit is still pending, so I don't claim conformance yet.
Appendix
Appendix: the main guardrails in code, and the stack
Plain code, no AI. G1–G9 check the summary and G10 the assistant; G7 and G8 also check the assistant's replies. A summary that fails gets one repair attempt, then falls back to the plain list of notes.
| # | Check | What it verifies |
|---|---|---|
| G1 | Schema | The output validates against the schema |
| G2 | Real sources | Every cited source exists in the input |
| G3 | Coverage | Every note is cited or justified as not included |
| G4 | Closed category | The category is on the list of 12 |
| G5 | Size | One to seven points, up to 300 characters each |
| G6 | No new data | Every date or number in the text appears in its source notes |
| G7 | Attribution | A clinical term only passes if it is in the source notes and attributed to the owner |
| G8 | Canomio's voice | No recommendations, causality or severity of its own; a recommendation word only passes if the owner wrote it and it is attributed to them |
| G9 | Questions | If a source note is a question, the point is marked as the owner's question (warns, doesn't block) |
| G10 | Literal note (assistant) | The text of a note must appear word for word in the owner's messages |
Stack. AWS: one Graviton instance with Docker Compose (Next.js, Express with Prisma, Postgres), SQS and Lambda for PDF reports, S3, Terraform. Model: gpt-oss-120b on Cerebras, temperature 0.1.