Econify brings top engineering talent to your toughest problems around AI workflows, Data readiness, Cloud cost containment, OTT and Web. Partner with us to get things done.
Challenge
Criteria for Success built Collavia, a sales enablement tool, with a team that had no professional software engineers on it. They worked in browser windows, prompting ChatGPT and Claude, and moved the output into place by hand. It took them further than anyone would predict as the people who understood the sales problem were the ones shaping the product. They had working software while a conventional build would still have been in discovery. However, it was not without problems.
What existed at the end of that was three separate scripts sharing data through an API proxying a storage bucket, running only when embedded inside another system. Releasing a change meant one person running a command from one laptop. No repository, no review trail, no way to trace how a decision had been made or roll back a step that broke something. The product’s own model calls ran against OpenAI at whatever tier had been reached for first, so the cost of serving a customer climbed with every new user and nobody could say by how much.
The constraint showed up commercially first. A product assembled this way demos well. Staffing it, selling it, or handing part of it to a new hire all assume there’s a codebase to hand over, and pricing it becomes challenging given the costs of running against the default OpenAI model.
Goals
Give the existing code a stable home with a review trail and a history
Define a real cloud architecture and ci/cd pipelines, taking deployment off a single laptop
Enable the team to work in the codebase with Claude Code rather than around it by building custom agents and skills for future development
Decide whether three applications should stay three applications
Criteria for Success (CFS) moved from a set of flies and ad-hoc scripts to a professionally deployed application ready for paying customers. Our engineers translated the application into one with a robust architecture. Now, CFS can work with their team to add features thanks to the application structure, infrastructure as code, and agentic harness configuration (custom agents and skills) for future development. A final benefit was the cost savings of using Claude’s Haiku model over the default OpenAI implementation that ChatGPT had driven.
What we found
The prompting was doing its job. The medium was the problem. Collavia existed, people used it daily, and the gap sat entirely in the infrastructure around the tooling: source control, environments, a deployment path that didn’t depend on one machine, and a data store the product actually owned.
Timing is part of the diagnosis. Teams notice this gap at the point where the software has become valuable enough that nobody wants to touch it, which is also the point where changing it costs the most. Criteria for Success was close to that line. Training wasn’t worth running until the code had a stable home, so custody of the code came first.
Our approach
One of our engineers moved everything the team had built into GitHub. Single repository, clean commit history, a branch-and-review workflow. That turned a pile of prompt output into a project other people could join.
Two days of training followed, run against the Collavia codebase rather than a sample project. Day one covered Claude Code fundamentals, model selection, data stores, version control, and the cloud services the team would deploy against. Day two moved to prompt best practices, context management, tests, skills, plugins, and the workflow to carry forward.
The throughline was the set of decisions that quietly cost money or create risk later. Which model to reach for, given that defaulting to the largest one on every task is an expensive habit to form early. How to keep context tight enough that the tool stays accurate as the codebase grows. When to commit, and why the honest answer is more often than feels necessary.
The re-architecture was scoped at the end of those two days, once the team had enough context to deliberate with us about it.
Into production
The three applications became one product in React and TypeScript, backed by DynamoDB and deployed to AWS behind automated CI/CD. The hard part was that Collavia had live users throughout. Nothing was rewritten wholesale. Each slice moved while the previous one kept serving people.
Quality control went into how the code got written rather than into a review at the end. A custom agent with over a dozen purpose-built Claude Code subagents ran continuously during the port, each scoped to a single concern (security, accessibility, typing, testing, and lint coverage, dependency hygiene, API contract conformance), so pressure on quality applied continuously instead of arriving in one pass. The custom agent ships inside the repository as a Claude plugin, along with custom skills and hooks. They’re the guardrails the team at Criteria for Success works inside today.
Two decisions carried measurable weight. Routing the product’s inference to Claude Haiku in place of the incumbent OpenAI model cut model spend by 50% at equivalent output quality, which turns an unpredictable line item into one that scales with seats sold. Refactoring the front end during consolidation improved Largest Contentful Paint by 35%, which matters for SEO and a good end user experience.
Who runs it now
Criteria for Success runs Collavia. Their team works in the repository directly, using Claude Code inside the workflow they were trained on. Releases go through CI/CD, so no single person’s availability gates a fix. Specs, tests and commits still come from the people who understand the sales problem, which was the original advantage and is now a durable one.
What they own
A login-protected product on a domain they control
One repository holding the entire codebase and infrastructure defined as code
Their data in a database they own, backed up automatically
A release process that survives any one person being unavailable
Code written so that engineers and agents can both change it safely
A custom agent, plus Claude skills and hooks, holding contributions inside the guardrails
50%
lower model spend after moving inference from OpenAI to Claude Haiku
35%
improvement in Largest Contentful Paint after the Claude Code refactor
12
subagents enforcing code quality on every change
