Platform & DevOps
Operate AI like any other service: a command line, Postgres, queues, AWS Fargate, tracing, resumable runs.
Azathoth 1.1.0 is out: now on AWS
Cut your AI costs.Prove quality held.
Eldritch benchmarks, optimizes and governs the AI workflows your company already runs, using Azathoth, our open-source engine that runs in your own AWS account. Every change comes with the evidence behind it.
- Open source
- Runs in your AWS account
- OpenRouter & Amazon Bedrock
- Nothing ships without proof
The problem
Nobody can say what your AI is really costing you.
Most AI workflows live inside application code or inside a model provider's agent loop. So the basic questions go unanswered:
- Which model does each step actually need?
- Would a cheaper one do just as well?
- What did yesterday's runs cost, step by step?
- Why did last week's output change?
Changing a prompt or a model means a code deploy, and the provider owns the flow. We think you should own it.
How it works
Measure. Improve. Ship on purpose.
Azathoth records every step of every run as evidence: which model ran, what it was given, what it cost, how long it took. That record is what makes safe optimization possible.
- 01
Benchmark
Turn your real cases into a test set, then measure what today's workflow costs, how fast it runs, and how often it's right.
- 02
Optimize
Try cheaper models one step at a time. Each candidate runs against the same benchmark, and the measurements pick the winner.
- 03
Promote
Nothing reaches production until someone deliberately promotes the winner. Every promotion is kept as an audit record.
Every step is evidence
Who it's for
AI work, moved into operations.
Developers build tools. Experts compose workflows from them. Platform teams run, benchmark, optimize and promote them. Each person does the part they're good at.
Engineering leaders
Developers build tools once. Domain experts compose workflows from them. Prompt and model changes stop needing a redeploy.
Finance & cost owners
See what every step of every run cost, and find cheaper models that pass the same benchmark.
Risk & compliance
An approved model list, least-privilege tools, append-only history, and traces that never carry content.
AWS shops
One CloudFormation template. Amazon Bedrock built in. A better home for AI workflows than Step Functions.
Domain experts
Compose and test workflows as plain documents, without waiting on a developer.
Start small
One workflow. Two weeks. Real numbers.
In a Workflow Cost Audit, we rebuild one of your AI workflows in Azathoth, build a benchmark from your real cases, and run optimization against it.
You get a report comparing what you pay today with two or three cheaper options, each with measured quality, latency and cost. Evidence, not opinion.
Built to be checked
Every claim, measured.
Judged the same way, every time
Results are scored by exact match, normalized match, JSON schema, or an AI judge. Every candidate faces the same judge on the same case.
Failures can't win on price
A run that fails scores zero and earns no credit for speed or cost, so a cheap model that breaks never looks like a bargain.
Nothing is edited in place
Every workflow, run, experiment and promotion is kept as a permanent revision. You can always see what changed, when, and why, and roll back by re-importing.
Who you'll work with
Eris Discordia Montano
Founder & Principal Engineer
Thirty years building systems people depend on: early online game worlds, a cell-tower alerting system Nextel's engineers relied on for years, and systems for healthcare, pharma, logistics and agriculture companies.
She brought GPT-3.5 into Ocelot Consulting when the field was brand new and started the firm down the path to AI consulting. Then she built Azathoth, solo and in the open, from first commit to 1.1.0 in about three months.
Open source
Try it yourself.
Azathoth is free and open source. Your team can install it today and run it on a laptop, or deploy it to AWS from one template.
pip install azathoth-aiNext step