Construct
Define explicit workflows made from models, prompts, tools, context, and execution steps.
Azathoth
Build AI systems that can measure and improve themselves.
Azathoth is an open-source framework for constructing, evaluating, and optimizing multi-model AI systems across quality, reliability, latency, cost, and capability.
Why Azathoth exists
AI software has become too complex to tune by hand.
Modern AI systems are assembled from models, prompts, tools, context, retrieval, memory, evaluators, and multi-step workflows.
Every choice creates another dimension in the search space. Which model should run this step? Which prompt performs best? Which configuration meets the quality requirement without wasting latency or money?
Azathoth treats those questions as engineering problems that can be measured experimentally instead of answered by intuition.
The system
Construct. Evaluate. Optimize.
Execute
Run workflows through provider-neutral abstractions with step-scoped model and tool bindings.
Evaluate
Measure outcomes against explicit goals and collect repeatable evidence about quality, reliability, latency, and cost.
Optimize
Compare candidates and search for configurations that produce better system-level tradeoffs.
Architecture
Built for systems, not isolated prompts.
Provider-neutral
Models are capabilities, not architecture. Azathoth separates workflow behavior from individual model providers.
Goal-directed
Evaluation begins with explicit goals so optimization has a definition of better.
Workflow-native
AI applications are multi-step software systems. Azathoth represents and evaluates them that way.
Evidence-driven
Decisions are based on measured runs, experiments, and scorecards rather than assumptions about which model or prompt should perform best.
Optimization dimensions
QualityDoes the system produce the right result?
ReliabilityDoes it keep producing acceptable results?
LatencyHow quickly can the workflow complete?
CostWhat does the system consume to produce the result?
CapabilityWhat can the system successfully accomplish?
Open source
Built in public. Designed to evolve.
Azathoth is being developed as open-source infrastructure for engineers building complex AI systems.
The project is designed around explicit architecture, strict typing, repeatable evaluation, and a test-driven development process.
Get started
Azathoth is coming to Python.
pip install azathothThe initial open-source release is coming soon.