sirmmo/aiproxy

Can another program use it through a documented, machine-readable surface?

• next: L4 A second surface: another transport (HTTP and a protocol, two protocols), an MCP server, a CLI with JSON output, or an SDK. (only one programmatic surface)

service ladder · level 3

• ✓ L1 A network surface exists: HTTP routes, an MCP server, or a standard protocol listener. app/routes/admin.py fastapi/flask: 15 route declarations
• ✓ L2 A contract exists: a checked-in OpenAPI/GraphQL/protobuf/AsyncAPI spec, a framework-generated spec, MCP tool schemas, or a standard application protocol that off-the-shelf clients speak (RESP, MQTT, gRPC, PostgreSQL wire, …). app/main.py FastAPI (auto OpenAPI)
• ✓ L3 A contract covers its transport: an HTTP contract covers at least 80% of declared routes (generated contracts do), or the application layer over a standard protocol (commands, topics, payloads) is specified in a protocol document or AsyncAPI. app/main.py generated from code: FastAPI (auto OpenAPI)
• ✗ L4 A second surface: another transport (HTTP and a protocol, two protocols), an MCP server, a CLI with JSON output, or an SDK. only one programmatic surface

Can integrators rely on it not changing under them?

• next: L2 A changelog exists and marks breaking changes. (no changelog)

all ladder · level 1

• ✓ L1 Releases are versioned (semver tags or versioned changelog sections). 1 semver tags, latest v0.1.0
• ✗ L2 A changelog exists and marks breaking changes. no changelog
• ✓ L3 A deprecation mechanism is used in code (warnings, @deprecated, Deprecation/Sunset headers) or the API is versioned. app/routes/chat.py 2 matchesapp/main.py 1 match
• ✗ L4 Across tagged releases, no OpenAPI operation, CLI option or __all__ symbol was removed without first being deprecated. fewer than two tagged releases to compare

Can others react to state changes without polling?

• next: L3 Signed webhooks registrable through the API, or two mechanisms (webhooks, streams, a broker). (one delivery mechanism, or webhooks that are unsigned or not registrable through the API)

service ladder · level 2

• ✓ L1 Internal events exist (signals, queues, an outbox, emitters). app/routes/chat.py 1 match
• ✓ L2 Webhooks sent by this system, or a streaming endpoint (SSE, WebSocket). app/routes/chat.py 1 match
• ✗ L3 Signed webhooks registrable through the API, or two mechanisms (webhooks, streams, a broker). one delivery mechanism, or webhooks that are unsigned or not registrable through the API
• ✗ L4 Events are described by AsyncAPI or CloudEvents. no AsyncAPI or CloudEvents contract

Can a machine act with its own revocable, least-privilege identity?

• next: L2 Per-client, revocable credentials (managed API keys, service accounts) or OAuth2. (no managed keys or OAuth)

service ladder · level 1

• ✓ L1 Programmatic authentication (bearer tokens, API keys). app/auth.py 1 matchapp/backends/anthropic_backend.py 1 match
• ✗ L2 Per-client, revocable credentials (managed API keys, service accounts) or OAuth2. no managed keys or OAuth
• ✗ L3 Credentials carry scopes or permissions. no scopes or permissions
• ✗ L4 Standard provisioning or federation (OIDC client credentials, SCIM). no OIDC client credentials or SCIM

Can its data get in and out, in bulk, in open formats, with an explicit schema?

Can an automated caller use it without guessing, and recover from failure?

• next: L2 Field-level validation errors, and paginated lists. (no pagination)

service ladder · level 1

• ✓ L1 Errors are machine-readable: consistent JSON over HTTP, or a standard protocol's native error replies (RESP errors, MQTT reason codes, gRPC status, SQLSTATE). app/routes/admin.py 12 matchesapp/routes/chat.py 8 matches
• ✗ L2 Field-level validation errors, and paginated lists. no pagination
• ✗ L3 Typed errors (RFC 9457 over HTTP, or a standard protocol's typed error codes), idempotent writes (Idempotency-Key), and rate-limit signals. no typed errors
• ✗ L4 Dry-run or validate-only, conditional requests, and agent docs. no dry-run

Can another system run and compose it without a human?

service ladder · level 4

• ✓ L1 Runnable from source with documented steps (a README with commands). README.md
• ✓ L2 Configured through environment variables, documented: a .env example, a config schema, or docs naming at least half of the variables the code reads. evals/two_model_ontorag/judge.py 3 matchesapp/config.py 2 matches
• ✓ L3 A container image. Dockerfileevals/two_model_ontorag/Dockerfile
• ✓ L4 Composition artifacts (compose, Helm, IaC). docker-compose.yml

Can behaviour be added without modifying the code?

• next: L1 Internal seams: a registry, registration function or plugin manager. (no registry or registration function)

all ladder · level 0

• ✗ L1 Internal seams: a registry, registration function or plugin manager. no registry or registration function
• ✗ L2 The docs show how to add behaviour through those seams (even in-tree). no registration functions to document
• ✗ L3 Out-of-tree extensions are discovered (entry points, a plugin directory, configuration) without editing the package. extensions must be added inside the package
• ✗ L4 The extension API is versioned, has lifecycle hooks, and has example extensions. the extension API isn't versioned

Can an operator, or an orchestrating agent, tell whether it works and why not?

• next: L2 Structured logs. (no structured logging)

service ladder · level 1

• ✓ L1 A health or readiness endpoint, or a container healthcheck. app/main.py 1 matchDockerfile 1 match
• ✗ L2 Structured logs. no structured logging
• ✗ L3 Metrics. no metrics
• ✗ L4 Distributed tracing (OpenTelemetry). no tracing

Can an agent change it safely with limited context?

• next: L2 CI runs the tests; applications pin dependencies (libraries and zero-dependency projects are exempt). (no lockfile (applications should pin dependencies))

all ladder · level 1

• ✓ L1 A README and some tests. README.mdscripts/smoke_test.py 2 test files
• ✗ L2 CI runs the tests; applications pin dependencies (libraries and zero-dependency projects are exempt). no lockfile (applications should pin dependencies)
• ✗ L3 At least 90% of modules fit 32k tokens and no file exceeds 12k; test/source token ratio at least 0.3; at least 70% of the code is typed. test/source token ratio 0.21 < 0.3
• ✗ L4 No import cycles, and agent docs (AGENTS.md, CLAUDE.md, llms.txt). no agent docs (AGENTS.md, CLAUDE.md, llms.txt)