cli/cli

Can another program use it through a documented, machine-readable surface?

• next: [cli] L4 The output format is specified by a JSON Schema. (no JSON Schema for the output)
• next: [library] L4 An API reference is generated from the code (autodoc, mkdocstrings, typedoc, …). (no generated API reference)

cli ladder · level 3

• ✓ L1 Invocable non-interactively: no prompts, or prompts guarded by a TTY check or a --yes/--no-input option. no interactive prompts
• ✓ L2 --help documents every option (at least 90% of options have help text). internal/ghcmd/cmd.go 5 matches
• ✓ L3 Machine-readable output (--json/--format) and documented exit statuses. script/build.go 1 match.github/skills/cli-exercise/references/examples.md 1 match
• ✗ L4 The output format is specified by a JSON Schema. no JSON Schema for the output

library ladder · level 3

• ✓ L1 An importable, packaged API exists. go.mod Go module with exported packages
• ✓ L2 The public API is delimited (__all__, an exports map, exported identifiers). go.mod exported identifiers
• ✓ L3 The public API is statically typed (py.typed, type declarations, a typed language). go.mod statically typed language
• ✗ L4 An API reference is generated from the code (autodoc, mkdocstrings, typedoc, …). no generated API reference

service ladder · level 4

• ✓ L1 A network surface exists: HTTP routes, an MCP server, or a standard protocol listener. api/client.go go-http: 27 route declarationsinternal/codespaces/connection/tunnels_api_server_mock.go WebSocket server
• ✓ L2 A contract exists: a checked-in OpenAPI/GraphQL/protobuf/AsyncAPI spec, a framework-generated spec, MCP tool schemas, or a standard application protocol that off-the-shelf clients speak (RESP, MQTT, gRPC, PostgreSQL wire, …). internal/codespaces/rpc/codespace/codespace_host_service.v1.proto protobuf, 2 operationsinternal/codespaces/rpc/jupyter/jupyter_server_host_service.v1.proto protobuf, 1 operations
• ✓ L3 A contract covers its transport: an HTTP contract covers at least 80% of declared routes (generated contracts do), or the application layer over a standard protocol (commands, topics, payloads) is specified in a protocol document or AsyncAPI. internal/codespaces/rpc/codespace/codespace_host_service.v1.proto protobuf, 2 operationsinternal/codespaces/rpc/jupyter/jupyter_server_host_service.v1.proto protobuf, 1 operations
• ✓ L4 A second surface: another transport (HTTP and a protocol, two protocols), an MCP server, a CLI with JSON output, or an SDK. api/client.go HTTPscript/build.go CLI JSON output

Can integrators rely on it not changing under them?

• next: L2 A changelog exists and marks breaking changes. (no changelog)

all ladder · level 1

• ✓ L1 Releases are versioned (semver tags or versioned changelog sections). 205 semver tags, latest v2.102.0
• ✗ L2 A changelog exists and marks breaking changes. no changelog
• ✗ L3 A deprecation mechanism is used in code (warnings, @deprecated, Deprecation/Sunset headers) or the API is versioned. no deprecation mechanism or API versioning
• ✓ L4 Across tagged releases, no OpenAPI operation, CLI option or __all__ symbol was removed without first being deprecated. 20 tagged releases compared

Can others react to state changes without polling?

• next: L3 Signed webhooks registrable through the API, or two mechanisms (webhooks, streams, a broker). (one delivery mechanism, or webhooks that are unsigned or not registrable through the API)

service ladder · level 2

• ✓ L1 Internal events exist (signals, queues, an outbox, emitters). internal/codespaces/connection/tunnels_api_server_mock.go 7 matches.github/extensions/terminal-mockup/extension.mjs 1 match
• ✓ L2 Webhooks sent by this system, or a streaming endpoint (SSE, WebSocket). internal/codespaces/connection/tunnels_api_server_mock.go 7 matches.github/extensions/terminal-mockup/extension.mjs 1 match
• ✗ L3 Signed webhooks registrable through the API, or two mechanisms (webhooks, streams, a broker). one delivery mechanism, or webhooks that are unsigned or not registrable through the API
• ✗ L4 Events are described by AsyncAPI or CloudEvents. no AsyncAPI or CloudEvents contract

Can a machine act with its own revocable, least-privilege identity?

• next: [service] L1 Programmatic authentication (bearer tokens, API keys). (no token auth)
• next: [cli] L4 Several profiles or scoped credentials per target. (no credential profiles)
• next: [library] L4 Several profiles or scoped credentials per target. (no credential profiles)

cli ladder · level 3

• ✓ L1 Credentials can be supplied non-interactively (environment variables). internal/ghcmd/cmd.go 2 matchesscript/build.go 1 match
• ✓ L2 Credentials never need to be on the command line, and their environment variables are documented. .github/skills/vhs-demo/SKILL.md 4 matches
• ✓ L3 No output or log statement includes a credential. no credential in print/log statements
• ✗ L4 Several profiles or scoped credentials per target. no credential profiles

library ladder · level 3

• ✓ L1 Credentials can be supplied non-interactively (environment variables). internal/ghcmd/cmd.go 2 matchesscript/build.go 1 match
• ✓ L2 The credential environment variables are documented. .github/skills/vhs-demo/SKILL.md 4 matches
• ✓ L3 No output or log statement includes a credential. no credential in print/log statements
• ✗ L4 Several profiles or scoped credentials per target. no credential profiles

service ladder · level 0

• ✗ L1 Programmatic authentication (bearer tokens, API keys). no token auth
• ✓ L2 Per-client, revocable credentials (managed API keys, service accounts) or OAuth2. internal/gh/gh.go 1 match
• ✓ L3 Credentials carry scopes or permissions. .github/extensions/terminal-mockup/extension.mjs 13 matchespkg/cmd/skills/list/list.go 12 matches
• ✓ L4 Standard provisioning or federation (OIDC client credentials, SCIM). pkg/cmd/attestation/verification/policy.go 1 matchpkg/cmd/attestation/verify/verify.go 1 match

Can its data get in and out, in bulk, in open formats, with an explicit schema?

Can an automated caller use it without guessing, and recover from failure?

• next: [service] L2 Field-level validation errors, and paginated lists. (no request validation)
• next: [cli] L4 --dry-run, limits for long or costly operations (timeouts, budgets), and agent docs. (no timeout/budget options)
• next: [library] L1 Errors are specific exception types, and library code never calls sys.exit. (no exception classes)

cli ladder · level 3

• ✓ L1 Never blocks on a prompt when stdin is not a TTY. no interactive prompts
• ✓ L2 Data on stdout, diagnostics on stderr, non-zero exit on failure. internal/codespaces/connection/tunnels_api_server_mock.go 9 matchesscript/api-host-gateway/gateway/main.go 5 matches
• ✓ L3 Documented exit statuses distinguish error classes, errors are JSON in JSON mode, and color is off when not a TTY. .github/skills/cli-exercise/references/examples.md 1 match.github/skills/cli-exercise/references/prerequisites.md 1 match
• ✗ L4 --dry-run, limits for long or costly operations (timeouts, budgets), and agent docs. no timeout/budget options

library ladder · level 0

• ✗ L1 Errors are specific exception types, and library code never calls sys.exit. no exception classes
• ✓ L2 Inputs are validated with messages naming the bad argument. .github/extensions/terminal-mockup/assets/app.js 5 matches.github/extensions/terminal-mockup/extension.mjs 2 matches
• ✗ L3 Timeouts and retries are configurable. no retry parameter
• ✗ L4 Agent docs, and examples that run as tests (doctests). examples don't run as tests

service ladder · level 1

• ✓ L1 Errors are machine-readable: consistent JSON over HTTP, or a standard protocol's native error replies (RESP errors, MQTT reason codes, gRPC status, SQLSTATE). pkg/cmd/api/api.go 2 matchespkg/httpmock/stub.go 1 match
• ✗ L2 Field-level validation errors, and paginated lists. no request validation
• ✗ L3 Typed errors (RFC 9457 over HTTP, or a standard protocol's typed error codes), idempotent writes (Idempotency-Key), and rate-limit signals. no typed errors
• ✗ L4 Dry-run or validate-only, conditional requests, and agent docs. no ETags

Can another system run and compose it without a human?

• next: [service] L2 Configured through environment variables, documented: a .env example, a config schema, or docs naming at least half of the variables the code reads. (the environment variables aren't documented)
• next: [cli] L3 Published to a public registry and run non-interactively in CI. (the command isn't run in CI)
• next: [library] L4 Optional features behind extras/features. (no optional extras)

cli ladder · level 2

• ✓ L1 Runnable from source (a declared entry point). .github/skills/cli-exercise/tool/main.go go maincmd/gen-docs/main.go go main
• ✓ L2 Installable as a package that provides the command. .github/skills/cli-exercise/tool/main.go go maincmd/gen-docs/main.go go main
• ✗ L3 Published to a public registry and run non-interactively in CI. the command isn't run in CI
• ✓ L4 Also shipped as a container image or a single binary. .goreleaser.yml

library ladder · level 3

• ✓ L1 Importable from source. go.mod Go module with exported packages
• ✓ L2 Installable as a package (a package manifest). go.mod package manifest
• ✓ L3 Published to a public registry, with at most 10 runtime dependencies. .github/workflows/deployment.yml 6 matches.github/workflows/dependabot-triage.lock.yml 1 match
• ✗ L4 Optional features behind extras/features. no optional extras

service ladder · level 1

• ✓ L1 Runnable from source with documented steps (a README with commands). README.md
• ✗ L2 Configured through environment variables, documented: a .env example, a config schema, or docs naming at least half of the variables the code reads. the environment variables aren't documented
• ✗ L3 A container image. no Dockerfile
• ✗ L4 Composition artifacts (compose, Helm, IaC). no compose/Helm/IaC

Can behaviour be added without modifying the code?

• next: L1 Internal seams: a registry, registration function or plugin manager. (no registry or registration function)

all ladder · level 0

• ✗ L1 Internal seams: a registry, registration function or plugin manager. no registry or registration function
• ✗ L2 The docs show how to add behaviour through those seams (even in-tree). no registration functions to document
• ✗ L3 Out-of-tree extensions are discovered (entry points, a plugin directory, configuration) without editing the package. extensions must be added inside the package
• ✗ L4 The extension API is versioned, has lifecycle hooks, and has example extensions. no lifecycle hooks

Can an operator, or an orchestrating agent, tell whether it works and why not?

• next: [service] L1 A health or readiness endpoint, or a container healthcheck. (no health endpoint or healthcheck)
• next: [cli] L2 Verbosity control for diagnostics (--quiet, --log-level, --debug). (no --quiet or log-level control)
• next: [library] L1 Logs through a standard logger namespace and never prints from library code. (no standard logger)

cli ladder · level 1

• ✓ L1 Meaningful exit statuses, and errors on stderr. internal/codespaces/connection/tunnels_api_server_mock.go 9 matchesscript/api-host-gateway/gateway/main.go 5 matches
• ✗ L2 Verbosity control for diagnostics (--quiet, --log-level, --debug). no --quiet or log-level control
• ✗ L3 Structured logs (e.g. --log-format json) or a machine-readable run summary. no structured diagnostics
• ✗ L4 Tracing hooks (OpenTelemetry). no tracing hooks

library ladder · level 0

• ✗ L1 Logs through a standard logger namespace and never prints from library code. no standard logger
• ✗ L2 Log levels are used meaningfully (at least two levels). fewer than two log levels used
• ✗ L3 Optional metrics or tracing hooks. no metrics or tracing hooks
• ✗ L4 OpenTelemetry instrumentation available. no OpenTelemetry instrumentation

service ladder · level 0

• ✗ L1 A health or readiness endpoint, or a container healthcheck. no health endpoint or healthcheck
• ✗ L2 Structured logs. no structured logging
• ✗ L3 Metrics. no metrics
• ✗ L4 Distributed tracing (OpenTelemetry). no tracing

Can an agent change it safely with limited context?

• next: L3 At least 90% of modules fit 32k tokens and no file exceeds 12k; test/source token ratio at least 0.3; at least 70% of the code is typed. (100% of modules fit 32k tokens; 2 files over 12k)

all ladder · level 2

• ✓ L1 A README and some tests. README.md.github/codeql/tests/unsanitized-response-to-terminal/hits_base64_after_json.go 437 test files
• ✓ L2 CI runs the tests; applications pin dependencies (libraries and zero-dependency projects are exempt). .github/workflows/acceptance.yml.github/workflows/agentics-maintenance.yml
• ✗ L3 At least 90% of modules fit 32k tokens and no file exceeds 12k; test/source token ratio at least 0.3; at least 70% of the code is typed. 100% of modules fit 32k tokens; 2 files over 12k
• ✓ L4 No import cycles, and agent docs (AGENTS.md, CLAUDE.md, llms.txt). AGENTS.mdCLAUDE.md