Skip to content

Getting started

Prerequisite

hybrid-rag-platform's own Docker Compose stack must already be running and reachable (default assumed at http://host.docker.internal:8000) if you want to exercise /v1/rag/query. This project does not start, vendor, or manage that stack — it's a separate repo with its own lifecycle. /v1/agent/query and /v1/chat/stream don't need it.

Quickstart

# 1. Bring up this project's own stack: gateway + its own Postgres (port 5434) + Jaeger.
docker compose -f docker/docker-compose.yml up --build

# 2. Provision a tenant and API key (prints the raw key exactly once — it is never stored).
.venv/bin/gateway-admin create-tenant --name demo
# Tenant ID: <tenant-id>
# API key: <raw-key>

# 3. (Optional, only for the RAG route) point that tenant at a tenant/key already provisioned
#    in hybrid-rag-platform itself:
.venv/bin/gateway-admin set-rag-credential \
  --tenant-id <tenant-id> --rag-tenant-id <rag-tenant-id> --rag-api-key <rag-key>

ANTHROPIC_API_KEY must be set in the environment for /v1/agent/query and /v1/chat/stream to return real answers (neither was exercised with a real key in this build environment — see eval/README.md).

API examples

# RAG proxy — needs a stored rag_credentials row for the tenant.
curl -X POST localhost:8080/v1/rag/query \
  -H "X-Api-Key: <key>" -H "Content-Type: application/json" \
  -d '{"query": "What tables track order history?"}'
# 200 {"answer": {...StructuredAnswer...}, "usage": {"input_tokens": ..., "output_tokens": ...}}

# In-process agent
curl -X POST localhost:8080/v1/agent/query \
  -H "X-Api-Key: <key>" -H "Content-Type: application/json" \
  -d '{"question": "How many orders shipped last week?"}'
# 200 {"summary": ..., "chart": ..., "sql_log": [...], "hop_count": ..., "failed": false, "n_queries": ...}

# Direct Claude passthrough — the one streaming route.
curl -N -X POST localhost:8080/v1/chat/stream \
  -H "X-Api-Key: <key>" -H "Content-Type: application/json" \
  -d '{"message": "Say hello in one sentence."}'
# event: token  data: {"text": "..."}         (repeated)
# event: done   data: {"input_tokens": ..., "output_tokens": ...}
# event: error  data: {"message": "..."}      (only on a mid-stream failure)

curl localhost:8080/health   # liveness — always {"status": "ok"}
curl localhost:8080/ready    # readiness — 200 {"status": "ready"} iff gateway's own Postgres answers

All three routes require 429 handling: once a tenant's rate-limit bucket or daily token budget is exhausted, the gateway returns 429 with either a Retry-After (rate limit) or X-RateLimit-Reset (token budget) header — never a bare 429.

AGENT_DATABASE_URL — required for /v1/agent/query, and a security warning

/v1/agent/query depends on db_action_agent reading AGENT_DATABASE_URL directly from the process environment (it is imported and run in-process). This variable is not one of the other DATABASE_URL/RAG_SERVICE_URL/etc. variables already listed in docker/docker-compose.yml; it must be set separately, and docker-compose.yml fails loudly at startup (AGENT_DATABASE_URL variable is not set) if it's missing, rather than silently defaulting to something dangerous.

Point it at db-action-agent's own Postgres stack (that project's own docker-compose.yml, default port 5433, using its read-only agent_ro role) — for example:

export AGENT_DATABASE_URL="postgresql://agent_ro:<password>@host.docker.internal:5433/pagila"

Never point it at this gateway's own DATABASE_URL/Postgres (port 5434, database ai_gateway). That database stores rag_credentials.rag_api_key — every tenant's RAG API key — in plaintext (see Known limitations). /v1/agent/query lets a caller's prompt drive arbitrary SQL generation against whatever database AGENT_DATABASE_URL points at; pointing it at the gateway's own database would let a prompted LLM query read other tenants' RAG credentials across the whole gateway. This project's own Postgres stack (port 5434) and db-action-agent's (port 5433) must stay two separate databases/roles.

CI status — the cross-repo PAT gap (resolved 2026-09-24)

pyproject.toml pins db-action-agent @ git+https://github.com/shaifulshabuj/db-action-agent.git@v0.1.0 as a hard runtime dependency of ai_gateway itself — a private repo. Locally this resolves fine because this machine already has ambient git/gh credentials with read access.

Both test and container jobs need it. Tests avoid calling the real Anthropic API by injecting a fake ClaudeClient at the same seam db-action-agent's own suite uses, but that fake subclasses the real ClaudeClient — so even the test job's pip install has to fetch the real package. The container job needs it too, during the Docker build's own pip install step. The owner has added DB_ACTION_AGENT_PAT as a repo secret (a fine-grained PAT, read-only, scoped to db-action-agent alone); docker/Dockerfile's builder stage now installs git and consumes it via a BuildKit RUN --mount=type=secret=… step, so the token is available only inside that one step and never lands in an image layer or docker history.

Known limitations

Pulled directly from the design spec's "Out of scope for v0.1.0":

  • Flat rate limiting. All three routes cost the same one rate-limit-bucket token per request, regardless of how expensive the underlying call actually is.
  • Plaintext RAG credential storage. rag_credentials.rag_api_key is stored unencrypted in the gateway's own Postgres (see docs/adr/0002-*.md for why it can't just be hashed like api_keys.key_hash). This is also why AGENT_DATABASE_URL above must never point at this same database.
  • No per-hop agent tracing. agent.run's OTel span covers the whole multi-hop db-action-agent run as one span; individual ReAct hops are not visible as child spans.
  • No RAG-internal tracing. hybrid-rag-platform's retrieval/rerank/generation stages are not visible as child spans of rag.proxy_call — that service has no OTel instrumentation of its own.
  • No document-ingestion proxying and no self-serve tenant/key signup — provisioning is CLI-only (gateway-admin), matching Projects 1 and 2's own scope calls.
  • No token accounting for a stream disconnected before done. If the client disconnects (or the stream fails) before DoneEvent ever arrives, the tokens Anthropic already billed for that partial response are not recorded — stream_chat doesn't expose partial usage counts.

Development

.venv/bin/ruff check .
.venv/bin/pytest tests/unit tests/integration -v

Docs build with MkDocs Material (mkdocs build --strict); deployed on push to main via .github/workflows/docs.yml.