Documentation

Step 1: Sign up

Go to /login and sign in with Google or GitHub. The Dev tier is free forever.

Step 2: Create an API key

Go to Dashboard → API Keys → Create Key. Copy the sk-... key.

⚠️ The key is shown only once. Store it securely.

Step 3: Send your first request

curl -X POST https://plumb.samji.in/v1/chat/completions \
  -H "Content-Type: application/json" \
  -H "Authorization: Bearer YOUR_LLM_PROVIDER_KEY" \
  -H "x-proxy-key: sk-YOUR-PROXY-KEY" \
  -H "x-upstream: groq" \
  -d '{
    "model": "llama-3.1-8b-instant",
    "messages": [{"role": "user", "content": "Hello!"}]
  }'

Step 4: Check your dashboard

Go to your dashboard — you'll see your request, token usage, cost, and cache status in real-time.

4-Layer Token Attribution

Every request is broken down into 4 cost layers:

• • Prompt tokens — your user messages
• • Tool tokens — function/tool definitions in the request
• • Memory tokens — system messages (RAG context, instructions)
• • Response tokens — model output (4x more expensive on most providers)

💡 Tip: Move large tool schemas to a shared system prompt to reduce per-request tool token costs.

Semantic Caching

Identical requests are served from cache instantly, saving LLM costs. Cache is per-tenant (no cross-tenant leaks).

• • Cache TTL: 15 minutes (configurable)
• • Cache key: blake3 hash of normalized request intent
• • Cache hit response includes x-cache: HIT header

💡 Tip: Agents that retry the same question benefit massively. 30%+ cache hit rates are common on repetitive agent workloads.

PII Redaction

Sensitive data is automatically masked before reaching the LLM provider:

• • SSNs (123-45-6789) → [PII_SSN_0001]
• • Emails → [PII_EMAIL_0002]
• • Credit cards → [PII_CC_0003]
• • Phone numbers → [PII_PHONE_0004]

💡 Masking is reversible — authorized users can reveal original values via the vault API.

Budget Caps & Rate Limiting

Prevent runaway agents from destroying your budget:

• • Daily budget cap — set a $ limit per tenant per day. Requests are rejected (429) when exceeded.
• • RPM rate limit — sliding window per-tenant (Free: 30, Growth: 200, Enterprise: 1000)
• • Token quota — daily token limit as a safety net

💡 Tip: Set conservative budget caps initially. You can always increase them from the dashboard.

RiskChain Detection

Detects multi-step attack patterns that look benign individually:

• • Read sensitive file → encode → POST to external URL = data exfiltration
• • Shell deobfuscation (hex-encoded commands decoded before analysis)
• • Session-aware tracking via x-session-id header

💡 Tip: Always send x-session-id for multi-turn agent conversations to enable RiskChain.

Circuit Breaker & Failover

If your primary provider goes down, requests auto-route to a backup:

• • 3 consecutive 5xx errors → circuit opens for 30 seconds
• • Automatic failover to next available provider
• • After 30s, one test request is sent to check recovery
• • On success → circuit closes, normal routing resumes

Context Window Protection

Requests that exceed a model's context window are blocked before reaching the provider:

• • GPT-4o: 128,000 tokens
• • Claude 3.5: 200,000 tokens
• • Llama 3.1: 131,072 tokens

💡 Saves you from paying for requests that would fail anyway.

Agent Loop-Breaker

Detects when a session re-issues the same request intent repeatedly and stops it before it burns budget:

• • Uses the same semantic intent fingerprint as the L1 cache (blake3 hash of messages + tools)
• • If the same fingerprint appears more than LOOP_MAX_REPEATS times (default 5) within LOOP_WINDOW_SECS (default 30s), the request is rejected
• • Returns 429 with code agent_loop_detected
• • Tracked per x-session-id (so separate conversations don't interfere)

💡 Set LOOP_MAX_REPEATS=0 to disable. Tune LOOP_WINDOW_SECS higher for long-running agent sessions.

Prompt-Injection & Tool-Poisoning Defense

Scans both message content and tool definitions for adversarial patterns:

• • Instruction override — "ignore previous instructions", system prompt extraction attempts
• • Tool-description poisoning — hidden imperatives, sensitive-resource references, encoded blobs in MCP/OpenAI tool schemas
• • Unicode-tag smuggling — invisible characters (U+E0000–U+E007F) used to hide instructions
• • High-confidence attacks → 403 prompt_injection_blocked
• • Lower-confidence signals → flagged in the activity feed (no block)

💡 This protects you from untrusted MCP servers and malicious RAG content injecting instructions into your agent's context.

Declarative Policy Rules

A rule engine evaluates each request against configurable security policies:

• • Actions: block (reject 403), redact (mask matched content), alert (log + continue), log (silent record)
• • Default rules catch credential exfiltration, SQL injection, and prompt-injection patterns
• • Rules use precompiled regexes — zero allocation overhead on the hot path
• • List active rules via GET /api/rules

💡 Blocked requests return 403 policy_blocked with the rule name in the response body.

Time-Travel Trace & Replay

Every request is traced end-to-end with a unique ID and can be replayed deterministically:

• • Every response includes an x-trace-id header
• • Traces record: governance steps, cache disposition, status, latency, token/cost outcome
• • GET /api/traces — list recent traces (admin-gated)
• • GET /api/traces/{id} — inspect a single trace
• • POST /api/traces/{id}/replay — replay the original request and diff the response (hash comparison)
• • Traces persist to SQLite so they survive restarts

💡 Use replay to verify that a policy change doesn't break working requests (regression testing for governance).

Enterprise Memory Fabric

A tenant-isolated, multi-tier memory API for stateful agent workflows:

• • Working memory — ephemeral session KV (auto-expires, scoped to a run)
• • Episodic memory — append-only event log (audit trail, learning from past actions)
• • Semantic memory — versioned facts with event-sourcing (CAS conflicts → 409)
• • Sensitivity labels + clearance-gated reads (confidential facts don't leak to lower clearance)
• • Full version history on semantic facts (GET /api/memory/semantic/{key}/history)
• • Deterministic keyword retrieval (no vector-search staleness)

💡 Use working memory for scratchpad, episodic for observability, semantic for distilled knowledge your agents share across sessions.

Cache-Aware Cost Accounting

Accurate spend tracking that accounts for provider prompt-cache discounts:

• • Parses OpenAI cached_tokens in usage responses
• • Parses Anthropic cache_read_input_tokens and cache_creation_input_tokens
• • Prices each token class at its true rate (cache-read ≈ 10% of list price)
• • Dashboard shows actual spend vs. naive spend (list-price × all tokens) and the savings delta

💡 This means your cost dashboards show real costs, not inflated estimates — critical for accurate FinOps.