Context-Stitcher 🪡
Zero-Copy Context Bridging Gateway for Multi-Agent GPU Inference.
💡 Core Value Proposition
In multi-agent collaborative workflows, separate agents often process the same long text context sequentially. For example:
Agent A (Legal Auditor): Reads a 200-page contract and runs compliance analyses (populating the GPU KV Cache).
Agent B (Financial Compliance): Reads the same 200-page contract and audits financial liabilities.
Under standard inference engines, Agent B is forced to repeat the expensive prefill phase, duplicate GPU activations, and suffer from high Time-to-First-Token (TTFT) latency.
Context-Stitcher solves this by bridging caches at the memory level:
Context Topological Hashing: Segmenting prompts into physical block-sizes and mapping them to cryptographic fingerprints (Merkle-chains).
Zero-Copy Block Stitching: Bypassing prefill for matched prefixes by mapping the logical attention table of Agent B directly to the physical GPU memory address of Agent A's cache blocks.
Zero-Trust Secure Gate: Enforcing boundary control lists so unauthorized agent sessions cannot access shared physical blocks.
📊 Performance Profiles
Below is the benchmark analysis of Context-Stitcher compared to standard vLLM cold-prefills when executing consecutive agents over a shared 200-page document:
⚙️ Installation & Quick Start
1. Install Dependencies
2. Launch the Gateway & Dashboard
Once booted, the gateway routes are active on http://localhost:8000:
API Proxy Gateway: http://localhost:8000/v1/chat/completions
Real-time Developer Console: Open http://localhost:8000 in your web browser.
🖥️ Visual Developer Portal
Context-Stitcher includes a responsive developer portal to monitor physical cache block states (idle, private allocations, shared/stitched pages, security alarms) in real time.
🛠️ Client Integration & Usage Guide
Context-Stitcher supports Python SDK Decorators and OpenAI-Compatible REST APIs for cross-application integrations:
Pattern A: Python SDK Decorator (For Python-based Agent Orchestration)
If your agent pipelines are written in Python, you can utilize the StitcherMesh and @stitch_agent decorators to link context memory:
Pattern B: OpenAI-Compatible REST API (For multi-language clients, Dify, or Flowise)
The gateway exposes standard OpenAI endpoints. Point your LLM client base URL to Context-Stitcher to activate sharing.
1. Python OpenAI SDK Client:
2. Raw HTTP cURL Request:
Pattern C: Controlling Security Rules via API (Zero-Trust Gate)
You can inspect, add, or revoke access authorization rules dynamically between agents.