Welcome to another edition of GitHub All-Stars, where we look at a key building block of agentic applications - UI- and how Human-in-the-Loop can be implemented in practice. Let’s be honest: despite the marketing, no agent solution is perfect, and there will always be moments when we, the protein-based organisms, are needed. That’s why any application that aims to solve the problem realistically has to face this challenge head-on.
Filling the Gap Between the Agent and the User
“Why”: The AI Agent Era and the “Last Mile” Problem
In recent years we’ve seen an explosion in AI, driven by powerful agent frameworks like LangGraph and CrewAI (which we’re testing in parallel with LangGraph and preparing a write-up on). These tools have changed how we think about automation, enabling advanced AI “brains” that can orchestrate complex back-office tasks. However, despite their ability to reason and act, these frameworks often leave developers with a fundamental challenge: how do you integrate these powerful, autonomous systems with the everyday applications your users actually touch - in an elegant, interactive way? We can’t just send users a prompt in Markdown or a blob of raw JSON.
This is what we define as the agentic last-mile problem. It’s the gap between a powerful yet invisible AI agent and the application’s graphical interface that gatekeeps access to real user workflows. How do we make an agent not only run tasks in the background, but also “see” the application’s real-time context and “touch” its elements in response to user commands? How do we build a smooth, bidirectional link between the AI “brain” and the user’s “eyes and hands” inside the app - our Human-in-the-Loop?
The Solution: CopilotKit
Enter CopilotKit, a project created by folks from the LangChain ecosystem to solve this very problem. Its mission is to give developers a complete toolkit—UI components plus solid infrastructure—for building deeply integrated AI copilots (and yes, “copilot” has basically become the standard term). These are not just external chatbots, but integral parts of your application that can perceive its context and act on the user’s behalf. The key value props: easy integration (even if you haven’t “eaten glass” on CSS), independence from any one agent framework, and production-ready UI components. Think of it as a framework for integrating agents. While LangGraph defines the agent’s logic and workflow (how it operates), CopilotKit provides the interface through which that workflow interacts with humans in the context of a specific app.
This positions CopilotKit as a crucial element of our Agentic Application Stack, filling a critical - often overlooked - niche: the last mile of front-end integration where real users live.
The Bird’s-Eye View: A Tale of Two Stacks (and the Protocol That Connects Them)
Key Components - High-Level Breakdown
CopilotKit’s architecture falls into three cooperating pillars:
- Frontend (TypeScript/React): Built mainly around two packages:
@copilotkit/react-core(logic + hooks) and@copilotkit/react-ui(ready-made UI components). Here, you define what the AI agent can “see” and “do” inside your app. - Backend Runtime (
@copilotkit/runtime): The server-side middle layer between the frontend, LLMs, and backend agents. You can self-host it or use the managed Copilot Cloud (because, of course, everyone has a cloud now).
- Backend SDK (CopilotKit for Python): A bridge to the massive Python AI ecosystem. It makes it easy to “expose” Python-based agents (e.g., LangGraph) to the CopilotKit Runtime.
The Cornerstone: The AG-UI Protocol
At the heart of CopilotKit is the AG-UI (Agent–User Interaction) protocol - a communication standard that provides a unified interface between any AI agent and the UI. This is what makes CopilotKit genuinely framework-agnostic.
AG-UI enables a clean, extensible “bring any compatible agent” architecture. It defines the contract for how an agent should stream its state, task progress, and requests for user interaction—and how the UI should render and respond. As a result, the frontend (@copilotkit/react-core) doesn’t need to care whether it’s talking to a LangGraph or a CrewAI agent; it only needs to understand AG-UI. Conversely, any agent framework that implements an adapter becomes compatible with CopilotKit’s frontend ecosystem. This separation future-proofs the project and lets backend agents and frontend interfaces evolve independently while staying interoperable.
Frontend Architecture: Building Interactive AI with React
The Provider Pattern: Centralized AI Context
The analysis starts with the <CopilotKit> component, a classic Context API provider. It wraps your app (or a section of it) to supply a centralized context for all AI interactions—managing state, API keys, the runtime URL, and, crucially, the registration of all actions and data the agent can read. Setup is simple:
Backend vs. Frontend Actions: A Key Design Pattern
CopilotKit introduces a clear split between frontend-defined actions (via useCopilotAction) and backend-defined actions—a crucial separation of concerns for security and performance.
- Frontend actions: Manipulate UI/state—highlight items, open modals, change sort order, etc.
- Backend actions: Perform secure/compute-heavy operations—DB queries, calling protected third-party APIs, file processing, complex calculations.
Hook-Based API: Declarative AI Interactions
CopilotKit’s frontend power lies in its hook-based API—the main interface developers use to implement core patterns.
useCopilotReadable - Giving the AI “Eyes”
useCopilotReadable feeds real-time application context to the agent. It serializes the provided value with a description and sends it to the runtime, where it’s injected into the model’s system prompt. This builds the agent’s situational awareness. Instead of vague asks, a user can say “sort this data” or “delete the first item,” and the agent will know which data and which list.
useCopilotAction - Giving the AI “Hands”
This is the heart of interactivity. useCopilotAction defines functions the agent can call in response to user commands.
Under the hood, the hook takes a config with name, description, and parameters (with types and descriptions). CopilotKit turns this into a JSON schema compatible with LLM “functions/tools” (e.g., OpenAI). The schema is sent to the LLM; the descriptions guide when and how to call the function. When the LLM decides to call it, the runtime routes the call back to the frontend, which executes your handler with the model-supplied args.
useCopilotChat - The Engine Behind Headless UI
If you want full control over look & feel, CopilotKit provides ready UI components like <CopilotChat> and is <CopilotPopup>, built atop useCopilotChat. This headless hook exposes raw state (messages, input text, loading status) plus methods (appendMessage, reload, etc.) so you can build a fully custom chat UI from scratch.
UI Components: Production-Ready Building Blocks
As noted, CopilotKit ships ready-made components—<CopilotChat>, <CopilotSidebar>, <CopilotPopup>—that are deeply integrated with the core. They manage state, stream agent responses, and render dynamic, generative UI elements (more on that below).
Here’s a quick summary of the core React hooks:
| Hook name | Primary purpose | Example use case |
| useCopilotReadable | Provide read-only app context to the agent | Inform the AI about the user’s current to-do list contents |
| useCopilotAction | Define a typed function the agent can execute | Let the AI add/remove items from a to-do list |
| useCopilotChat | Headless chat state & methods for custom UIs | Build a bespoke chat interface from scratch |
| useCoAgent | Manage state & interaction with a backend “CoAgent” | Run a multi-step research agent and track its progress |
| useCoAgentStateRender | Render UI based on the CoAgent’s current backend state | Show a progress bar while the research agent is working |
Backend Architecture: Runtime and the Python SDK
@copilotkit/runtime - The Central Nervous System
On the backend, @copilotkit/runtime is the secure server-side proxy and central communication point. It handles:
- Credential management: Store API keys (e.g.,
OPENAI_API_KEY) safely on the server—never expose them to the client. - LLM communication: Abstract calls to different model providers via adapters (e.g.,
OpenAIAdapter,GroqAdapter). - Action orchestration: Route function calls—back to the frontend for UI actions, or to backend logic for secure operations.
- Streaming: Manage connections (e.g., SSE) to stream responses to the client for a real-time feel.
A typical Next.js App Router endpoint is concise:
CopilotKit leans into the full-stack model popularized by Vercel/Next.js: runtime as a serverless function under /api, frontend in React. This is instantly familiar to modern web devs, making “add AI to your app” feel like a natural extension of standard web development—not a detour into MLOps.
The Python copilotkit SDK bridges JavaScript and the “native” AI ecosystem. It helps you expose agents (often built with LangGraph) via FastAPI endpoints that speak AG-UI, letting the frontend interact seamlessly with Python-authored agents.
Advanced Patterns and Techniques
Generative UI: More Than Text
One of CopilotKit’s most impressive features is the ability for an agent to render custom React components directly in the chat. You define an action with a render property; when the agent calls it, the frontend renders the specified component, passing the LLM-provided args. You can go far beyond Markdown—charts, forms, interactive maps, etc.
Image: Example UI for document generation powered by CopilotKit
Human-in-the-Loop: Building Trust and Control
For production use, control and trust are critical. CopilotKit implements an explicit Human-in-the-Loop pattern via renderAndWait (or renderAndWaitForResponse) in useCopilotAction. The agent can complete part of a task, then pause and present results in a custom component (e.g., a confirmation dialog). It waits for explicit user consent before proceeding—essential for irreversible or high-impact actions like emailing customers or deleting data.
Shared State with useCoAgent: Real-Time Sync with the Agent
For advanced integrations, useCoAgent and useCoAgentStateRender let the frontend subscribe to a backend agent’s state in real time. As the agent moves through steps in its task graph (e.g., a LangGraph flow), it can emit state updates. With useCoAgentStateRender, the UI declaratively renders different components based on the agent’s current node/status (“fetching data,” “analyzing,” “writing summary”). This transparency shows users exactly what the AI is doing—vital for trust and comprehension.
Summary
As AI becomes a natural part of software, we need robust patterns for the last mile of integration. CopilotKit is a compelling addition to the stack. First, it showcases the power of a protocol-centric architecture (AG-UI) for separation and flexibility. Second, it underscores the wisdom of splitting frontend actions (UI manipulation) from backend actions (secure data ops). Third, it demonstrates the elegance of a hook-based, declarative API that abstracts complex asynchronous communication.
It’s also worth noting CopilotKit’s open-core model: choose between free, self-hosted @copilotkit/runtime (maximum control and privacy) and the managed Copilot Cloud (scalability and premium features like advanced guardrails, analytics, and auth management), however we decided to self host.
A well-deserved star on GitHub from me 😉
