Building Tool Systems for LLM Agents

This guide covers the architectural patterns for implementing tool systems that enable LLMs to interact with external systems. We draw from both Claude Code and Codex implementations to present a comprehensive approach.

Core Concepts

What is a Tool?

In LLM agent systems, a tool is:

1. A capability the agent can invoke (read files, run commands, search code)
2. A schema defining the tool's parameters
3. A handler that executes the tool and returns results
4. A response format the model can interpret

Tool Lifecycle

┌─────────────────────────────────────────────────────────────┐
│                     TOOL LIFECYCLE                          │
├─────────────────────────────────────────────────────────────┤
│                                                             │
│  1. REGISTRATION                                            │
│     ┌──────────┐   ┌──────────┐   ┌──────────────┐         │
│     │  Schema  │ + │ Handler  │ → │  Registry    │         │
│     │  (JSON)  │   │ (Code)   │   │  (name→impl) │         │
│     └──────────┘   └──────────┘   └──────────────┘         │
│                                                             │
│  2. INVOCATION                                              │
│     ┌──────────┐   ┌──────────┐   ┌──────────────┐         │
│     │  Model   │ → │  Router  │ → │  Handler     │         │
│     │ Decision │   │  Lookup  │   │  Execution   │         │
│     └──────────┘   └──────────┘   └──────────────┘         │
│                                                             │
│  3. RESULT                                                  │
│     ┌──────────┐   ┌──────────┐   ┌──────────────┐         │
│     │  Output  │ → │  Format  │ → │  To Model    │         │
│     │  (raw)   │   │ (struct) │   │  (context)   │         │
│     └──────────┘   └──────────┘   └──────────────┘         │
│                                                             │
└─────────────────────────────────────────────────────────────┘

Tool Schema Design

JSON Schema for Tool Parameters

Both systems use JSON Schema to define tool parameters. This enables:

• LLM understanding of parameter types and constraints
• Validation of tool invocations
• Auto-generation of documentation

Example: File Read Tool

{
  "name": "read_file",
  "description": "Reads a local file with line numbers",
  "parameters": {
    "type": "object",
    "properties": {
      "file_path": {
        "type": "string",
        "description": "Absolute path to the file"
      },
      "offset": {
        "type": "number",
        "description": "Starting line number (1-indexed)"
      },
      "limit": {
        "type": "number",
        "description": "Maximum lines to return"
      }
    },
    "required": ["file_path"],
    "additionalProperties": false
  }
}

Schema Design Best Practices

1. Clear descriptions: Each parameter should explain its purpose
2. Sensible defaults: Don't require parameters that have obvious defaults
3. Type safety: Use the most specific type (number vs string for IDs)
4. Validation rules: Use required, enum, minimum, maximum constraints
5. No additional properties: Set additionalProperties: false to catch errors

Codex's Schema Implementation

Codex uses a Rust enum to represent JSON Schema:

#[derive(Debug, Clone, Serialize, Deserialize)]
#[serde(tag = "type", rename_all = "lowercase")]
pub enum JsonSchema {
    Boolean { description: Option<String> },
    String { description: Option<String> },
    Number { description: Option<String> },
    Array {
        items: Box<JsonSchema>,
        description: Option<String>,
    },
    Object {
        properties: BTreeMap<String, JsonSchema>,
        required: Option<Vec<String>>,
        additional_properties: Option<AdditionalProperties>,
    },
}

This provides compile-time safety when defining tool schemas.

Tool Registry Pattern

The Registry Abstraction

A tool registry maps tool names to their implementations:

// TypeScript approach (Claude Code style)
interface Tool {
  name: string;
  schema: JSONSchema;
  handler: (args: Record<string, unknown>) => Promise<ToolResult>;
}

class ToolRegistry {
  private tools: Map<string, Tool> = new Map();

  register(tool: Tool): void {
    this.tools.set(tool.name, tool);
  }

  get(name: string): Tool | undefined {
    return this.tools.get(name);
  }

  async dispatch(name: string, args: unknown): Promise<ToolResult> {
    const tool = this.tools.get(name);
    if (!tool) {
      return { error: `Unknown tool: ${name}` };
    }
    return tool.handler(args as Record<string, unknown>);
  }
}

// Rust approach (Codex style)
pub struct ToolRegistry {
    handlers: HashMap<String, Arc<dyn ToolHandler>>,
}

impl ToolRegistry {
    pub fn new(handlers: HashMap<String, Arc<dyn ToolHandler>>) -> Self {
        Self { handlers }
    }

    pub fn handler(&self, name: &str) -> Option<Arc<dyn ToolHandler>> {
        self.handlers.get(name).map(Arc::clone)
    }

    pub async fn dispatch(
        &self,
        invocation: ToolInvocation,
    ) -> Result<ResponseInputItem, FunctionCallError> {
        let handler = self.handler(&invocation.tool_name)
            .ok_or(FunctionCallError::RespondToModel("unsupported tool"))?;
        
        handler.handle(invocation).await
    }
}

Builder Pattern for Registry Construction

Codex uses a builder pattern for clean registration:

pub struct ToolRegistryBuilder {
    handlers: HashMap<String, Arc<dyn ToolHandler>>,
    specs: Vec<ConfiguredToolSpec>,
}

impl ToolRegistryBuilder {
    pub fn new() -> Self { /* ... */ }

    pub fn push_spec(&mut self, spec: ToolSpec) {
        self.specs.push(ConfiguredToolSpec::new(spec, false));
    }

    pub fn register_handler(&mut self, name: impl Into<String>, handler: Arc<dyn ToolHandler>) {
        self.handlers.insert(name.into(), handler);
    }

    pub fn build(self) -> (Vec<ConfiguredToolSpec>, ToolRegistry) {
        (self.specs, ToolRegistry::new(self.handlers))
    }
}

// Usage
let mut builder = ToolRegistryBuilder::new();
builder.push_spec(create_shell_command_tool());
builder.register_handler("shell_command", Arc::new(ShellCommandHandler));
let (specs, registry) = builder.build();

Tool Handler Implementation

Handler Trait Pattern

Define a trait that all tool handlers implement:

#[async_trait]
pub trait ToolHandler: Send + Sync {
    /// What kind of tool this is (Function, MCP, etc.)
    fn kind(&self) -> ToolKind;

    /// Whether this handler can process the given payload type
    fn matches_kind(&self, payload: &ToolPayload) -> bool;

    /// Whether this tool mutates state (affects parallelism)
    fn is_mutating(&self, invocation: &ToolInvocation) -> bool {
        false
    }

    /// Execute the tool and return results
    async fn handle(&self, invocation: ToolInvocation) -> Result<ToolOutput, FunctionCallError>;
}

Concrete Handler Example

pub struct ReadFileHandler;

#[async_trait]
impl ToolHandler for ReadFileHandler {
    fn kind(&self) -> ToolKind {
        ToolKind::Function
    }

    fn matches_kind(&self, payload: &ToolPayload) -> bool {
        matches!(payload, ToolPayload::Function { .. })
    }

    async fn handle(&self, invocation: ToolInvocation) -> Result<ToolOutput, FunctionCallError> {
        // Parse arguments
        let args: ReadFileArgs = serde_json::from_str(&invocation.payload.arguments())
            .map_err(|e| FunctionCallError::RespondToModel(format!("Invalid args: {e}")))?;

        // Execute tool logic
        let content = tokio::fs::read_to_string(&args.file_path)
            .await
            .map_err(|e| FunctionCallError::RespondToModel(format!("Read error: {e}")))?;

        // Return structured output
        Ok(ToolOutput::Function {
            content: format_with_line_numbers(&content, args.offset, args.limit),
            content_items: None,
            success: Some(true),
        })
    }
}

Tool Routing

Router Pattern

The router determines how to dispatch tool calls based on the request:

pub struct ToolRouter {
    registry: ToolRegistry,
    specs: Vec<ConfiguredToolSpec>,
}

impl ToolRouter {
    /// Build a ToolCall from a model response item
    pub async fn build_tool_call(
        session: &Session,
        item: ResponseItem,
    ) -> Result<Option<ToolCall>, FunctionCallError> {
        match item {
            ResponseItem::FunctionCall { name, arguments, call_id, .. } => {
                // Check if this is an MCP tool
                if let Some((server, tool)) = session.parse_mcp_tool_name(&name).await {
                    Ok(Some(ToolCall {
                        tool_name: name,
                        call_id,
                        payload: ToolPayload::Mcp { server, tool, raw_arguments: arguments },
                    }))
                } else {
                    Ok(Some(ToolCall {
                        tool_name: name,
                        call_id,
                        payload: ToolPayload::Function { arguments },
                    }))
                }
            }
            ResponseItem::LocalShellCall { id, call_id, action, .. } => {
                // Handle local shell invocations
                // ...
            }
            _ => Ok(None),
        }
    }

    /// Dispatch a tool call to the appropriate handler
    pub async fn dispatch_tool_call(
        &self,
        session: Arc<Session>,
        turn: Arc<TurnContext>,
        tracker: SharedTurnDiffTracker,
        call: ToolCall,
    ) -> Result<ResponseInputItem, FunctionCallError> {
        let invocation = ToolInvocation {
            session,
            turn,
            tracker,
            call_id: call.call_id,
            tool_name: call.tool_name,
            payload: call.payload,
        };

        self.registry.dispatch(invocation).await
    }
}

Payload Types

Different tool sources produce different payload types:

pub enum ToolPayload {
    // Standard function call from model
    Function { arguments: String },
    
    // Custom tool call format
    Custom { input: String },
    
    // Local shell execution
    LocalShell { params: ShellToolCallParams },
    
    // Unified exec (PTY-based)
    UnifiedExec { arguments: String },
    
    // MCP tool invocation
    Mcp {
        server: String,
        tool: String,
        raw_arguments: String,
    },
}

Tool Output Formatting

Output Structure

Tool outputs must be formatted for the model to understand:

pub enum ToolOutput {
    Function {
        content: String,           // Plain text output
        content_items: Option<Vec<ContentItem>>,  // Structured content
        success: Option<bool>,     // Execution status
    },
    Mcp {
        result: Result<CallToolResult, String>,
    },
}

impl ToolOutput {
    pub fn into_response(self, call_id: &str, payload: &ToolPayload) -> ResponseInputItem {
        match self {
            ToolOutput::Function { content, content_items, success } => {
                ResponseInputItem::FunctionCallOutput {
                    call_id: call_id.to_string(),
                    output: FunctionCallOutputPayload {
                        content,
                        content_items,
                        success,
                    },
                }
            }
            ToolOutput::Mcp { result } => {
                ResponseInputItem::McpToolCallOutput {
                    call_id: call_id.to_string(),
                    result,
                }
            }
        }
    }
}

Output Formatting Best Practices

1. Structured output: Include metadata (exit codes, duration) when relevant
2. Truncation: Large outputs should be truncated with clear indication
3. Error messages: Provide actionable error messages the model can work with
4. Success indicators: Let the model know if the operation succeeded

pub fn format_exec_output_for_model(exec_output: &ExecToolCallOutput) -> String {
    let duration_seconds = exec_output.duration.as_secs_f32();
    let formatted = truncate_text(&exec_output.output, MAX_OUTPUT_SIZE);
    
    let payload = ExecOutput {
        output: &formatted,
        metadata: ExecMetadata {
            exit_code: exec_output.exit_code,
            duration_seconds,
        },
    };
    
    serde_json::to_string(&payload).expect("serialize")
}

Configuration-Driven Tool Registration

Feature Flags

Codex uses feature flags to conditionally enable tools:

pub struct ToolsConfig {
    pub shell_type: ConfigShellToolType,
    pub apply_patch_tool_type: Option<ApplyPatchToolType>,
    pub web_search_request: bool,
    pub include_view_image_tool: bool,
    pub experimental_supported_tools: Vec<String>,
}

pub fn build_specs(config: &ToolsConfig, mcp_tools: Option<HashMap<String, mcp_types::Tool>>) -> ToolRegistryBuilder {
    let mut builder = ToolRegistryBuilder::new();

    // Conditionally register shell tool
    match &config.shell_type {
        ConfigShellToolType::Default => {
            builder.push_spec(create_shell_tool());
            builder.register_handler("shell", Arc::new(ShellHandler));
        }
        ConfigShellToolType::Disabled => {}
        // ...
    }

    // Conditionally register experimental tools
    if config.experimental_supported_tools.contains(&"grep_files".to_string()) {
        builder.push_spec(create_grep_files_tool());
        builder.register_handler("grep_files", Arc::new(GrepFilesHandler));
    }

    // Register MCP tools
    if let Some(mcp_tools) = mcp_tools {
        for (name, tool) in mcp_tools {
            let converted = mcp_tool_to_openai_tool(name.clone(), tool);
            builder.push_spec(ToolSpec::Function(converted));
            builder.register_handler(name, Arc::new(McpHandler));
        }
    }

    builder
}

Model-Specific Tool Configuration

Different models may support different tools:

pub struct ModelFamily {
    pub shell_type: ConfigShellToolType,
    pub apply_patch_tool_type: Option<ApplyPatchToolType>,
    pub experimental_supported_tools: Vec<String>,
}

impl ToolsConfig {
    pub fn new(params: &ToolsConfigParams) -> Self {
        let model_family = params.model_family;
        let features = params.features;
        
        Self {
            shell_type: if features.enabled(Feature::UnifiedExec) {
                ConfigShellToolType::UnifiedExec
            } else {
                model_family.shell_type
            },
            // ...
        }
    }
}

Claude Code's Declarative Approach

Command-Based Tools

Claude Code defines tools through Markdown commands:

---
description: Search codebase for patterns
allowed-tools: [Grep, Read]
argument-hint: [pattern] [path]
---

Search for "$1" in $2 and summarize the findings.
Include:
- File locations
- Context around matches
- Frequency analysis

Agent-Based Tools

Autonomous agents act as complex, multi-step tools:

---
name: code-reviewer
description: Use when user asks for code review
model: inherit
color: blue
tools: ["Read", "Grep", "Glob"]
---

You are a code review specialist.

**Process:**
1. Analyze code structure
2. Check for common issues
3. Review style consistency
4. Identify potential bugs

**Output:**
Provide structured feedback with severity levels.

Summary: Building Your Own Tool System

1. Define tool schemas using JSON Schema for clear parameter contracts
2. Implement a registry to map tool names to handlers
3. Create handler abstractions (traits/interfaces) for consistent implementation
4. Build a router to dispatch calls based on tool type and source
5. Format outputs consistently for model consumption
6. Use configuration to enable/disable tools based on context
7. Consider extensibility (compile-time vs runtime) based on your needs

The next guides cover specific aspects in more detail: MCP integration, sandboxing/safety, and parallel execution.