# Token A **token** is the basic unit of text that large language models (LLMs) process. Instead of working character-by-character or word-by-word, LLMs operate on tokens — chunks of text that are determined by the model's tokenizer. ## What Is a Token? Tokens don't map cleanly to words or characters. They're determined by frequency patterns in training data: - Common words are often a single token (`the`, `agent`, `task`) - Rare words are split into multiple tokens (`tokenization` → `token` + `ization`) - Code and symbols often tokenize differently than prose - Roughly 1 token ≈ 4 characters ≈ 0.75 words in English ## Why Tokens Matter for Agents Tokens matter in three key ways: ### 1. Context Window Size Every LLM has a maximum [context window](context-window) measured in tokens. Claude's models support up to 200K tokens. Everything the model sees — system prompt, conversation history, tool results, injected data — counts against this limit. ### 2. API Cost Most LLM providers charge per token for both input (prompt) and output (completion). Long-running [agentic workflows](agentic-workflow) with many tool calls can accumulate significant token costs. ### 3. Latency More tokens to process = more time to generate a response. Agents with very long context windows will have higher latency than those with trimmed, focused prompts. ## Token Usage in AgentRQ AgentRQ helps minimize token waste by externalizing agent-human communication. Instead of maintaining long conversation histories inside the model's context, agents use `reply` and `getTaskMessages` [MCP tools](mcp-tool) to communicate, keeping the agent's context window lean. ## Token Estimation | Content | Approximate Tokens | |---------|------------------| | This glossary page | ~600 tokens | | A 500-line Go file | ~2,000 tokens | | A 10-page PDF | ~5,000 tokens | | Claude's max context | 200,000 tokens | ## Related Terms - [Context Window](context-window) - [Agent](agent) - [Claude Code](claude-code) - [Agentic Workflow](agentic-workflow)