Smart Agent with Memory

What You Will Build

You will turn the stateless streaming API from Phase 1 into a real AI agent. This agent reasons about a problem, decides which tools to call, runs them, and folds the results back into its answer. By the end of this phase, you will have a ReAct agent wired to four live tools (weather, calculator, Wikipedia, date/time). You will also have Redis-backed session memory that survives page refreshes and pod restarts, and an SSE stream that sends an event for every step of the reasoning loop, not just the final tokens.

Data Flow Visualization

Use the interactive diagram above to trace a full agent request: message in, history loaded from Redis, the ReAct loop with a tool call, the streamed answer out, and the turn saved back to Redis. Click any node to see the real data structure at that point. Press Animate Flow to step through all eight stages.

Why This Phase Exists

In Phase 1 the LLM was a pure function: message in, tokens out, no memory, no side effects. That model is easy to reason about. But it stops working the moment a user asks something the model cannot answer from its frozen training data ("what's the weather right now?") or expects it to remember ("what did I just tell you my name was?").

Phase 2 is where the LLM stops being a text generator and becomes an agent. Two abilities make that jump: it can act on the world through tools, and it can remember across turns through session memory. Almost every "AI product" that is more than a thin chat wrapper is built on these two abilities. Get them right here — the reasoning loop, the tool schemas, the memory trimming — and Phases 4 (multi-agent), 5 (production API), and 7 (observability) all extend this same skeleton instead of replacing it.

How the ReAct Loop Actually Works

Problem: an LLM on its own cannot tell you today's weather. It was trained months ago. It has no live data and no way to run code. If you just ask it, it will confidently make something up. This is the exact failure that makes people distrust AI.

ReAct (Reason + Act, Yao et al., 2022 — Google Research) fixes this by structuring the model's output into a loop instead of a single answer:

• ▸Thought — the agent reasons about what it knows and what it still needs to find out.
• ▸Action — it decides which tool to call, and with what arguments.
• ▸Observation — the tool runs and its result is fed back into the agent.
• ▸The loop repeats until the agent has enough to produce a Final Answer.

Here is a real trace for "What is the weather in Tokyo and should I bring an umbrella tomorrow?":

Thought: I need to find the current weather in Tokyo. I'll use the weather tool.
Action:  weather({ city: 'Tokyo' })
Observation: { temp: 18°C, condition: 'Partly cloudy', tomorrow: 'Rain 80%' }

Thought: I have the weather data. Tomorrow has 80% rain probability.
Final Answer: Tokyo is currently 18°C and partly cloudy. Tomorrow has an 80%
chance of rain — yes, bring an umbrella.

The gotcha that surprises people: this is not a special "agent mode" inside the model. ReAct is a prompt-engineering pattern. The loop lives in your code (LangGraph, here). Your code calls the same ordinary chat model again and again, appends the tool result to the conversation, and calls it again. The model never "runs" anything itself. It only ever produces text. Once you understand that the orchestration is yours, not the model's, you can debug agents when they misbehave.

Why LangChain + LangGraph for This Phase

You could build the ReAct loop by hand: parse the model's "Action:" line yourself, call the right function, stringify the result, re-prompt. People do this. Then they spend a week on edge cases such as malformed tool calls, infinite loops, and streaming intermediate steps.

LangGraph's createReactAgent gives you that loop as a tested state machine. This is where the abstraction you built in Phase 1 pays off:

• ▸In Phase 1 you wrapped Groq behind a service instead of calling the SDK directly from your route. Switching that for LangChain's ChatGroq is a one-file change. Your route handler barely moves.
• ▸LangGraph handles the loop mechanics (when to call a tool, when to stop, how to thread state), so your code only describes what the tools are and how to stream events out.
• ▸The same streamEvents API surfaces every internal step: tool starts, tool ends, token chunks. This is what lets you show the user "🔧 Using tool: get_weather" live instead of a frozen spinner.

The trade-off is honest: you take on a dependency and its abstractions. For a production agent loop, that is a good trade. You are not in the business of maintaining your own agent framework.

The Four Tools — What Each One Does

A tool is just a TypeScript function plus a JSON Schema that describes it. The schema is the whole trick. The LLM reads it to learn what the tool does and what arguments it takes, then produces a structured call. Phase 2 ships four:

Each tool returns a JSON string (not an object) because that string becomes the "Observation" text fed back to the model. Keep it small. The search_wikipedia tool slices the summary to about 1,500 characters on purpose, so a single lookup does not overflow the context window.

Tool Calling — How the LLM Actually Decides

Problem: how does the model know that "what's 25% of 4500?" should call calculate but "who are you?" should not? There is no if statement doing that routing.

The answer is the description field on each tool's schema. When you register a tool with DynamicStructuredTool, its name, description, and parameter schema are added to the prompt the model sees. So the model is not guessing. It matches your query against the descriptions you wrote:

 Copy
export const weatherTool = new DynamicStructuredTool({
  name: 'get_weather',
  description:
    'Get current weather and forecast for a city. Use when the user ' +
    'asks about weather, temperature, rain, or what to wear.',
  schema: z.object({
    city: z.string().describe('City name, e.g. Tokyo, New York, London'),
    units: z.enum(['celsius', 'fahrenheit']).default('celsius'),
  }),
  func: async ({ city, units }) => {
    /* fetch wttr.in, return JSON.stringify(...) */
  },
});

The gotcha most people learn the hard way: the description is prompt engineering, not documentation. A vague description ("gets weather") makes the model call the tool at the wrong times or miss it entirely. The phrase "Use when the user asks about..." is doing real work. It tells the model the trigger conditions. If your agent picks the wrong tool, the first thing to fix is almost never the code. It is the description.

Session Memory You Cannot Skip

Problem: HTTP is stateless. Without somewhere to store history, every message starts from zero. The agent cannot remember your name from one turn to the next, and the "conversation" is an illusion.

Phase 2 uses three layers of memory, and you need to know which is which:

The one you cannot skip is session memory in Redis. The reason is the same reason you will containerize in Phase 8 and scale in Phase 9: the moment you run more than one API instance behind a load balancer, in-process memory breaks. Request one lands on pod A, request two on pod B, and pod B has never heard of this user. Redis is a shared service both pods talk to, so the conversation survives a refresh, a restart, or a scale-out.

Context Window Management — The Trim Pattern

Problem: session memory that only ever grows is a slow-motion outage. A 200-turn conversation reloaded on every request means every request re-sends all 200 turns to the model. Latency climbs, cost climbs linearly, and eventually you go past the context window and the request hard-fails.

The Phase 2 defense is a hard cap applied on every write. This is the same fail-fast instinct behind Phase 1's env validation: enforce the rule at the boundary so nothing downstream has to worry about it.

 Copy
const SESSION_TTL = 60 * 60 * 24 * 7; // 7 days
const MAX_HISTORY = 50;               // keep only the most recent turns

async appendTurn(sessionId: string, turn: ConversationTurn): Promise<void> {
  const key = `session:${sessionId}:history`;
  const history = await this.getHistory(sessionId);
  history.push(turn);
  const trimmed = history.slice(-MAX_HISTORY); // drop the oldest
  await redis.setex(key, SESSION_TTL, JSON.stringify(trimmed));
}

slice(-MAX_HISTORY) keeps the newest 50 turns and drops the rest. setex re-stamps the 7-day TTL on every write, so active sessions stay alive and abandoned ones expire on their own. This is the crude version on purpose. A hard cutoff loses old context entirely. Phase 3 replaces it with a sliding window plus summarization (compress the old turns instead of deleting them). Phase 5 makes the cap per-tier (free vs pro vs enterprise). Phase 7 alerts when average context per user drifts up. Ship the hard cap now. You cannot skip having a limit.

Core Agent Loop — Code Deep Dive

This is the heart of the phase. It is an AsyncGenerator that loads memory, runs the ReAct loop, streams every intermediate step, and saves the turn. Note how it reuses the same streaming shape from Phase 1's streamChat, just with richer event types:

 Copy
export async function* runAgentStream(
  userMessage: string,
  sessionId: string,
): AsyncGenerator<AgentStreamEvent> {
  // 1. Load conversation history from Redis
  const history = await memoryService.getHistory(sessionId);

  // 2. Rebuild the LangChain message array from stored turns
  const messages = [
    new SystemMessage(SYSTEM_PROMPT),
    ...history.map((t) =>
      t.role === 'user' ? new HumanMessage(t.content) : new AIMessage(t.content),
    ),
    new HumanMessage(userMessage),
  ];

  const toolsUsed: string[] = [];
  let finalResponse = '';

  // 3. Stream the ReAct loop — one event per internal step
  const stream = await agent.streamEvents({ messages }, { version: 'v2' });
  for await (const event of stream) {
    if (event.event === 'on_tool_start') {
      toolsUsed.push(event.name);
      yield { type: 'tool_call', content: `Using tool: ${event.name}` };
    }
    if (event.event === 'on_chat_model_stream' && event.data?.chunk?.content) {
      const token = event.data.chunk.content;
      finalResponse += token;
      yield { type: 'token', content: token };
    }
  }

  // 4. Persist BOTH turns back to Redis
  await memoryService.appendTurn(sessionId, {
    role: 'user', content: userMessage, timestamp: Date.now(),
  });
  await memoryService.appendTurn(sessionId, {
    role: 'assistant', content: finalResponse, timestamp: Date.now(), toolsUsed,
  });

  yield { type: 'done', content: finalResponse, metadata: { toolsUsed } };
}

Let's walk the four steps. Load rehydrates the conversation from Redis so the model has context. Rebuild converts your stored turns into LangChain message objects (SystemMessage / HumanMessage / AIMessage). Stream is the ReAct loop: streamEvents emits on_tool_start when the agent decides to act and on_chat_model_stream for each answer token, which you forward as SSE events so the UI can show tool use in real time. Persist saves the user message and the assistant reply, with toolsUsed recorded for later observability. The generator shape is the same as Phase 1. The caller still does for await (const event of runAgentStream(...)). You have only upgraded what each event means.

Redis vs In-Memory vs Postgres vs Filesystem

The interview question hiding in this phase is: "Why Redis for session state — why not just a Map, or your database?" Here is the honest comparison:

Redis wins because session memory has an unusual profile. It is read and written on every single request, so it must be fast. It should expire on its own, so built-in TTL matters. And it must be shared the instant you run more than one instance. In-memory fails the sharing test. Postgres fails the latency test. The filesystem fails both. Redis is the right answer from here all the way through Phase 9.

Redis + PostgreSQL — Why Now, Not Later

Problem: adding stateful infrastructure to a running system is much more painful than declaring it from the start. Connection handling, health checks, and startup ordering all have to be retrofitted under load.

So Phase 2 adds both services to docker-compose.yml now, with health checks and startup dependencies wired in:

 Copy
services:
  api:
    depends_on:
      redis:    { condition: service_healthy }
      postgres: { condition: service_healthy }
  redis:
    image: redis:7-alpine
    healthcheck:
      test: ['CMD', 'redis-cli', 'ping']
      interval: 5s
      retries: 5
  postgres:
    image: postgres:16-alpine
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U agenttelar']
      interval: 5s
      retries: 5

Here is what depends_on with condition: service_healthy solves. Without it, your API boots faster than Redis and crashes on its first connection attempt. The health check makes Compose wait until redis-cli ping actually answers before starting the API. You use Redis right away for session memory. PostgreSQL is provisioned in the same Compose file but unused by Phase 2 app code — so you get comfortable with a DB in the stack before Phase 3. Phase 3 runs its own Compose project (agenttelar-phase3) with a pgvector-enabled image and a separate volume; it does not reuse Phase 2's Postgres data.

Test It — curl Commands + Expected Output

This is the verification gate. Run one phase stack at a time on your machine. Volumes are isolated per Compose project (agenttelar-phase2), but host ports are shared: Phase 1 holds :3000; Phase 2 also needs :6379 and :5432. From your phase2/ folder:

 Copy
# Required if Phase 1 is still running — frees shared host ports (volumes stay):
(cd ../phase1/docker && docker compose down)

cp apps/api/.env.example apps/api/.env
# Open apps/api/.env and paste your free Groq key from https://console.groq.com
# (Phase 2 has its own .env — it does not reuse Phase 1's)

cd docker && docker compose up --build -d

Once docker compose ps shows healthy services, run the curls below in order.

1. First message — capture the session ID:

 Copy
curl -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d '{"message": "My name is Alex and I am learning AI engineering"}' \
  --no-buffer -i
# Copy the sessionId from the X-Session-ID response header:
SESSION_ID=<paste-from-header>

2. Second message — memory must persist:

 Copy
curl -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d "{\"message\": \"What is my name?\", \"sessionId\": \"$SESSION_ID\"}" \
  --no-buffer
# Expected: the agent answers "Your name is Alex..." — proof Redis reload worked.

3. Tool use — watch the reasoning loop stream:

 Copy
curl -X POST http://localhost:3000/api/chat \
  -H 'Content-Type: application/json' \
  -d "{\"message\": \"What is the weather in Tokyo right now?\", \"sessionId\": \"$SESSION_ID\"}" \
  --no-buffer

Expected — a tool_call event, then token events, not one blob:

event: tool_call
data: {"content":"Using tool: get_weather","sessionId":"..."}
event: token
data: {"content":"Tokyo is currently 18°C..."}
event: done
data: {"content":"...","metadata":{"toolsUsed":["get_weather"]}}

4. Fresh session = no memory (the test people skip):

Send "What is my name?" with a brand-new random sessionId. The agent should have no idea. If it still says "Alex", your sessions are not isolated and you are leaking history across users.

5. History and clear:

 Copy
curl http://localhost:3000/api/chat/session/$SESSION_ID/history
# Expected: the full conversation plus an estimated token count

curl -X DELETE http://localhost:3000/api/chat/session/$SESSION_ID
# Then GET history again — the conversation should be gone

Try It Live

Run the actual phase_2 code in your browser. No local install needed. Once the editor loads, copy .env.example to .env and paste your Groq key. Session memory needs a real Redis; in the browser set REDIS_URL to a hosted instance (e.g. free Upstash), or use the local Docker path above for the full memory checks.

Cost Awareness — Numbers Table

Agents cost more than plain chat, and it helps to know why before the bill explains it for you. Two multipliers appear in Phase 2. Every tool call adds an extra LLM round-trip, because the model is called again after each observation. And reloaded history means you pay to re-send the whole conversation on every turn.

Completion Checklist — Technical

Before you consider Phase 2 done, every one of these should be true:

• ▸Sending "What is the weather in Paris?" streams a tool_call event followed by token events
• ▸"My name is X" then "What is my name?" in the same session — the agent remembers
• ▸The same "What is my name?" with a new sessionId — the agent has no memory of it
• ▸"25% of 4500" streams a tool_call for calculate and returns the correct number
• ▸GET /api/chat/session/:id/history returns the full conversation with an estimated token count
• ▸DELETE /api/chat/session/:id clears the session
• ▸docker compose up --build -d starts Redis + PostgreSQL + API with all three health checks passing

Conceptual Checklist — Can You Explain?

If you can't answer these out loud without checking your code, re-read the relevant section before moving on:

• ▸Trace the ReAct loop — what happens at Thought, Action, and Observation, and where does that loop actually live?
• ▸Why does the agent sometimes answer without calling any tool, and why is that correct?
• ▸What is the difference between session memory and long-term memory?
• ▸Why did abstracting the LLM client in Phase 1 make adopting LangChain a one-file change?
• ▸What happens when Redis goes down, and what degradation would you choose?
• ▸Why do you store toolsUsed on every turn saved to Redis?
• ▸What does slice(-MAX_HISTORY) protect you from — name both the failure it prevents and the cost it controls?

What You Will Learn

• ▸Implement the ReAct reasoning loop and understand that the orchestration is your code, not the model
• ▸Give the agent real tools with schemas the LLM uses to decide when to call them
• ▸Persist conversations across sessions and pods with Redis, with a clear trim strategy
• ▸Understand context-window management as both a reliability control and a cost control

Next Steps

Clone the phase2 folder on GitHub. Stop Phase 1 with docker compose down in phase1/docker (frees :3000), copy apps/api/.env.example to apps/api/.env, add your Groq key, then cd docker && docker compose up --build -d. Work through the curl tests until memory and tools both behave as shown. Complete the quiz before Phase 3. When you leave for Phase 3, stop this stack the same way so :3000, :6379, and :5432 are free.