Tag: langgraph
All the articles with the tag "langgraph".
-
Eval Study #4 — HITL Agent Regression Test, Passed but for the Wrong Reason
Following the previous post (agent tool selection · multi-step eval), this is a regression test for an HITL agent. This time, instead of a (question · expected tool) test set, I wrote unit-test style functions covering 3 axes of pitfalls (entering the breakpoint for dangerous tools / responding on every turn in multi-turn conversations / not misclassifying safe tools), with sys.exit(1) as the exit code on regression. The first run passed 3/3 — but that wasn't the right answer. The "call only after confirming clear intent" prompt that had been an issue before was still lingering, and the LLM was passing the test via its own self-defense mechanism. When I provoked it with a delete case, our HITL didn't trigger at all — only the LLM's own confirmation remained. Removing the prompt → the failure shifted to the tool simply not being called at all. Eventually, minimizing the prompt to "call the tool that matches the task the user requested" made it stop exactly at the delete_user tool (HITL working, exit code 0). Passing an eval by itself isn't a safety signal — verifying why it passed is the real safety. Pausing the LLM study series here for now, to resume after wrapping up backend studies.
-
MCP Study #3 — What Resources / Prompts Actually Are + Integration with LangGraph (`MultiServerMCPClient` · `ainvoke`)
Beyond Tools covered in #1 / #2, this post covers the other two components of MCP: Resources (data for the LLM to read, background context, read-only) and Prompts (predefined templates). I check a new server with Inspector + Claude Desktop → integrate the MCP server into LangGraph using langchain-mcp-adapters. Covers why `ainvoke` is needed since MCP communication is asynchronous, the secret behind how MultiServerMCPClient loads servers as-is (= MCP standard compliance), and the difference in domain response quality when injecting Resources as a system prompt.
-
LangGraph Study #3 — Human-in-the-Loop (Approving Risky Tools) · Checkpoints and the Interrupt Mechanism
When an agent automatically calls risky tools like file deletion, sending emails, or payments, things go wrong. Implementing the HITL pattern using LangGraph's checkpoint + Interrupt mechanism to get human approval right before a risky tool call. An unexpected trap: the LLM's own safety guard can block execution before it ever reaches HITL. How to separate the system prompt from the HITL gate.
-
LangGraph Study Log #2 — Extending State + Dynamic System Prompts + Chatbot (and the Truth Behind 'AI Remembers')
I extended State beyond just messages to include user name, tool call count, and session start time, then used that State to dynamically build the system prompt. Along the way, building a chatbot led me to the real revelation of the day: what 'memory' actually is.