Tag: llm
All the articles with the tag "llm".
-
Claude API Error Handling and Retry — Distinguishing Transient, Permanent, and Format Errors + Exponential Backoff
What errors should I handle, and how, to build a reliable LLM service? I classify errors into transient / permanent / user input / response format categories, and practice through 4 stages: basic try-except wrapper → Exponential backoff → Anthropic SDK built-in retry → JSON response validation.
-
Prompt Caching — Bringing Input Cost Close to Zero for Repeated Inputs
Prompt caching is essential for chatbots, agents, and RAG that send long system prompts or the same context on every call. I ran a direct comparison with and without caching using STM32/embedded questions — cost dropped by half, and time dropped slightly.
-
Claude Vision — How Do You Send Images In, and Which Model Should Receive Them
I tried both ways of sending images — base64 encoding and URL — and found them accurate but more token-costly, hit limits with small text and complex shapes, and got a sense of when to move up to a bigger model for tasks that need accuracy.
-
Giving LLMs Tools — Claude Tool Use and the Agent Loop
LLMs don't know large-number arithmetic or today's weather. I tried out Tool Use, which patches that weakness with external function calls. Message flow, automatic multi-tool selection, and the agent loop.