Tag: ollama
All the articles with the tag "ollama".
-
Ollama Local LLM Routing #2 — I was talking to a local model using OpenAI API format
I opened up code I had only set up but never read last time. Tracing it from the pydantic models in types.py to the abstract class in backends, I found out that requests sent to the local ollama were in OpenAI API format.
-
Ollama Local LLM Routing #1 — One big model would do, but I split it into four small ones
I decided to try routing prompts across several open-source LLMs on my MacBook first. Both my home computer and MacBook fell short on specs, so I had to pick different models, and I finished installing ollama.
-
FEMS Project #3 — Streamlit Comparison Dashboard + 3-Backend Question Evaluation (Claude 9 / exaone 7 / gpt 7)
Put a Streamlit dashboard on top of FEMS RAG and threw the same question at 3 backends (exaone3.5:7.8b / claude-opus-4-8 / gpt-4o) simultaneously to compare. Q1 (air compressor anomaly in May) — exaone and gpt said 'no data', while Claude inferred 'weekend-hours anomaly' from summary stats alone without raw data + disclosed its limitations. Q2 (savings measures from a manager's perspective) — Claude 9/10 (incomplete due to token truncation), exaone 7/10 (broken index), gpt 7/10 (concise but hallucinated 'capacitor'). Results of 5-axis scoring.
-
FEMS Project #2 — A Real 230-Page Corpus + Chunk Quality Gates + the Chinese-Character Mixing Trap in a Local LLM
Built a corpus of roughly 230 pages / 35,000 rows from the Korea BEMS Association guides, Korea Energy Agency materials, and the UCI Steel dataset. After paragraph-based chunking (target 800 chars) + bge-m3 + Chroma indexing, a quality gate (ratio of complete Hangul/ASCII characters) excluded 7 chunks from table-of-contents pages. Then I hit a trap — qwen2.5:7b mixed in Chinese characters on the second question and suffered generation collapse (spitting out unrelated Chinese city coordinates as GeoJSON). Partially fixed with temperature / system prompt → ultimately switched to exaone3.5:7.8b for clean handling.