Tag: openai
All the articles with the tag "openai".
-
RAG Embedding Comparison — Measuring recall@k on My Blog Data (OpenAI vs bge-m3)
After establishing the ["feeling-based benchmarking → numeric benchmarking" principle](/en/posts/quant-study-00-pandas) in my quant retrospective, I actually quantified an embedding model comparison this time. I indexed 441 chunks from my blog posts with OpenAI text-embedding-3-small and bge-m3 respectively, then measured recall@3 with a test set of 20 question-answer source pairs. Overall: OpenAI 80% vs bge-m3 90%. bge-m3 hit 100% on hard-difficulty questions — the decisive factor was connecting to the source text by meaning even when words didn't overlap. On easy questions, the misses turned out to be caused by typos (cladue, underscores) — a twist showing the grading criteria itself was wrong.
-
Building a RAG Q&A System with Chroma + My Blog — From Parts to System
Yesterday I played with the parts of RAG (embedding / cosine similarity / chunks). Today, I put them together into a system. I introduced the Chroma vector DB, indexed 270 chunks from my blog, and built a Q&A system with OpenAI + Claude. The most memorable moment was when it answered 'I don't know' to a question about information not covered on my blog.
-
Building RAG from Scratch — Embedding, Cosine Similarity, Chunk Search (with Korean vs English Comparison)
Carving out the inside of RAG (Retrieval-Augmented Generation) with actual code, from what embedding even is to debugging suspicious results. I pulled 1536-dimension vectors with OpenAI's text-embedding-3-small, compared them with cosine similarity, compared Korean vs English performance, and worked through chunk splitting. Surprisingly, Korean embedding similarity turned out to be lower.