Building a Production-Ready RAG Chatbot with Next.js, LangChain, and Pinecone

A RAG chatbot sounds simple in a diagram: embed some documents, search them, feed the results to an LLM. In practice, every one of those steps has a decision buried in it that determines whether the bot actually answers correctly or confidently makes something up — and this project exists to work through those decisions on a real, deployed system rather than a notebook demo.

The bot's job: answer questions about my own portfolio — projects, skills, background — grounded in the actual site content instead of the model's general training knowledge.

Chunking: the decision everyone underrates

How you split source content into chunks determines what the retriever can even find. Chunks too large dilute relevance (a match buried in mostly-irrelevant text still gets retrieved and confuses the model); chunks too small lose context (a fragment with no surrounding information the model needs to answer well). The chunking strategy here is tuned to the shape of the actual content — project descriptions and blog posts — rather than a generic fixed-size splitter applied blindly.

Embeddings and vector search

Each chunk gets embedded and stored in Pinecone, which handles the approximate-nearest-neighbor search at query time. When a question comes in, it's embedded the same way, and the top-matching chunks come back as retrieval context — this is the "retrieval" half of retrieval-augmented generation, and its quality caps how good the generation half can possibly be, no matter how good the LLM is downstream.

Generation: LangChain orchestrating OpenAI

LangChain handles the orchestration layer — taking the retrieved chunks, assembling them into a prompt with the user's question, and calling OpenAI's API to generate the response. The retrieved context is explicitly there to ground the answer, which is what separates this from a bot that just hallucinates plausible-sounding facts about projects that don't exist.

Streaming, not a spinner

Responses stream token-by-token to the frontend rather than waiting for the full generation to complete before showing anything — which matters more for perceived latency than actual latency; a RAG pipeline with retrieval + generation can easily take several seconds, and streaming is what keeps that from feeling broken.

Why build this instead of just... answering questions in a FAQ page

A static FAQ can't handle a question phrased differently than expected. RAG lets the same underlying content (project descriptions, blog posts) answer arbitrarily-phrased questions without needing every possible phrasing pre-written — the tradeoff is the engineering complexity of chunking, retrieval quality, and grounding that this project works through directly.