How to Build a RAG Chatbot for Your Docs with Node.js, Google Gemini, and pgvector

<p>I was helping a team that had a 200-page API documentation PDF. Every new engineer spent their first two weeks Ctrl+F-ing through it, asking the same questions in Slack, getting redirected to the same paragraphs on page 47.</p> <p>The doc was accurate. It was even well-written. But nobody could find anything in it fast enough for it to be useful.</p> <p>That's the problem RAG, or Retrieval-Augmented Generation, solves.</p> <p>The naïve approach is to stuff your entire PDF into a prompt and let the model figure it out. That breaks down fast: context windows overflow, costs spike on every request, and the model loses the thread somewhere in the wall of text.</p> <p>RAG takes a different approach. Your documents get broken into small chunks upfront. Ask it a question and it digs out the 3 or 4 chunks that best match it — those are what the model actually sees. The model gets a tight, focused context. The answer comes from what your document actually says — not from whatever the LLM memorized during training.</p> <p>In this tutorial, you'll build that from scratch. Upload any PDF — an API reference, an internal spec, a research paper — and ask questions about it in plain English. The system finds the relevant sections and answers from the document itself, not from general training data.</p> <p>The stack: Node.js with Express, Google Gemini for embeddings, Groq for text generation, and pgvector running in Docker. Every piece of it is free — no credit card, no trial period.</p> <p>The complete code is on GitHub at <a href="https://github.com/ziaongit/nodejs-rag-chatbot">nodejs-rag-chatbot</a>.</p> <h2 id="heading-table-of-contents">Table of Contents</h2> <ul> <li><p><a href="#heading-how-rag-works">How RAG Works</a></p> </li> <li><p><a href="#heading-what-were-building">What We're Building</a></p> </li> <li><p><a href="#heading-prerequisites">Prerequisites</a></p> </li> <li><p><a href="#heading-project-setup">Project Setup</a></p> </li> <li><p><a href="#heading-set-up-postgres-with-pgvector-using-docker">Set Up Postgres with pgvector Using Docker</a></p> </li> <li><p><a href="#heading-connect-to-the-database">Connect to the Database</a></p> </li> <li><p><a href="#heading-build-the-ingestion-pipeline">Build the Ingestion Pipeline</a></p> </li> <li><p><a href="#heading-build-the-query-pipeline">Build the Query Pipeline</a></p> </li> <li><p><a href="#heading-build-the-chat-api-with-express">Build the Chat API with Express</a></p> </li> <li><p><a href="#heading-test-the-chatbot">Test the Chatbot</a></p> </li> <li><p><a href="#heading-troubleshooting">Troubleshooting</a></p> </li> <li><p><a href="#heading-how-to-swap-in-openai">How to Swap in OpenAI</a></p> </li> <li><p><a href="#heading-what-to-build-next">What to Build Next</a></p> </li> </ul> <h2 id="heading-how-rag-works">How RAG Works</h2> <p>RAG has two phases, and the code maps directly to both.</p> <p><strong>Ingestion phase</strong> — runs once when you upload a document:</p> <ol> <li><p>Pull the raw text out of the PDF</p> </li> <li><p>Break it into chunks of 400 to 600 characters each, with a bit of overlap so nothing important gets cut at a boundary</p> </li> <li><p>Run each chunk through an embedding model, which turns it into a vector (a long list of numbers that captures what the text means)</p> </li> <li><p>Store each chunk and its vector in Postgres</p> </li> </ol> <p><strong>Query phase</strong> — runs every time someone asks a question:</p> <ol> <li><p>Embed the user's question using the same model</p> </li> <li><p>Search the database for chunks whose vectors are closest to the question vector</p> </li> <li><p>Take the top 5 matching chunks and assemble them into a context block</p> </li> <li><p>Send <code>context + question</code> to the LLM and return its answer</p> </li> </ol> <p>The reason this works better than keyword search: the embedding model captures <em>meaning</em>, not just exact words. If your doc says "terminate the process" and the user asks "how do I stop it?", vector similarity finds that match. Regular string matching doesn't.</p> <p>One thing that trips people up: you must use the same embedding model at query time as you did at ingestion. The model defines the geometric space those vectors live in. Switch models halfway through and the coordinates stop meaning the same thing — you'd be comparing apples to completely different apples.</p> <h2 id="heading-what-were-building">What We're Building</h2> <p>The architecture is intentionally minimal: two endpoints, with nothing you don't need:</p> <ul> <li><p><code>POST /ingest</code>: accepts a PDF upload, chunks it, embeds each chunk, stores vectors in pgvector</p> </li> <li><p><code>POST /chat</code>: accepts a question, retrieves the most relevant chunks, returns an LLM-generated answer</p> </li> </ul> <p>The full tech stack:</p> <ul> <li><p><strong>Node.js + Express</strong> — API layer</p> </li> <li><p><strong>Google Gemini free API</strong> — <code>gemini-embedding-001</code> for embeddings (3,072 dimensions per chunk)</p> </li> <li><p><strong>Groq free API</strong> — <code>llama-3.1-8b-instant</code> for text generation</p> </li> <li><p><strong>PostgreSQL + pgvector</strong> — vector storage and cosine similarity search, running in Docker</p> </li> <li><p><strong>pdf-parse</strong> — extracts raw text from PDF buffers</p> </li> </ul> <p>Gemini handles embeddings and Groq handles generation. Splitting them across two providers isn't arbitrary. Gemini's generation API has a quota limit of zero in certain regions (including Pakistan), while Groq works everywhere with no restrictions. Using Groq for generation means this tutorial runs the same way regardless of where you are.</p> <h2 id="heading-prerequisites">Prerequisites</h2> <p>Before you start:</p> <ul> <li><p>Node.js 20+ installed on your machine</p> </li> <li><p>Docker Desktop running (this is how we'll run Postgres locally)</p> </li> <li><p>A free Google Gemini API key (for embeddings)</p> </li> <li><p>A free Groq API key (for text generation)</p> </li> </ul> <h3 id="heading-how-to-get-your-free-gemini-api-key">How to Get Your Free Gemini API Key</h3> <ol> <li><p>Go to <a href="https://aistudio.google.com/app/apikey">aistudio.google.com/app/apikey</a> and sign in with a Google account</p> </li> <li><p>Click "Create API key"</p> </li> <li><p>Select "Create API key in new project"</p> </li> <li><p>Copy the key — it starts with <code>AIzaSy...</code></p> </li> </ol> <p>No credit card or billing required.</p> <h3 id="heading-how-to-get-your-free-groq-api-key">How to Get Your Free Groq API Key</h3> <ol> <li><p>Go to <a href="https://console.groq.com">console.groq.com</a> and sign up with Google</p> </li> <li><p>Click "API Keys" in the left sidebar</p> </li> <li><p>Click "Create API Key", give it a name, copy the key — it starts with <code>gsk_...</code></p> </li> </ol> <p>Groq is free with generous rate limits and works in all regions.</p> <h2 id="heading-project-setup">Project Setup</h2> <p>Create the project directory and initialize it:</p> <pre><code class="language-bash">mkdir nodejs-rag-chatbot cd nodejs-rag-chatbot npm init -y </code></pre> <p>Install dependencies:</p> <pre><code class="language-bash">npm install express pg pdf-parse uuid dotenv multer npm install --save-dev nodemon </code></pre> <p>A quick note on the packages: <code>multer</code> is what makes file uploads work on the <code>/ingest</code> endpoint. Without it, Express can't parse multipart form data.</p> <p><code>pdf-parse</code> does the heavy lifting on PDFs, though watch out for scanned PDFs. Those are just images with no text layer underneath, so you'll get back an empty string.</p> <p><code>pg</code> talks to Postgres, <code>uuid</code> gives each row a unique ID, and <code>dotenv</code> loads your keys before the app does anything.</p> <p>Create a <code>.env</code> in the project root. It needs seven values:</p> <pre><code class="language-plaintext">GEMINI_API_KEY=AIzaSy... ← your Gemini key from Google AI Studio GROQ_API_KEY=gsk_... ← your Groq key from console.groq.com POSTGRES_USER=rag_user POSTGRES_PASSWORD=rag_pass ← choose any password, this is local only POSTGRES_DB=rag_db DATABASE_URL=postgresql://rag_user:rag_pass@localhost:5432/rag_db PORT=3000 </code></pre> <p>One thing: the password in <code>POSTGRES_PASSWORD</code> and the one in <code>DATABASE_URL</code> must match exactly. I changed just one of them once and spent way too long debugging a "password authentication failed" error before realising the two values were out of sync.</p> <p>Update <code>package.json</code> scripts:</p> <pre><code class="language-json">"scripts": { "start": "node src/index.js", "dev": "nodemon src/index.js" } </code></pre> <p>Create the <code>src</code> directory:</p> <pre><code class="language-bash">mkdir src </code></pre> <p>Your final folder structure will look like this:</p> <pre><code class="language-plaintext">nodejs-rag-chatbot/ ├── src/ │ ├── index.js ← Express app entry point │ ├── db.js ← Postgres connection and schema setup │ ├── embeddings.js ← Gemini embedding + Groq generation │ ├── ingest.js ← Document ingestion pipeline │ └── query.js ← RAG query pipeline ├── docker-compose.yml ├── .env └── package.json </code></pre> <h2 id="heading-set-up-postgres-with-pgvector-using-docker">Set Up Postgres with pgvector Using Docker</h2> <p>pgvector adds a <code>vector</code> column type to Postgres and the operators needed to search it by similarity. Normally you'd have to install it yourself, but the <code>pgvector/pgvector</code> Docker image ships with it already baked in. Just pull the image and you're good.</p> <p>Now add <code>docker-compose.yml</code> to the project root:</p> <pre><code class="language-yaml">services: postgres: image: pgvector/pgvector:pg16 environment: POSTGRES_USER: ${POSTGRES_USER} POSTGRES_PASSWORD: ${POSTGRES_PASSWORD} POSTGRES_DB: ${POSTGRES_DB} ports: - "5432:5432" volumes: - pgdata:/var/lib/postgresql/data volumes: pgdata: </code></pre> <p>Those <code>${VARIABLE}</code> references get swapped out from <code>.env</code> when Compose starts — so <code>docker-compose.yml</code> itself stays clean. This is worth doing from day one. I've seen people skip this and regret it after a repo goes public.</p> <p>Start it:</p> <pre><code class="language-bash">docker compose up -d </code></pre> <h2 id="heading-connect-to-the-database">Connect to the Database</h2> <p>Create <code>src/db.js</code>. This sets up the connection pool and creates the <code>documents</code> table on first run:</p> <pre><code class="language-javascript">const { Pool } = require('pg'); const pool = new Pool({ connectionString: process.env.DATABASE_URL, }); async function initDb() { await pool.query(`CREATE EXTENSION IF NOT EXISTS vector`); await pool.query(` CREATE TABLE IF NOT EXISTS documents ( id UUID PRIMARY KEY, content TEXT NOT NULL, source TEXT NOT NULL, embedding VECTOR(3072) ) `); console.log('Database ready'); } module.exports = { pool, initDb }; </code></pre> <p>The <code>VECTOR(3072)</code> dimension matches the output of Gemini's <code>gemini-embedding-001</code> model exactly. If you use a different embedding model in the future, check its output dimensions and update this number to match.</p> <h2 id="heading-build-the-ingestion-pipeline">Build the Ingestion Pipeline</h2> <p>Start with <code>embeddings.js</code>. This file is the bridge to both external APIs — Gemini for turning text into vectors, Groq for generating the final answer. Keeping both in one place means a single file to touch if you ever swap providers.</p> <p><strong>src/embeddings.js:</strong></p> <pre><code class="language-javascript">const GEMINI_KEY = process.env.GEMINI_API_KEY; const GEMINI_BASE = 'https://generativelanguage.googleapis.com/v1/models'; async function embedText(text) { const res = await fetch( `${GEMINI_BASE}/gemini-embedding-001:embedContent?key=${GEMINI_KEY}`, { method: 'POST', headers: { 'Content-Type': 'application/json' }, body: JSON.stringify({ content: { parts: [{ text }] } }), } ); const data = await res.json(); if (!res.ok) throw new Error(JSON.stringify(data)); return data.embedding.values; } async function generateAnswer(context, question) { const res = await fetch( 'https://api.groq.com/openai/v1/chat/completions', { method: 'POST', headers: { 'Content-Type': 'application/json', 'Authorization': `Bearer ${process.env.GROQ_API_KEY}`, }, body: JSON.stringify({ model: 'llama-3.1-8b-instant', messages: [ { role: 'system', content: 'You are a helpful assistant. Answer the question using only the context provided. If the context does not contain enough information, say so clearly.', }, { role: 'user', content: `Context:\n${context}\n\nQuestion: ${question}`, }, ], }), } ); const data = await res.json(); if (!res.ok) throw new Error(JSON.stringify(data)); return data.choices[0].message.content; } module.exports = { embedText, generateAnswer }; </code></pre> <p>We're calling both APIs directly with Node.js's built-in <code>fetch</code> rather than the official SDKs. The reason is practical: Google's Node.js SDK routes requests through the <code>v1beta</code> endpoint by default, and <code>gemini-embedding-001</code> isn't available there — only on <code>v1</code>. Direct fetch sidesteps that entirely and keeps the dependency count low.</p> <p><strong>src/ingest.js:</strong></p> <pre><code class="language-javascript">const pdfParse = require('pdf-parse'); const { v4: uuidv4 } = require('uuid'); const { pool } = require('./db'); const { embedText } = require('./embeddings'); function chunkText(text, chunkSize = 500, overlap = 50) { const chunks = []; let start = 0; while (start &lt; text.length) { const end = Math.min(start + chunkSize, text.length); chunks.push(text.slice(start, end).trim()); start += chunkSize - overlap; } return chunks.filter(chunk =&gt; chunk.length &gt; 50); } async function ingestDocument(buffer, filename) { const { text } = await pdfParse(buffer); const chunks = chunkText(text); console.log(`Processing ${chunks.length} chunks from "${filename}"`); for (const chunk of chunks) { const embedding = await embedText(chunk); await pool.query( `INSERT INTO documents (id, content, source, embedding) VALUES ($1, $2, $3, $4::vector)`, [uuidv4(), chunk, filename, JSON.stringify(embedding)] ); } return chunks.length; } module.exports = { ingestDocument }; </code></pre> <p>500 characters per chunk, with 50 characters of overlap between neighbours.</p> <p>Why the overlap? Without it, a sentence that straddles a boundary gets split, half in one chunk, half in the next — and neither piece makes sense on its own when retrieved. The overlap keeps those boundary sentences intact.</p> <p>For most technical docs, 500 is a good starting point. Dense legal or financial text tends to need something closer to 300.</p> <h2 id="heading-build-the-query-pipeline">Build the Query Pipeline</h2> <p><strong>src/query.js:</strong></p> <pre><code class="language-javascript">const { pool } = require('./db'); const { embedText, generateAnswer } = require('./embeddings'); async function queryDocuments(question) { const questionEmbedding = await embedText(question); const { rows } = await pool.query( `SELECT content, source, 1 - (embedding &lt;=&gt; $1::vector) AS similarity FROM documents ORDER BY embedding &lt;=&gt; $1::vector LIMIT 5`, [JSON.stringify(questionEmbedding)] ); if (rows.length === 0) { return { answer: 'No relevant documents found.', sources: [] }; } const context = rows.map(r =&gt; r.content).join('\n\n---\n\n'); const answer = await generateAnswer(context, question); return { answer, sources: [...new Set(rows.map(r =&gt; r.source))], topSimilarity: parseFloat(rows[0].similarity).toFixed(3), }; } module.exports = { queryDocuments }; </code></pre> <p>The <code>&lt;=&gt;</code> operator is pgvector's cosine distance. Semantically similar text produces vectors that point in the same direction — so the distance between them is small. Flip that with <code>1 - distance</code> and you get a similarity score, where anything close to 1 means a strong match.</p> <p>I found 0.7 to be a reliable threshold in my testing — chunks above that were almost always relevant. Anything below 0.5 and the retrieval was really stretching, pulling chunks that shared a keyword or two but weren't actually answering the question.</p> <p>When that happens, the system prompt instruction ("if the context does not contain enough information, say so clearly") becomes important. A well-behaved model will tell the user it doesn't know rather than guess.</p> <p>We also surface the source filename. Once you've ingested more than one document, users need to know whether that answer came from the architecture spec or the incident report.</p> <h2 id="heading-build-the-chat-api-with-express">Build the Chat API with Express</h2> <p><strong>src/index.js:</strong></p> <pre><code class="language-javascript">require('dotenv').config(); const express = require('express'); const multer = require('multer'); const { initDb } = require('./db'); const { ingestDocument } = require('./ingest'); const { queryDocuments } = require('./query'); const app = express(); const upload = multer({ storage: multer.memoryStorage() }); app.use(express.json()); app.post('/ingest', upload.single('file'), async (req, res) =&gt; { if (!req.file) { return res.status(400).json({ error: 'No file uploaded' }); } if (!req.file.mimetype.includes('pdf')) { return res.status(400).json({ error: 'Only PDF files are supported' }); } try { const count = await ingestDocument(req.file.buffer, req.file.originalname); res.json({ message: `Ingested ${count} chunks from "${req.file.originalname}"` }); } catch (err) { console.error(err); res.status(500).json({ error: 'Ingestion failed', detail: err.message }); } }); app.post('/chat', async (req, res) =&gt; { const { question } = req.body; if (!question || typeof question !== 'string') { return res.status(400).json({ error: 'question is required' }); } try { const result = await queryDocuments(question); res.json(result); } catch (err) { console.error(err); res.status(500).json({ error: 'Query failed', detail: err.message }); } }); const PORT = process.env.PORT || 3000; initDb().then(() =&gt; { app.listen(PORT, () =&gt; { console.log(`RAG chatbot running on port ${PORT}`); }); }); </code></pre> <p><code>memoryStorage()</code> keeps the uploaded file in a buffer instead of writing it to disk. We parse it and store the chunks immediately, so there's nothing to save.</p> <h2 id="heading-test-the-chatbot">Test the Chatbot</h2> <p>Start the server:</p> <pre><code class="language-bash">npm run dev </code></pre> <p>You should see:</p> <pre><code class="language-plaintext">Database ready RAG chatbot running on port 3000 </code></pre> <p>Upload a PDF. Any PDF works. I tested with a copy of a Node.js best practices guide:</p> <pre><code class="language-bash"># Linux / macOS curl -X POST http://localhost:3000/ingest -F "file=@your-document.pdf" # Windows PowerShell curl.exe -X POST http://localhost:3000/ingest -F "file=@your-document.pdf" </code></pre> <p>Response:</p> <pre><code class="language-json">{ "message": "Ingested 142 chunks from \"your-document.pdf\"" } </code></pre> <p>Now ask a question:</p> <pre><code class="language-bash"># Linux / macOS curl -X POST http://localhost:3000/chat \ -H "Content-Type: application/json" \ -d '{ "question": "How should I handle errors in async functions?" }' # Windows PowerShell curl.exe -X POST http://localhost:3000/chat -H "Content-Type: application/json" -d "{\"question\": \"How should I handle errors in async functions?\"}" </code></pre> <p>Response:</p> <pre><code class="language-json">{ "answer": "For async functions in Node.js, wrap your logic in a try/catch block to handle rejected promises. In Express, pass the caught error to next(err) to trigger your error-handling middleware. Alternatively, you can create a wrapper function that wraps any async route handler in a promise and calls next on rejection, keeping your route handlers clean...", "sources": ["your-document.pdf"], "topSimilarity": "0.841" } </code></pre> <p>The <code>topSimilarity</code> score tells you how well the retrieval went. Above 0.7 and the chunks pulled were genuinely relevant. Below 0.5, and the search was struggling: it found something, but not something that actually answers the question.</p> <p>Try asking about something your PDF doesn't mention. If the system prompt is doing its job, the model should say it doesn't have enough information rather than making something up. That's the behaviour you want in production.</p> <p>The repo includes two diagnostic scripts that are useful if anything isn't working:</p> <ul> <li><p><code>node test-keys.js</code> — tests both API keys live and reports whether each one succeeds</p> </li> <li><p><code>node list-models.js</code> — fetches the full list of Gemini models available to your API key</p> </li> </ul> <p>Run these before diving into the troubleshooting section below.</p> <h2 id="heading-troubleshooting">Troubleshooting</h2> <p>Everything in this section is a real error I hit while building this. Nothing hypothetical.</p> <h3 id="heading-port-5432-is-already-in-use">Port 5432 is already in use</h3> <pre><code class="language-plaintext">Error: bind: address already in use </code></pre> <p>Something else — probably a local Postgres install — is already on that port. Two fixes are needed. First, in <code>docker-compose.yml</code>:</p> <pre><code class="language-yaml">ports: - "5433:5432" </code></pre> <p>Second, update <code>DATABASE_URL</code> in <code>.env</code>:</p> <pre><code class="language-plaintext">DATABASE_URL=postgresql://rag_user:rag_pass@localhost:5433/rag_db </code></pre> <p>The container itself still listens on 5432 internally. You're just changing which port your machine uses to reach it.</p> <h3 id="heading-password-authentication-failed-for-user-raguser">Password authentication failed for user "rag_user"</h3> <pre><code class="language-plaintext">Error: password authentication failed for user "rag_user" </code></pre> <p>The password Postgres was initialized with doesn't match what your app is sending. Open <code>.env</code> and compare <code>POSTGRES_PASSWORD</code> with the password embedded in <code>DATABASE_URL</code>. They need to be character-for-character identical.</p> <p>After fixing the mismatch, the old volume still has the wrong password baked into it. You must destroy it and start fresh:</p> <pre><code class="language-bash">docker compose down -v docker compose up -d </code></pre> <p>The <code>-v</code> flag deletes the data volume. Postgres reinitializes on the next start with the credentials from your current <code>.env</code>.</p> <h3 id="heading-gemini-model-not-found-404">Gemini model not found (404)</h3> <pre><code class="language-json">{ "error": { "code": 404, "message": "models/text-embedding-004 is not found" } } </code></pre> <p>The Google AI model naming has changed. Older tutorials and blog posts reference model names that no longer exist on the v1 endpoint. The correct model for this stack is <code>gemini-embedding-001</code>. That's what this repo uses.</p> <p>If you want to see every model available to your API key, run:</p> <pre><code class="language-bash">node list-models.js </code></pre> <p>That script fetches the live list directly from the API so you're not guessing.</p> <h3 id="heading-vector-dimension-mismatch">Vector dimension mismatch</h3> <pre><code class="language-plaintext">ERROR: expected 768 dimensions, not 3072 </code></pre> <p>This error appears when your database table was created with a different dimension count than what your embedding model produces. <code>gemini-embedding-001</code> outputs 3,072-dimensional vectors. The <code>documents</code> table in this tutorial uses <code>VECTOR(3072)</code> to match.</p> <p>If you get this error, it means either an old table exists with the wrong dimension, or you changed embedding models without recreating the table. Drop the data volume and restart:</p> <pre><code class="language-bash">docker compose down -v docker compose up -d </code></pre> <h3 id="heading-vector-index-dimension-limit">Vector index dimension limit</h3> <pre><code class="language-plaintext">ERROR: ivfflat index type only supports up to 2000 dimensions </code></pre> <p>pgvector's <code>ivfflat</code> and <code>hnsw</code> index types have a maximum dimension of 2000. Since <code>gemini-embedding-001</code> produces 3,072-dimensional vectors, neither index type works.</p> <p>This tutorial drops the index and lets pgvector do a full scan — fine for development and any reasonably sized corpus. Scaling to thousands of documents in production? Pick a model under 2000 dimensions. OpenAI's <code>text-embedding-3-small</code> outputs 1536 and plays nicely with both index types.</p> <h3 id="heading-port-3000-is-already-in-use">Port 3000 is already in use</h3> <pre><code class="language-plaintext">Error: EADDRINUSE: address already in use :::3000 </code></pre> <p>Some other process got there first. Swap the port number in <code>.env</code>:</p> <pre><code class="language-plaintext">PORT=3002 </code></pre> <p>Save it and restart the server.</p> <h3 id="heading-gemini-generation-returns-quota-exceeded-limit-0">Gemini generation returns quota exceeded (limit: 0)</h3> <pre><code class="language-json">{ "error": { "status": "RESOURCE_EXHAUSTED", "message": "Quota exceeded for quota metric ... with limit 0" } } </code></pre> <p>That <code>limit 0</code> means Google has switched off free generation in your country entirely — not that you've used it up. I hit this myself while testing from Pakistan.</p> <p>That's exactly why this tutorial uses Groq instead. Make sure <code>GROQ_API_KEY</code> is in your <code>.env</code> and that <code>generateAnswer</code> in <code>src/embeddings.js</code> is pointing at <code>api.groq.com</code>.</p> <p>To verify both keys work, run:</p> <pre><code class="language-bash">node test-keys.js </code></pre> <p>It tests the Gemini embedding endpoint and the Groq generation endpoint independently and reports whether each succeeds.</p> <h3 id="heading-nodemon-doesnt-pick-up-changes-to-env">nodemon doesn't pick up changes to <code>.env</code></h3> <p>nodemon only watches <code>.js</code> files — <code>.env</code> changes don't trigger a restart. Switch to the terminal running the server and type <code>rs</code>, then hit Enter. That forces a restart and picks up whatever you changed.</p> <h3 id="heading-curl-doesnt-work-in-windows-powershell"><code>curl</code> doesn't work in Windows PowerShell</h3> <pre><code class="language-plaintext">curl : The term 'curl' is not recognized </code></pre> <p>or</p> <pre><code class="language-plaintext">curl : Cannot bind parameter because parameter 'Method' is specified more than once </code></pre> <p>PowerShell has a built-in <code>curl</code> alias that points to <code>Invoke-WebRequest</code> — completely different flags, completely different behaviour. Add <code>.exe</code> and you bypass the alias and hit the real binary.</p> <p>So instead of <code>curl</code>, type <code>curl.exe</code>:</p> <pre><code class="language-powershell"># Ingest curl.exe -X POST http://localhost:3000/ingest -F "file=@your-document.pdf" # Chat curl.exe -X POST http://localhost:3000/chat -H "Content-Type: application/json" -d "{\"question\": \"How do I handle async errors?\"}" </code></pre> <p>That <code>.exe</code> is the whole fix.</p> <h3 id="heading-docker-desktop-stopped-running">Docker Desktop stopped running</h3> <p>Docker Desktop doesn't start automatically after a reboot on most setups. If your Docker commands suddenly fail with connection errors, that's probably why. Open Docker Desktop, wait until it says "Engine running", then try again.</p> <h2 id="heading-how-to-swap-in-openai">How to Swap in OpenAI</h2> <p>If you want to use the OpenAI API instead of Gemini, it's three changes.</p> <p>1. Install the OpenAI SDK:</p> <pre><code class="language-bash">npm install openai </code></pre> <p>2. Replace <code>src/embeddings.js</code> entirely:</p> <pre><code class="language-javascript">const OpenAI = require('openai'); const client = new OpenAI({ apiKey: process.env.OPENAI_API_KEY }); async function embedText(text) { const result = await client.embeddings.create({ model: 'text-embedding-3-small', input: text, }); return result.data[0].embedding; } async function generateAnswer(context, question) { const result = await client.chat.completions.create({ model: 'gpt-4o-mini', messages: [ { role: 'system', content: 'Answer only from the context provided. If the context is insufficient, say so.' }, { role: 'user', content: `Context:\n${context}\n\nQuestion: ${question}` }, ], }); return result.choices[0].message.content; } module.exports = { embedText, generateAnswer }; </code></pre> <p>3. Update the vector dimension in <code>src/db.js</code>:</p> <p>Open <code>db.js</code> and swap <code>VECTOR(3072)</code> for <code>VECTOR(1536)</code> — that's the output size of <code>text-embedding-3-small</code>. Then kill the volume so the table gets recreated with the right dimensions:</p> <pre><code class="language-bash">docker compose down -v docker compose up -d </code></pre> <p>Nothing else needs touching. The ingestion and query logic works the same regardless of which model you plugged in.</p> <h2 id="heading-what-to-build-next">What to Build Next</h2> <p>What you've built works. But there are some gaps that come up quickly once you put it in front of real users.</p> <p>The most noticeable one is <strong>streaming</strong>. Right now <code>/chat</code> holds the connection open until Groq finishes generating the full answer, then returns everything at once. On a short question that's fine. On a longer one, the user stares at nothing for a few seconds and wonders if the request hung.</p> <p>The Groq API supports streaming — add <code>stream: true</code> to the request body and tokens start coming back as they're generated. Piping those through Express with <code>res.write()</code> is maybe 15 minutes of work and the difference in feel is immediate.</p> <p><strong>Metadata filtering</strong> is the second thing you'll want. Once you've loaded more than a few documents, queries bleed across everything: ask about the API spec and you'll get chunks from the onboarding guide too.</p> <p>The fix is a <code>metadata JSONB</code> column where you store the document ID on ingest, then add <code>WHERE metadata-&gt;&gt;'doc_id' = $1</code> to the similarity query. Expose it as an optional body field on <code>/chat</code>: <code>{ "question": "...", "docId": "api-spec-v2" }</code>. Users get scoped results, and you get much cleaner answers.</p> <p>When your corpus grows into the hundreds of documents, look at <strong>re-ranking</strong>. Vector similarity retrieval is fast but approximate — it finds chunks that are semantically close to the question, not necessarily the ones that most directly answer it.</p> <p>The pattern is: retrieve the top 20 by cosine distance, then run a cross-encoder over them to re-score by actual relevance, then take the best 5 from that second pass. LangChain.js has a cross-encoder wrapper if you don't want to implement it yourself.</p> <p>The last thing most people forget until they actually need it is <strong>document management</strong> — the ability to list what's ingested, delete a specific file, and re-ingest an updated version.</p> <p>A <code>DELETE FROM documents WHERE source = $1</code> handles the delete case. Add a <code>GET /documents</code> endpoint that queries <code>SELECT DISTINCT source FROM documents</code> and you have a complete enough API for real use.</p> <p>RAG isn't magic. It's a well-scoped retrieval problem combined with a language model that's been told to stay within its lane.</p> <p>The quality of your answers depends on three things: how cleanly your PDFs parse, how well your chunk size fits the content type, and how clearly your system prompt instructs the model to say "I don't know" rather than guess. Get those right and you've built something genuinely useful: the kind of thing that saves a new engineer's first two weeks.</p>