Heck yes—now that ingestion’s chugging, let’s get you a clean, local RAG service + tiny web UI that:

• talks to Postgres over the unix socket (no IP),
• embeds queries with the same model you used to ingest,
• does pgvector similarity search over content.chunks,
• optionally calls a local LLM (e.g., Ollama) to generate the final answer,
• serves a simple one-file web UI so you don’t have to live in a terminal.

I’ll drop a complete FastAPI app you can run as-is. It keeps all your “no network” and socket-only rules intact.

What you’ll get

• rag_app.py — the whole service (retrieval + optional generation + web UI)
• an index command you can run once for pgvector speed
• exact run commands (GPU/CPU both ok)

If you want this in a side-by-side canvas to copy easily, say the word and I’ll pop it in there. For now, here’s the full code inline.

0) One-time pgvector index (fast top-k)

Run once (as postgres) to add an IVFFLAT index and ANALYZE:

sudo -u postgres psql -p 5433 -d tokenspace <<'SQL'
CREATE EXTENSION IF NOT EXISTS vector;

-- cosine distance index on your 768-d vectors
CREATE INDEX IF NOT EXISTS idx_chunks_embedding_ivfflat
ON content.chunks USING ivfflat (embedding vector_cosine_ops)
WITH (lists = 200);

ANALYZE content.chunks;
SQL

1) Install deps in your venv

source ~/Documents/ougway_venv/bin/activate
pip install fastapi uvicorn psycopg2-binary pydantic==2.* requests
# sentence-transformers is already installed from ingest; if not:
# pip install sentence-transformers

2) Save this as ~/Documents/tokenspace/rag_app.py

#!/usr/bin/env python3
"""
Local RAG service for TokenSpace (socket-only Postgres + pgvector).

- Retrieval: content.chunks (embedding vector(768)) using cosine distance
- Embedding: same SentenceTransformer you used to ingest
- Generation: optional via local Ollama HTTP if configured; otherwise returns retrieval-only draft

Env (sane defaults):
  DB_DSN=postgresql:///tokenspace?host=/var/run/postgresql&port=5433
  MODEL_DIR=/mnt/storage/models/noemic-embed-768
  DEVICE=cuda | cpu   (auto if unset)
  TOP_K=8
  MAX_CONTEXT_CHARS=8000
  OLLAMA_HOST=http://127.0.0.1:11434
  OLLAMA_MODEL=llama3.1:8b
"""

import os, json, time, math, pathlib
from typing import List, Optional

import psycopg2
import psycopg2.extras

from fastapi import FastAPI, Query
from fastapi.responses import HTMLResponse, JSONResponse
from pydantic import BaseModel

# Embeddings
from sentence_transformers import SentenceTransformer
import numpy as np

APP_NAME = "rag_app"

DB_DSN  = os.getenv("DB_DSN", "postgresql:///tokenspace?host=/var/run/postgresql&port=5433")
MODEL_DIR = os.getenv("MODEL_DIR", "/mnt/storage/models/noemic-embed-768")
DEVICE = os.getenv("DEVICE")  # 'cuda' or 'cpu' or None
TOP_K = int(os.getenv("TOP_K", "8"))
MAX_CONTEXT_CHARS = int(os.getenv("MAX_CONTEXT_CHARS", "8000"))

# Optional local LLM via Ollama
OLLAMA_HOST  = os.getenv("OLLAMA_HOST")  # e.g. http://127.0.0.1:11434
OLLAMA_MODEL = os.getenv("OLLAMA_MODEL", "llama3.1:8b")

def get_conn():
    return psycopg2.connect(DB_DSN, application_name=APP_NAME)

def load_model():
    kw = {"trust_remote_code": True}
    if DEVICE:  # allow override
        kw["device"] = DEVICE
    try:
        m = SentenceTransformer(MODEL_DIR, **kw)
    except TypeError:
        # older ST versions may not accept 'device'
        m = SentenceTransformer(MODEL_DIR, trust_remote_code=True)
        if DEVICE:
            try:
                m = SentenceTransformer(MODEL_DIR, trust_remote_code=True, device=DEVICE)
            except Exception:
                pass
    return m

model = load_model()

def embed_query(text: str) -> np.ndarray:
    v = model.encode([text], show_progress_bar=False)
    v = np.asarray(v, dtype=np.float32)[0]
    # pgvector wants a literal like: [0.1, -0.2, ...]
    return v

def to_pgvector_literal(vec: np.ndarray) -> str:
    return "[" + ",".join(f"{float(x):.6f}" for x in vec.tolist()) + "]"

def sanitize_for_prompt(s: str) -> str:
    # Clean control chars that might sneak in
    return "".join(ch if (ch in ("\n", "\t") or ord(ch) >= 32) else " " for ch in s)

def retrieve(query: str, k: int = TOP_K):
    vec = embed_query(query)
    vlit = to_pgvector_literal(vec)

    sql = """
    SELECT d.doc_id,
           d.title,
           d.external_id,
           c.chunk_id,
           c.seq,
           c.text,
           (1 - (c.embedding <#> %(qvec)s::vector)) AS cosine_sim
    FROM content.chunks c
    JOIN content.documents d ON d.doc_id = c.doc_id
    ORDER BY c.embedding <#> %(qvec)s::vector
    LIMIT %(k)s;
    """

    with get_conn() as conn:
        with conn.cursor(cursor_factory=psycopg2.extras.DictCursor) as cur:
            cur.execute(sql, {"qvec": vlit, "k": k})
            rows = cur.fetchall()

    results = []
    for r in rows:
        results.append({
            "doc_id": int(r["doc_id"]),
            "title": r["title"],
            "external_id": r["external_id"],
            "chunk_id": int(r["chunk_id"]),
            "seq": int(r["seq"]),
            "score": float(r["cosine_sim"]),
            "text": r["text"],
        })
    return results

def build_context(snippets: List[dict], max_chars: int = MAX_CONTEXT_CHARS) -> str:
    buf, used = [], 0
    for s in snippets:
        t = sanitize_for_prompt(s["text"])
        add = f"\n[doc:{s['doc_id']} seq:{s['seq']} score:{s['score']:.3f} path:{s['external_id']}]\n{t}\n"
        if used + len(add) > max_chars:
            break
        buf.append(add)
        used += len(add)
    return "".join(buf).strip()

def call_ollama(prompt: str) -> Optional[str]:
    if not OLLAMA_HOST:
        return None
    import requests
    url = f"{OLLAMA_HOST}/api/generate"
    payload = {"model": OLLAMA_MODEL, "prompt": prompt, "stream": False}
    try:
        r = requests.post(url, json=payload, timeout=120)
        r.raise_for_status()
        data = r.json()
        return data.get("response")
    except Exception as e:
        return f"[ollama error] {e}"

def make_prompt(user_q: str, context: str) -> str:
    return f"""You are a careful assistant with access to retrieved context chunks.
Answer the question using ONLY the context below. If the answer isn't present,
say you don't have enough information.

# QUESTION
{user_q}

# CONTEXT
{context}
"""

app = FastAPI(title="TokenSpace RAG (local)")

class SearchRequest(BaseModel):
    q: str
    k: int = TOP_K

class ChatRequest(BaseModel):
    q: str
    k: int = TOP_K
    use_ollama: bool = True

@app.get("/", response_class=HTMLResponse)
def index():
    return """<!doctype html>
<html>
<head>
  <meta charset="utf-8"/>
  <title>TokenSpace RAG (local)</title>
  <style>
    body{font-family:system-ui,-apple-system,Segoe UI,Roboto,Arial;max-width:1000px;margin:40px auto;padding:0 16px}
    .row{margin:12px 0}
    textarea{width:100%;height:120px}
    pre{white-space:pre-wrap;background:#f6f8fa;padding:12px;border-radius:8px}
    .hit{border:1px solid #ddd;border-radius:8px;padding:8px;margin:8px 0}
    .score{font-size:12px;color:#555}
    .doc{font-size:12px;color:#555}
    button{padding:8px 14px;border-radius:8px;border:1px solid #bbb;background:#fff;cursor:pointer}
    button:hover{background:#f2f2f2}
  </style>
</head>
<body>
  <h1>TokenSpace RAG (local)</h1>
  <div class="row">
    <label>Query</label>
    <textarea id="q"></textarea>
  </div>
  <div class="row">
    <label>Top-K:</label>
    <input id="k" value="8" size="3"/>
    <label><input type="checkbox" id="use_ollama" checked/> Use Ollama (if configured)</label>
    <button onclick="doSearch()">Search</button>
    <button onclick="doChat()">Ask</button>
  </div>
  <div id="answer"></div>
  <h3>Results</h3>
  <div id="results"></div>

<script>
async function doSearch(){
  const q = document.getElementById('q').value;
  const k = parseInt(document.getElementById('k').value||'8',10);
  const r = await fetch('/search', {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({q,k})});
  const data = await r.json();
  renderResults(data.results||[]);
}

async function doChat(){
  const q = document.getElementById('q').value;
  const k = parseInt(document.getElementById('k').value||'8',10);
  const use_ollama = document.getElementById('use_ollama').checked;
  const r = await fetch('/chat', {method:'POST', headers:{'Content-Type':'application/json'}, body:JSON.stringify({q,k,use_ollama})});
  const data = await r.json();
  document.getElementById('answer').innerHTML = '<h3>Answer</h3><pre>'+ (data.answer||'(no answer)') +'</pre>';
  renderResults(data.results||[]);
}

function renderResults(list){
  const el = document.getElementById('results');
  el.innerHTML = '';
  for(const it of list){
    const div = document.createElement('div');
    div.className='hit';
    div.innerHTML =
      '<div class="score">score: '+it.score.toFixed(3)+'</div>'+
      '<div class="doc">doc:'+it.doc_id+' seq:'+it.seq+'<br/>'+it.external_id+'</div>'+
      '<pre>'+it.text.substring(0,1200)+'</pre>';
    el.appendChild(div);
  }
}
</script>
</body>
</html>"""

@app.post("/search")
def api_search(req: SearchRequest):
    hits = retrieve(req.q, req.k)
    return JSONResponse({"results": hits})

@app.post("/chat")
def api_chat(req: ChatRequest):
    hits = retrieve(req.q, req.k)
    ctx = build_context(hits, MAX_CONTEXT_CHARS)
    prompt = make_prompt(req.q, ctx)
    answer = None
    if req.use_ollama and OLLAMA_HOST:
        answer = call_ollama(prompt)
    if not answer:
        # Fallback: retrieval-only response
        answer = "I used the retrieved context below to draft an answer, but no generator is configured.\n\n" + prompt
    return JSONResponse({"answer": answer, "results": hits})

if __name__ == "__main__":
    import uvicorn
    uvicorn.run("rag_app:app", host="127.0.0.1", port=8000, reload=False)

3) Run it (socket-only DB; GPU for embeddings if available)

# venv on
source ~/Documents/ougway_venv/bin/activate

# env (adjust if your model lives elsewhere)
export DB_DSN='postgresql:///tokenspace?host=/var/run/postgresql&port=5433'
export MODEL_DIR=/mnt/storage/models/noemic-embed-768
export SENTENCE_TRANSFORMERS_TRUST_REMOTE_CODE=true
export DEVICE=cuda     # or 'cpu' if you prefer

# (optional) if you have Ollama running locally:
# export OLLAMA_HOST=http://127.0.0.1:11434
# export OLLAMA_MODEL=llama3.1:8b

python3 ~/Documents/tokenspace/rag_app.py
# Then open http://127.0.0.1:8000 in your browser

You can use it right away just for retrieval (Search). If you also run Ollama locally, check “Use Ollama” for full RAG answers.

4) Example SQL you can test in psql (mirrors the code)

-- Replace 'your question' with anything you'd ask
WITH q AS (
  SELECT '[0.1,0.2, ...]'::vector  -- supply a query vector if testing manually
)
SELECT d.doc_id, d.title, d.external_id, c.seq, 1 - (c.embedding <#> q0) AS score
FROM content.chunks c
JOIN content.documents d ON d.doc_id = c.doc_id,
LATERAL (SELECT (ARRAY[0]::float4[])) s(dummy), -- placeholder
LATERAL (SELECT NULL::vector AS q0) qv
ORDER BY c.embedding <#> q0
LIMIT 8;

(You won’t usually run this one—your app already does it with a real vector.)

5) Hardening notes (later)

• If you want deterministic doc ordering when ingesting/searching, we can add a ORDER BY d.external_id, c.seq.
• For speed on giant corpora, consider increasing lists on the IVFFLAT index and maintenance_work_mem temporarily during index build.
• Optional: add application_name="rag_app" in every DB connect (already done) so pg_stat_activity shows who’s who.

If you want me to paste this into a canvas so you can copy all of it in one grab (and iterate on it later), I’ll do that. Or, if you prefer Flask over FastAPI, I can swap it out—same retrieval core.