Heck yeah—we can spin up a fully local trial on this machine with:

• your model loaded once in llama.cpp server mode (stays warm),
• a tiny RAG API (does retrieval, builds the prompt, sends to llama server),
• a super simple web page (token box + chat box),
• an invite token check, and
• a single-user lock so only one turn runs at a time.

All paste-once, no nano. 👇

0) Prep: env you already use

source ~/ougway_env/venv/bin/activate

export GGML_CUDA=1
export QWEN="$HOME/ougway_env/llama.cpp/models/qwen3/qwen3-8b-q6_k.gguf"
export LLAMA_CTX=32768
export LLAMA_NGL=999
export LLAMA_BATCH=32

export TS_DSN="dbname=tokenspace user=darren host=localhost password=LittleOugway"
export EMBED_MODEL="nomic-ai/nomic-embed-text-v1.5"
export HF_HUB_OFFLINE=1

1) Start llama.cpp server (in a separate terminal)

"$HOME/ougway_env/llama.cpp/build/bin/llama-server" \
  -m "$QWEN" -c "$LLAMA_CTX" -ngl "$LLAMA_NGL" -b "$LLAMA_BATCH" -t "$(nproc)" \
  --host 127.0.0.1 --port 8080

This loads the model once and exposes an OpenAI-compatible endpoint at http://127.0.0.1:8080/v1/chat/completions.

2) Create a simple invites table + one token

psql "$TS_DSN" -c "
CREATE SCHEMA IF NOT EXISTS cog;
CREATE TABLE IF NOT EXISTS cog.invites (
  token      text PRIMARY KEY,
  issued_at  timestamptz DEFAULT now(),
  expires_at timestamptz NOT NULL,
  used_at    timestamptz
);
INSERT INTO cog.invites(token, expires_at)
VALUES ('TRY-OUGWAY-LOCAL', now() + interval '7 days')
ON CONFLICT (token) DO NOTHING;
"

3) Install deps

pip install fastapi uvicorn requests sentence-transformers psycopg2-binary numpy

4) Paste-once: RAG server + web UI (single file)

cat > ~/Desktop/rag_server.py <<'PY'
#!/usr/bin/env python3
import os, uuid, json, time, fcntl, hashlib
from typing import List, Tuple, Optional
import numpy as np
import psycopg2, requests
from fastapi import FastAPI, HTTPException, Request
from fastapi.responses import HTMLResponse, JSONResponse
from pydantic import BaseModel

# ------------ Config ------------
DSN          = os.environ.get("TS_DSN", "dbname=tokenspace user=darren host=localhost password=LittleOugway")
LLAMA_URL    = os.environ.get("LLAMA_URL", "http://127.0.0.1:8080/v1/chat/completions")
EMBED_MODEL  = os.environ.get("EMBED_MODEL", "nomic-ai/nomic-embed-text-v1.5")
TOKEN_BUDGET = int(os.environ.get("RAG_MAX_TOKENS", "2048"))
TOP_K        = int(os.environ.get("RAG_K", "6"))
LOCK_PATH    = "/tmp/oogway.lock"   # single-user gate
SESSION_TTL  = 60 * 15              # 15 min soft session (demo)

# ------------ Embeddings ------------
_USE_ST = False
_st = None
try:
    from sentence_transformers import SentenceTransformer
    _st = SentenceTransformer(EMBED_MODEL, trust_remote_code=True)
    dim = getattr(_st, "get_sentence_embedding_dimension", lambda: None)()
    _USE_ST = (dim == 768)
except Exception:
    _USE_ST = False

def embed_one(text: str) -> np.ndarray:
    if _USE_ST:
        v = _st.encode([text], normalize_embeddings=True)
        return np.asarray(v, dtype=np.float32)[0]
    # deterministic 768-d fallback
    h = hashlib.sha256(text.encode("utf-8")).digest()
    seed = int.from_bytes(h[:8], "big") % (2**31 - 1)
    rng = np.random.default_rng(seed)
    v = rng.normal(0.0, 1.0, 768).astype(np.float32)
    n = float(np.linalg.norm(v)); 
    if n > 0: v /= n
    return v

def vec_literal(v: np.ndarray) -> str:
    return "[" + ",".join(f"{x:.6f}" for x in v.tolist()) + "]"

# ------------ DB helpers ------------
def db():
    return psycopg2.connect(DSN)

def check_token(token: str) -> bool:
    with db() as conn, conn.cursor() as cur:
        cur.execute("""SELECT used_at, expires_at FROM cog.invites WHERE token=%s""", (token,))
        row = cur.fetchone()
        if not row: return False
        used_at, expires_at = row
        if time.time() > expires_at.timestamp(): return False
        if not used_at:
            cur.execute("""UPDATE cog.invites SET used_at=now() WHERE token=%s""", (token,))
        return True

def retrieve(qvec_lit: str, k: int):
    with db() as conn, conn.cursor() as cur:
        cur.execute(f"""
            SELECT ch.chunk_id, left(ch.text, 900) AS preview, d.title, s.uri,
                   ch.embedding <=> %s::vector AS dist
            FROM content.chunks ch
            JOIN content.documents d ON d.doc_id = ch.doc_id
            LEFT JOIN content.sources s ON s.source_id = d.source_id
            ORDER BY dist ASC
            LIMIT %s;
        """, (qvec_lit, k))
        return cur.fetchall()

# ------------ Prompt ------------
def build_prompt(question: str, hits, think_mode: bool) -> str:
    ctx_blocks=[]
    for i,(cid,preview,title,uri,dist) in enumerate(hits,1):
        meta=[]
        if title: meta.append(f"title: {title}")
        if uri:   meta.append(f"path: {uri}")
        meta.append(f"chunk_id: {cid}  dist: {dist:.4f}")
        ctx_blocks.append(f"### Context {i} ({' | '.join(meta)})\n{preview}")
    context="\n\n".join(ctx_blocks)
    sys_inst=("You are a concise assistant. Use ONLY the provided context. "
              "If the answer is not in the context, say you do not find it.")
    user_q = question
    if not think_mode:
        user_q += "\n\nGive only the final answer — no <think> or hidden reasoning."
    return f"[SYSTEM]\n{sys_inst}\n\n[CONTEXT]\n{context}\n\n[USER]\n{user_q}\n\n[ASSISTANT]\n"

# ------------ Llama call (non-interactive) ------------
def call_llama(prompt: str, max_tokens: int) -> str:
    payload = {
        "model": "qwen3",  # name is ignored by llama.cpp server
        "messages": [{"role":"user","content": prompt}],
        "temperature": 0.2,
        "top_p": 0.9,
        "max_tokens": max_tokens,
        "stream": False
    }
    r = requests.post(LLAMA_URL, json=payload, timeout=600)
    if r.status_code != 200:
        raise RuntimeError(f"llama server error {r.status_code}: {r.text[:2000]}")
    data = r.json()
    return data["choices"][0]["message"]["content"]

# ------------ Single-user lock ------------
class Lock:
    def __enter__(self):
        self.f = open(LOCK_PATH, "w")
        try:
            fcntl.flock(self.f, fcntl.LOCK_EX | fcntl.LOCK_NB)
        except BlockingIOError:
            raise HTTPException(status_code=409, detail="Oogway is busy. Please try again in a moment.")
        return self
    def __exit__(self, exc_type, exc, tb):
        try:
            fcntl.flock(self.f, fcntl.LOCK_UN)
        finally:
            self.f.close()

# ------------ FastAPI app ------------
app = FastAPI(title="Oogway RAG API (Local)")

HTML = """
<!doctype html><html><head><meta charset="utf-8"/>
<title>Oogway (Local)</title>
<style>
body{font-family:system-ui,Segoe UI,Arial;margin:2rem;max-width:900px}
label{display:block;margin:.5rem 0 .25rem}
input,textarea,button,select{font:inherit;padding:.5rem;border:1px solid #ccc;border-radius:8px;width:100%}
button{cursor:pointer}
.row{display:flex;gap:1rem}
.row>*{flex:1}
.card{padding:1rem;border:1px solid #eee;border-radius:12px;box-shadow:0 2px 6px rgba(0,0,0,.05)}
.small{font-size:.9em;color:#666}
pre{white-space:pre-wrap}
</style>
</head><body>
<h1>Oogway — Local RAG</h1>
<div class="card">
  <div class="row">
    <div>
      <label>Invite Token</label>
      <input id="tok" placeholder="TRY-OUGWAY-LOCAL" value="TRY-OUGWAY-LOCAL">
    </div>
    <div>
      <label>Top-K</label>
      <input id="k" type="number" min="2" max="20" value="6">
    </div>
    <div>
      <label>Max tokens</label>
      <input id="mt" type="number" min="256" max="4096" value="2048">
    </div>
  </div>
  <div class="row">
    <label><input id="think" type="checkbox" checked> Allow <think> (think mode)</label>
  </div>
  <label>Question</label>
  <textarea id="q" rows="4" placeholder="Ask about your corpus…"></textarea>
  <div style="margin-top:.75rem"><button onclick="ask()">Ask</button></div>
</div>

<h3>Answer</h3>
<pre id="ans" class="card"></pre>
<p class="small">Single-user lock is enabled. If you see “busy”, wait for the current turn to finish.</p>

<script>
async function ask(){
  const tok=document.getElementById('tok').value.trim();
  const q=document.getElementById('q').value.trim();
  const think=document.getElementById('think').checked;
  const k=parseInt(document.getElementById('k').value||'6',10);
  const mt=parseInt(document.getElementById('mt').value||'2048',10);
  const ans=document.getElementById('ans');
  ans.textContent = 'Thinking…';
  try{
    const r = await fetch('/chat', {method:'POST', headers:{'Content-Type':'application/json'},
      body: JSON.stringify({token:tok, question:q, top_k:k, max_tokens:mt, think:think})});
    const j = await r.json();
    if(!r.ok){ ans.textContent = 'Error: ' + (j.detail || JSON.stringify(j)); return; }
    ans.textContent = j.answer;
  }catch(e){ ans.textContent = 'Error: '+e; }
}
</script>
</body></html>
"""

@app.get("/", response_class=HTMLResponse)
def home():
    return HTML

class ChatIn(BaseModel):
    token: str
    question: str
    top_k: Optional[int] = TOP_K
    max_tokens: Optional[int] = TOKEN_BUDGET
    think: Optional[bool] = True

@app.post("/chat")
def chat(body: ChatIn):
    if not body.token or not check_token(body.token):
        raise HTTPException(status_code=401, detail="Invalid or expired invite token.")
    if not body.question or not body.question.strip():
        raise HTTPException(status_code=400, detail="Empty question.")
    # single-user lock
    with Lock():
        q = body.question.strip()
        qv = embed_one(q)
        hits = retrieve(vec_literal(qv), int(body.top_k or TOP_K))
        if not hits:
            return JSONResponse({"answer": "No results found in context."})
        prompt = build_prompt(q, hits, bool(body.think))
        ans = call_llama(prompt, int(body.max_tokens or TOKEN_BUDGET))
        return JSONResponse({"answer": ans})

PY
chmod +x ~/Desktop/rag_server.py

5) Run the local server

python ~/Desktop/rag_server.py

Open: http://127.0.0.1:8000
– Paste the invite token TRY-OUGWAY-LOCAL.
– Ask a question.
– Toggle Think mode and adjust Top-K / Max tokens.
– Single-user lock is enforced; a second request while one is running gets a “busy”.

Notes / what this setup guarantees

• No mid-answer pauses: We call the llama server non-interactively per turn with a large token budget; the page waits and shows the full answer.
• One user at a time: File lock at /tmp/oogway.lock prevents overlap.
• Invite-only: A simple token in Postgres; you can mint more later (or make them one-time/expiring).
• Stays local: Everything runs on your box; llama weights are loaded once.

When you’re ready, we can add:
– logging into cog.turns for the dream loop,
– a file-upload endpoint to call your ingest script,
– or switch the web to a fancier UI.

Perfect — let’s do a super clean local trial with just:

1) llama.cpp in server mode (kept warm)
2) a tiny RAG API (retrieval → prompt → call llama server)
3) a minimal web page (chat box + think toggle + sliders)

No invites, no locks, no accounts. Paste-once, run, test.

0) Prep (env you already use)

source ~/ougway_env/venv/bin/activate

export GGML_CUDA=1
export QWEN="$HOME/ougway_env/llama.cpp/models/qwen3/qwen3-8b-q6_k.gguf"
export LLAMA_CTX=32768
export LLAMA_NGL=999
export LLAMA_BATCH=32

export TS_DSN="dbname=tokenspace user=darren host=localhost password=LittleOugway"
export EMBED_MODEL="nomic-ai/nomic-embed-text-v1.5"
export HF_HUB_OFFLINE=1

1) Start llama.cpp server (new terminal)

"$HOME/ougway_env/llama.cpp/build/bin/llama-server" \
  -m "$QWEN" -c "$LLAMA_CTX" -ngl "$LLAMA_NGL" -b "$LLAMA_BATCH" -t "$(nproc)" \
  --host 127.0.0.1 --port 8080

This exposes http://127.0.0.1:8080/v1/chat/completions (OpenAI-style).

2) Install deps for the API

pip install fastapi uvicorn requests sentence-transformers psycopg2-binary numpy

3) Paste-once: RAG API + Web page (single file)

cat > ~/Desktop/rag_server_simple.py <<'PY'
#!/usr/bin/env python3
import os, hashlib, numpy as np, psycopg2, requests
from fastapi import FastAPI, HTTPException
from fastapi.responses import HTMLResponse, JSONResponse

# ---------- Config ----------
DSN          = os.environ.get("TS_DSN", "dbname=tokenspace user=darren host=localhost password=LittleOugway")
LLAMA_URL    = os.environ.get("LLAMA_URL", "http://127.0.0.1:8080/v1/chat/completions")
EMBED_MODEL  = os.environ.get("EMBED_MODEL", "nomic-ai/nomic-embed-text-v1.5")
TOKEN_BUDGET = int(os.environ.get("RAG_MAX_TOKENS", "2048"))
TOP_K        = int(os.environ.get("RAG_K", "6"))

# ---------- Embeddings ----------
_USE_ST = False
_st = None
try:
    from sentence_transformers import SentenceTransformer
    _st = SentenceTransformer(EMBED_MODEL, trust_remote_code=True)
    dim = getattr(_st, "get_sentence_embedding_dimension", lambda: None)()
    _USE_ST = (dim == 768)
except Exception:
    _USE_ST = False

def embed_one(text: str) -> np.ndarray:
    if _USE_ST:
        v = _st.encode([text], normalize_embeddings=True)
        return np.asarray(v, dtype=np.float32)[0]
    h = hashlib.sha256(text.encode("utf-8")).digest()
    seed = int.from_bytes(h[:8], "big") % (2**31 - 1)
    rng = np.random.default_rng(seed)
    v = rng.normal(0.0, 1.0, 768).astype(np.float32)
    n = float(np.linalg.norm(v)); 
    if n > 0: v /= n
    return v

def vec_literal(v: np.ndarray) -> str:
    return "[" + ",".join(f"{x:.6f}" for x in v.tolist()) + "]"

# ---------- Retrieval ----------
def retrieve(qvec_lit: str, k: int):
    with psycopg2.connect(DSN) as conn, conn.cursor() as cur:
        cur.execute(f"""
            SELECT ch.chunk_id, left(ch.text, 900) AS preview, d.title, s.uri,
                   ch.embedding <=> %s::vector AS dist
            FROM content.chunks ch
            JOIN content.documents d ON d.doc_id = ch.doc_id
            LEFT JOIN content.sources s ON s.source_id = d.source_id
            ORDER BY dist ASC
            LIMIT %s;
        """, (qvec_lit, k))
        return cur.fetchall()

# ---------- Prompt ----------
def build_prompt(question: str, hits, think_mode: bool) -> str:
    ctx_blocks=[]
    for i,(cid,preview,title,uri,dist) in enumerate(hits,1):
        meta=[]
        if title: meta.append(f"title: {title}")
        if uri:   meta.append(f"path: {uri}")
        meta.append(f"chunk_id: {cid}  dist: {dist:.4f}")
        ctx_blocks.append(f"### Context {i} ({' | '.join(meta)})\n{preview}")
    context="\n\n".join(ctx_blocks)
    sys_inst=("You are a concise assistant. Use ONLY the provided context. "
              "If the answer is not in the context, say you do not find it.")
    user_q = question.strip()
    if not think_mode:
        user_q += "\n\nGive only the final answer — no <think> or hidden reasoning."
    return f"[SYSTEM]\n{sys_inst}\n\n[CONTEXT]\n{context}\n\n[USER]\n{user_q}\n\n[ASSISTANT]\n"

# ---------- Llama call (non-interactive) ----------
def call_llama(prompt: str, max_tokens: int) -> str:
    payload = {
        "model": "qwen3",
        "messages": [{"role":"user","content": prompt}],
        "temperature": 0.2,
        "top_p": 0.9,
        "max_tokens": max_tokens,
        "stream": False
    }
    r = requests.post(LLAMA_URL, json=payload, timeout=600)
    if r.status_code != 200:
        raise RuntimeError(f"llama server error {r.status_code}: {r.text[:800]}")
    data = r.json()
    return data["choices"][0]["message"]["content"]

# ---------- Web app ----------
app = FastAPI(title="Oogway RAG (Local Simple)")

HTML = """
<!doctype html><html><head><meta charset="utf-8"/>
<title>Oogway (Local Simple)</title>
<style>
body{font-family:system-ui,Segoe UI,Arial;margin:2rem;max-width:900px}
label{display:block;margin:.5rem 0 .25rem}
input,textarea,button{font:inherit;padding:.5rem;border:1px solid #ccc;border-radius:8px;width:100%}
button{cursor:pointer}
.row{display:flex;gap:1rem}
.row>*{flex:1}
.card{padding:1rem;border:1px solid #eee;border-radius:12px;box-shadow:0 2px 6px rgba(0,0,0,.05)}
pre{white-space:pre-wrap}
.small{font-size:.9em;color:#666}
</style>
</head><body>
<h1>Oogway — Local RAG (Simple)</h1>
<div class="card">
  <div class="row">
    <div>
      <label>Top-K</label>
      <input id="k" type="number" min="2" max="20" value="6">
    </div>
    <div>
      <label>Max tokens</label>
      <input id="mt" type="number" min="256" max="4096" value="2048">
    </div>
    <div>
      <label> </label>
      <label><input id="think" type="checkbox" checked> Think mode (allow <think>)</label>
    </div>
  </div>
  <label>Question</label>
  <textarea id="q" rows="4" placeholder="Ask about your corpus…"></textarea>
  <div style="margin-top:.75rem"><button onclick="ask()">Ask</button></div>
</div>

<h3>Answer</h3>
<pre id="ans" class="card"></pre>
<p class="small">This demo calls llama.cpp server non-interactively per turn (no mid-answer pauses).</p>

<script>
async function ask(){
  const q=document.getElementById('q').value.trim();
  const think=document.getElementById('think').checked;
  const k=parseInt(document.getElementById('k').value||'6',10);
  const mt=parseInt(document.getElementById('mt').value||'2048',10);
  const ans=document.getElementById('ans');
  if(!q){ ans.textContent='Please enter a question.'; return; }
  ans.textContent = 'Thinking…';
  try{
    const r = await fetch('/chat', {method:'POST', headers:{'Content-Type':'application/json'},
      body: JSON.stringify({question:q, top_k:k, max_tokens:mt, think:think})});
    const j = await r.json();
    if(!r.ok){ ans.textContent = 'Error: ' + (j.detail || JSON.stringify(j)); return; }
    ans.textContent = j.answer;
  }catch(e){ ans.textContent = 'Error: '+e; }
}
</script>
</body></html>
"""

@app.get("/", response_class=HTMLResponse)
def home():
    return HTML

@app.post("/chat")
async def chat(payload: dict):
    q = (payload.get("question") or "").strip()
    if not q:
        raise HTTPException(status_code=400, detail="Empty question.")
    k = int(payload.get("top_k") or TOP_K)
    mt = int(payload.get("max_tokens") or TOKEN_BUDGET)
    think = bool(payload.get("think") if "think" in payload else True)

    qv = embed_one(q)
    hits = retrieve(vec_literal(qv), k)
    if not hits:
        return JSONResponse({"answer": "No results found in context."})

    prompt = build_prompt(q, hits, think)
    try:
        ans = call_llama(prompt, mt)
        return JSONResponse({"answer": ans})
    except Exception as e:
        raise HTTPException(status_code=500, detail=str(e))

PY
chmod +x ~/Desktop/rag_server_simple.py

4) Run the local RAG server + page

python ~/Desktop/rag_server_simple.py

Open http://127.0.0.1:8000
– Ask a question
– Toggle Think mode
– Adjust Top-K and Max tokens

This uses your Postgres RAG + llama.cpp server with non-interactive calls (so no mid-answer pauses), while keeping the model loaded once in VRAM.

Notes

• You can tweak defaults via env vars: RAG_K, RAG_MAX_TOKENS, LLAMA_URL, etc.
• When you’re ready, we can add (in one shot): logging turns to Postgres, file upload → ingest, invite tokens, and a single-user lock.

You’re right — it didn’t start a web server. That file defines a FastAPI app, but we didn’t launch an ASGI server. Easy fix—no edits needed.

1) Make sure the llama.cpp server is running

(New terminal)

"$HOME/ougway_env/llama.cpp/build/bin/llama-server" \
  -m "$HOME/ougway_env/llama.cpp/models/qwen3/qwen3-8b-q6_k.gguf" \
  -c 32768 -ngl 999 -b 32 -t "$(nproc)" \
  --host 127.0.0.1 --port 8080

Quick check (new tab/terminal):

curl -s http://127.0.0.1:8080/v1/models | head

(Should print a JSON blob; if it errors, the model server isn’t up.)

2) Start the RAG API + web page with uvicorn

From your venv:

# either of these works — pick one:

# a) run from the Desktop folder
cd ~/Desktop
python -m uvicorn rag_server_simple:app --host 127.0.0.1 --port 8000

# b) run from anywhere with --app-dir
python -m uvicorn rag_server_simple:app --host 127.0.0.1 --port 8000 --app-dir ~/Desktop

You should see uvicorn logs like “Uvicorn running on http://127.0.0.1:8000”.

Then open: http://127.0.0.1:8000

3) If it still says “nope”

• Make sure you installed uvicorn (you likely did):
  pip install uvicorn
• If port 8000 is busy, try another:
  python -m uvicorn rag_server_simple:app --host 127.0.0.1 --port 7860 --app-dir ~/Desktop
• If the page loads but answers error out, verify the llama server is up (step 1 curl).

That’s it—once uvicorn is running, your local web page should work.

Short answer: yes—right now it’s strict RAG.
In the simple web server we spun up, the prompt literally says:

> “Use ONLY the provided context. If the answer is not in the context, say you do not find it.”

So it will ignore its built-in knowledge on purpose. That’s why you’re seeing “it won’t answer” unless the DB has it.

Your options (no code changes required right now)

• Let it use base knowledge this turn: just add this sentence to your question:
  > “If the context is missing, answer from your general knowledge.”

Example:
> “What is a vector equilibrium? If the context is missing, answer from your general knowledge.”

• Prefer RAG but allow fallback:
  Use this stronger instruction:
  > “Prefer the provided context; if it’s insufficient or missing, answer from your general knowledge. Make that explicit at the top (‘From context’ vs ‘From general knowledge’).”

• Pure base model (no RAG) for a turn:
  Ask:
  > “Ignore retrieved context and answer from your general knowledge only.”

(Those work because our server passes your full question into the prompt block.)

If you want it permanent (when you’re ready for a code update)

We can flip to a “RAG-preferred with fallback” policy in the server so you don’t have to type extra words:

• New default system rule: “Prefer context; if not found, answer from general knowledge and say so.”
• Optional checkbox in the web UI: Strict RAG (on/off).

We can also add tiny heuristics (e.g., if top-K distances are weak or there are 0 hits) → auto fall back to base knowledge.

TL;DR

It’s behaving exactly as configured: strict context-only.
For now, add that one line to your question to allow fallback. When you’re ready, I’ll give you a single paste-once update that makes “RAG-preferred with fallback” the default and adds a Strict RAG toggle to the page.