NPM installation issue — Part 2

Yeah, this thing is just hard-locking on that CLIP tokenizer. The rest of the stack is actually fine; the failure is 100% here:

> CLIPTokenizer.from_pretrained('openai/clip-vit-large-patch14')

And it never finishes loading a model because of it.

Also, to answer this:

> “Also, who is the OpenAI directory? I don't see it anywhere in the file manager.”

That "openai/clip-vit-large-patch14" is not a directory in your tree.
It’s a HuggingFace model ID (owner: openai, repo: clip-vit-large-patch14).
Transformers tries to download that from the internet unless you point it at a local path.

Since the automatic HuggingFace route is being a pain, the cleanest fix now is:

> Bypass HuggingFace completely and point SD at a local CLIP tokenizer.

We’ll force it to use a directory you control.

---

Plan

1. Manually download the CLIP tokenizer files once
2. Put them in a known local folder on ougway
3. Patch SD’s encoder code to use that folder instead of "openai/clip-vit-large-patch14"

After that, it won’t try to talk to HuggingFace at all.

---

1️⃣ Create a local CLIP folder on ougway

On ougway:

mkdir -p /home/darren/ai/clip-vit-large-patch14

That will be our “fake HuggingFace model” directory.

---

2️⃣ Download the tokenizer files (on any machine with a browser)

On any machine that can reach the web:

1. Open your browser
2. Go to the HuggingFace repo:
   https://huggingface.co/openai/clip-vit-large-patch14
3. Click Files and versions
4. Download these files (names may be exactly like this):

• tokenizer.json
• vocab.json
• merges.txt
• special_tokens_map.json
• tokenizer_config.json

You don’t need the model weights, just the tokenizer stuff.

5. Move those files onto ougway into:

/home/darren/ai/clip-vit-large-patch14/

You can use a USB stick, scp, shared folder, whatever you like.
End state should be:

ls /home/darren/ai/clip-vit-large-patch14
# something like:
# merges.txt  vocab.json  tokenizer.json  tokenizer_config.json  special_tokens_map.json

---

3️⃣ Test CLIPTokenizer against the local folder

Now on ougway:

cd ~/ai/stable-diffusion-webui
source venv/bin/activate  # if not already active

Then:

python - << 'EOF'
from transformers import CLIPTokenizer
print("Trying local CLIP tokenizer...")
tok = CLIPTokenizer.from_pretrained("/home/darren/ai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF

✅ If that prints something like:

Trying local CLIP tokenizer...
Done, vocab size: 49408

…then the local tokenizer is good.

If it errors here, it means the file set is incomplete or in the wrong place — but at least it’ll be a direct, localized error instead of that HF model ID mess.

---

4️⃣ Patch SD to use the local path instead of the HF ID

Now we tell SD:

> “Stop using openai/clip-vit-large-patch14 by name.
> Use /home/darren/ai/clip-vit-large-patch14 as your tokenizer.”

From the SD webui dir:

cd ~/ai/stable-diffusion-webui
nano repositories/stable-diffusion-stability-ai/ldm/modules/encoders/modules.py

Look for this line (or very similar) in FrozenCLIPEmbedder.__init__:

version = config.get("version", "openai/clip-vit-large-patch14")

Change it to:

version = config.get("version", "/home/darren/ai/clip-vit-large-patch14")

We’re only changing the default value.
Save and exit:

• Ctrl+O, Enter
• Ctrl+X

That’s it: now, if the config doesn’t override version, it will point straight at your local directory instead of that HuggingFace ID.

---

5️⃣ Restart A1111 and test

Back in the terminal where webui was running, stop it if needed (Ctrl+C), then:

cd ~/ai/stable-diffusion-webui
./webui.sh --xformers

Watch the logs:

• It will still say “Loading weights […] v1-5-pruned-emaonly.safetensors”
• But this time it should not spit out the openai/clip-vit-large-patch14 error during model creation

In the browser:

1. Go to http://127.0.0.1:7860
2. Make sure your checkpoint is set to v1-5-pruned-emaonly.safetensors or v1-5-pruned.safetensors
3. Try a simple prompt:

> gold torus on a wooden table, realistic, 512x512

If all went well:

• The progress bar moves
• No red CLIP error
• You finally get an image in the bottom-right panel

---

About that “openai directory” again

Just to close the loop:

• There is a Python package directory in the venv:
  /home/darren/ai/stable-diffusion-webui/venv/lib/python3.10/site-packages/openai
  That’s the OpenAI Python library, unrelated to CLIP.

• The openai/clip-vit-large-patch14 string is just how HuggingFace names model repos.
  We’ve now bypassed that entirely by giving it a real local path.

---

If you’d rather not mess with code and prefer a different UI (like ComfyUI), we can also switch to that — Comfy can be friendlier about explicit model paths. But the patch above is the shortest path to “this exact A1111 install works and never phones HuggingFace for CLIP again.”

Yeah, your memory there actually fits perfectly with what we’re seeing.

If we told this box “don’t ever talk to Hugging Face,” then:

• CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
• will always fail unless it finds a local copy.
• And right now it’s neither allowed to download from HF nor correctly pointed at a local folder → infinite faceplant.

So yes: that “don’t use HuggingFace” decision can absolutely be what’s biting us now.

---

What’s going on in plain terms

• "openai/clip-vit-large-patch14" is not a local directory.
• It’s a remote model name that transformers resolves via HuggingFace.
• If HF is blocked (hosts file, env var, firewall), the call fails and you get exactly:

> OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'

A1111 expects either:

1. it can reach HuggingFace to pull that model, or
2. you have a compatible CLIP tokenizer locally and the config points to it.

Right now you have:
❌ no HF access (by design)
❌ no local tokenizer path configured
→ so it dies.

---

At this point you’ve got two sane options, and you can pick your poison:

---

Option A – Temporarily let it talk to HuggingFace (easy mode)

This is the “eh, let it download once, then we lock it down again” path.

1. Check /etc/hosts for a manual block:

bash
grep -i huggingface /etc/hosts

If you see a line like:

text
127.0.0.1 huggingface.co

then edit hosts:

bash
sudo nano /etc/hosts

and put a # in front of that line to comment it out, e.g.:

text
#127.0.0.1 huggingface.co

Save and exit.

2. Check for HF offline env vars:

bash
env | grep -i HF_

If you see something like:

text
HF_HUB_OFFLINE=1

then just:

bash
unset HF_HUB_OFFLINE

3. Now retry the tokenizer test inside the venv:

“`bash
cd ~/ai/stable-diffusion-webui
source venv/bin/activate

python – << 'EOF'
from transformers import CLIPTokenizer
print("Downloading CLIP tokenizer…")
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF
“`

If that succeeds (you see Done, vocab size: ...), then:

• CLIP is cached locally.
• You can re-block HF afterward if you want (un-comment the hosts entry, re-set any env var).

4. Restart A1111:

bash
./webui.sh --xformers

and try a simple txt2img again.

---

Option B – Keep HF blocked and force a local tokenizer (offline mode)

If you don’t want this machine talking to HF at all, then what I outlined last message is the right path:

1. Create a local model folder:

bash
mkdir -p /home/darren/ai/clip-vit-large-patch14

2. On any machine that can reach the web, download the tokenizer files from
   https://huggingface.co/openai/clip-vit-large-patch14 (Files & versions):

• tokenizer.json
• vocab.json
• merges.txt
• special_tokens_map.json
• tokenizer_config.json

Then copy them into:

bash
/home/darren/ai/clip-vit-large-patch14

3. Test locally:

“`bash
cd ~/ai/stable-diffusion-webui
source venv/bin/activate

python – << 'EOF'
from transformers import CLIPTokenizer
print("Trying local CLIP tokenizer…")
tok = CLIPTokenizer.from_pretrained("/home/darren/ai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF
“`

4. Patch A1111’s encoder to use that local path instead of the HF name:

bash
cd ~/ai/stable-diffusion-webui
nano repositories/stable-diffusion-stability-ai/ldm/modules/encoders/modules.py

Find the line that looks like:

python
version = config.get("version", "openai/clip-vit-large-patch14")

Change it to:

python
version = config.get("version", "/home/darren/ai/clip-vit-large-patch14")

Save and exit.

5. Restart:

bash
./webui.sh --xformers

Now SD will never try to hit HuggingFace for that CLIP model; it’ll only use your local copy.

---

What I’d suggest pragmatically

Given where you’re at and that we’re already deep in the weeds:

• If you’re okay with a one-time outbound call → Option A is fastest.
• If you want ougway to remain strictly offline from HF → Option B is the clean, permanent fix.

Either way, the key point:
Yes, our “no HuggingFace” lockdown is very likely why this has been such a stubborn bastard.

Pick A or B, run through it, and then tell me whether:

• the CLIPTokenizer test succeeds, and
• SD finally spits out that gold torus instead of another novel-length traceback.

Yep, that makes total sense now — if we explicitly told Hugging Face “offline only, use cache,” and then later we deleted the cached openai/clip-vit-large-patch14 bits, we’ve basically put it in a chokehold:

• It’s forbidden to download,
• and the cache entry it expects isn’t there anymore,
• so it just keeps throwing that tokenizer error.

So the move now is:

> Temporarily undo “offline only,” let it pull this one model, then you can lock it back down if you want.

Let’s do this cleanly and visibly.

---

1️⃣ See what Hugging Face / transformers env vars are set

On ougway, in any terminal:

env | egrep 'HF_|TRANSFORMERS'

You’re looking for things like:

• HF_HUB_OFFLINE=1
• TRANSFORMERS_OFFLINE=1
• HF_HOME=...
• HF_HUB_CACHE=...

Those *_OFFLINE ones are the problem children.

---

2️⃣ Check if we hard-wired this in your shell config

Still in a terminal:

grep -ni 'HF_' ~/.bashrc ~/.profile ~/.bash_profile 2>/dev/null
grep -ni 'TRANSFORMERS_OFFLINE' ~/.bashrc ~/.profile ~/.bash_profile 2>/dev/null

If you see lines like:

export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1

that means every new shell is forcing offline mode.

Fix:

Open ~/.bashrc in nano:

nano ~/.bashrc

• Put a # in front of any HF_HUB_OFFLINE / TRANSFORMERS_OFFLINE lines, e.g.:

bash
# export HF_HUB_OFFLINE=1
# export TRANSFORMERS_OFFLINE=1

• Save & exit (Ctrl+O, Enter, Ctrl+X)

Then reload your shell config:

source ~/.bashrc

Now confirm they’re gone from this shell:

env | egrep 'HF_|TRANSFORMERS'

You should not see HF_HUB_OFFLINE or TRANSFORMERS_OFFLINE anymore.

---

3️⃣ See what’s actually in the HF cache for CLIP

Check whether the clip-vit-large-patch14 model is cached or not:

ls -R ~/.cache/huggingface/hub 2>/dev/null | grep -i 'clip-vit-large-patch14' || echo "no clip-vit-large-patch14 in cache"

If you get “no clip-vit-large-patch14 in cache”, that’s fine — it just confirms we really do need to download it once.

---

4️⃣ Try the tokenizer download again (now allowed to go online)

From your SD webui dir:

cd ~/ai/stable-diffusion-webui
source venv/bin/activate

Then run:

python - << 'EOF'
from transformers import CLIPTokenizer
print("Downloading CLIP tokenizer (online allowed)...")
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF

If this succeeds, you’ll see something like:

Downloading CLIP tokenizer (online allowed)...
Done, vocab size: 49408

At that point:

• the tokenizer is in the local cache
• A1111 will stop complaining about it
• and you can re-enable offline-only later if you really want

If it still fails, the error text should now be more honest about whether it’s a network / TLS / DNS issue instead of that generic “can’t load tokenizer” message.

---

5️⃣ Restart A1111 and test

Once the snippet succeeds:

cd ~/ai/stable-diffusion-webui
./webui.sh --xformers

Then in the browser:

1. Go to http://127.0.0.1:7860
2. Make sure v1-5-pruned-emaonly.safetensors or v1-5-pruned.safetensors is selected
3. Try your prompt again, e.g.:

> gold torus on a wooden table, realistic, 512x512

You should not see the openai/clip-vit-large-patch14 error anymore; you should see an image.

---

6️⃣ If you want to go back to “cache-only” afterward

Once CLIP is safely cached, you can re-enable offline mode by adding back, e.g., in ~/.bashrc:

export HF_HUB_OFFLINE=1
export TRANSFORMERS_OFFLINE=1

Then:

source ~/.bashrc

Future runs will then:

• still use the cached clip-vit-large-patch14
• and won’t try to hit HuggingFace for it again.

---

If you run:

env | egrep 'HF_|TRANSFORMERS'

and paste what you see, I can tell you exactly which line is doing the blocking and what to comment out.

Yup, that’s the smoking gun right there. 😄

TRANSFORMERS_OFFLINE=1
HF_HUB_OFFLINE=1
HF_HOME=/mnt/storage/hf-cache
HF_HUB_DISABLE_TELEMETRY=1

Those two:

• TRANSFORMERS_OFFLINE=1
• HF_HUB_OFFLINE=1

are exactly what’s telling the machine:

> “Never talk to Hugging Face, only use what’s already cached.”

Which is fine until you delete the cache for openai/clip-vit-large-patch14… then you get the endless “can’t load tokenizer” loop you’ve been stuck in.

Let’s open the gate just long enough to download this one tokenizer, then you can slam it shut again if you want.

---

1️⃣ Temporarily disable offline mode in this shell

From your stable-diffusion-webui directory:

cd ~/ai/stable-diffusion-webui

# Turn off offline mode for THIS shell only
unset HF_HUB_OFFLINE
unset TRANSFORMERS_OFFLINE

# Sanity check
env | egrep 'HF_|TRANSFORMERS'

You should still see:

• HF_HOME=/mnt/storage/hf-cache
• HF_HUB_DISABLE_TELEMETRY=1

…but not HF_HUB_OFFLINE or TRANSFORMERS_OFFLINE anymore.

---

2️⃣ Download the CLIP tokenizer now that it’s allowed online

Activate the venv and run the test:

source venv/bin/activate

python - << 'EOF'
from transformers import CLIPTokenizer
print("Downloading CLIP tokenizer (online allowed)...")
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF

If this works, you’ll see:

Downloading CLIP tokenizer (online allowed)...
Done, vocab size: 49408

That means:

• the tokenizer is now cached under /mnt/storage/hf-cache (your HF_HOME)
• future runs can use it offline just fine

If it errors, paste just that traceback next time, but I expect this to finally succeed now that offline mode is lifted.

---

3️⃣ Restart A1111 and generate something

Kill the running webui if it’s still up (Ctrl+C in that terminal), then:

cd ~/ai/stable-diffusion-webui
./webui.sh --xformers

In the browser (http://127.0.0.1:7860):

1. Make sure your checkpoint is one of the v1.5 ones (e.g. v1-5-pruned-emaonly.safetensors).
2. Try a simple prompt again, e.g.:

> gold torus on a wooden table, realistic, 512x512

If the CLIP tokenizer is now cached correctly, you should:

• not see the openai/clip-vit-large-patch14 error
• actually get an image out

---

4️⃣ (Optional) Re-enable “cache-only” afterward

Once you’ve confirmed SD works and CLIP is cached, you can put the offline flags back permanently if you like.

They’re probably set in ~/.bashrc or ~/.profile. To see where:

grep -ni 'HF_HUB_OFFLINE\|TRANSFORMERS_OFFLINE' ~/.bashrc ~/.profile ~/.bash_profile 2>/dev/null

If you want them back:

echo 'export HF_HUB_OFFLINE=1' >> ~/.bashrc
echo 'export TRANSFORMERS_OFFLINE=1' >> ~/.bashrc
source ~/.bashrc

But I’d wait until after you’ve confirmed SD is working cleanly.

---

So next concrete step for you:

• run unset for those two vars
• run the CLIPTokenizer Python snippet
• restart webui and try “gold torus” again

My bet: that finally breaks the loop.