NPM installation issue — Part 2

Yeah, that combo of:

> Stable diffusion model failed to load
> and
> OSError: Can't load tokenizer for 'openai/clip-vit-large-patch14'

after a clean venv rebuild tells us something pretty specific:

• A1111 itself is fine
• Your SD 1.5 checkpoints are fine
• The failure is entirely in trying to pull that CLIP tokenizer from HuggingFace
• And Transformers ends up with no valid vocab file, so the model load aborts

At this point it’s one of three things:

1. Something local is still shadowing that model id
2. The HuggingFace cache for it is broken (we already cleared once)
3. Or ouqway can’t actually fetch that model cleanly from HF (network / TLS / whatever)

Rather than keep smacking the same code, here’s a concrete step-by-step path you can follow and interpret on your own.

---

1️⃣ Double-check no local folder is shadowing openai/clip-vit-large-patch14

From the webui dir:

cd ~/ai/stable-diffusion-webui
find . -maxdepth 5 -type d \( -name "openai" -o -name "clip-vit-large-patch14" \)

Interpretation:

• If this prints only stuff under ./venv/lib/python3.10/site-packages → that’s fine, ignore.
• If it prints something like ./openai or ./models/openai (outside venv/) → that can confuse from_pretrained.

If you see a top-level ./openai or similar that clearly isn’t the Python package, temporarily rename it:

mv openai openai.bak   # adjust path if needed

Leave the venv’s site-packages/openai alone.

---

2️⃣ Check whether ougway can actually reach HuggingFace

Still on ougway:

curl -I https://huggingface.co/openai/clip-vit-large-patch14

You’ll either see:

• HTTP/2 200 or a redirect → network OK, HF is reachable
• Some TLS error, timeout, name lookup issue → network problem, Transformers will never be able to fetch that model automatically

If the curl fails, that’s your root cause: the machine simply can’t pull from HF, and every attempt to auto-download the tokenizer will die. In that case the “fix” is either:

• fix whatever is blocking HF (DNS, firewall, ESM, etc.), or
• manually download the tokenizer files on another machine and copy them into ~/.cache/huggingface on ougway (totally doable, just more manual).

---

3️⃣ If HF is reachable, but the tokenizer still errors

Then likely it’s a compatibility mismatch:

• HF model layout changed
• A1111’s pinned transformers version doesn’t parse it correctly
• so vocab_file in the config ends up as None

In that case, inside the A1111 venv (after the fresh rebuild you just did):

cd ~/ai/stable-diffusion-webui
source venv/bin/activate

pip install --upgrade "transformers>=4.42.0" "huggingface_hub>=0.23.0"
rm -rf ~/.cache/huggingface

Then re-run the test:

python - << 'EOF'
from transformers import CLIPTokenizer
print("Downloading CLIP tokenizer...")
tok = CLIPTokenizer.from_pretrained("openai/clip-vit-large-patch14")
print("Done, vocab size:", tok.vocab_size)
EOF

If that finally prints:

Downloading CLIP tokenizer...
Done, vocab size: 49408

…then CLIP is fixed and cached. Restart:

./webui.sh --xformers

Select v1-5-pruned.safetensors, try a simple prompt, and it should now happily generate.

---

4️⃣ Reality check / fallback

If:

• curl to HuggingFace fails, or
• even with upgraded transformers and a clear cache, the tokenizer still can’t load,

then you have two pragmatic options:

1. We work around this by manually seeding the tokenizer.
   – Download the openai/clip-vit-large-patch14 files on another machine using a browser or huggingface-cli
   – Copy them into ougway’s ~/.cache/huggingface/hub/models--openai--clip-vit-large-patch14/...
   – Once those files exist locally, A1111/transformers will happily use them without trying to pull from the network.

2. Or we sidestep this specific CLIP dependency entirely.
   Since SD 1.5 itself doesn’t inherently need HF’s CLIP repo, we can:
   – temporarily disable any features/extensions that call that model, or
   – use a different UI (e.g. ComfyUI) that bundles only the CLIP it actually needs for SD 1.5.

---

You’ve done all the “right” moves so far:

• fresh venv
• proper 3.10
• A1111 from git
• checkpoints in place

What’s blocking you now is purely:
> something about how this environment is fetching or parsing openai/clip-vit-large-patch14.

The shortest path from here is:

• run that curl check yourself
• decide whether you’d rather:
• upgrade transformers and let HF handle it, or
• go fully manual (download-and-copy tokenizer once and forget about it).

If you tell me:

• whether curl -I https://huggingface.co/openai/clip-vit-large-patch14 works,
  I can give you a very specific “do this next” branch instead of broad options.

Yeah, that TCMalloc message is nothing to worry about. 😊

That line:

> Cannot locate TCMalloc. Do you have tcmalloc or google-perftool installed on your system? (improves CPU memory usage)

is just AUTOMATIC1111 being chatty. It’s saying:

• “If you install this extra library, I can use a slightly better memory allocator.”
• It is optional.
• It does not block Stable Diffusion from working.
• It has nothing to do with the CLIP tokenizer issue.

By default, most people just ignore it.

---

If you want to get rid of the warning (optional)

You can install it like this:

sudo apt install google-perftools

That package provides libtcmalloc, and on a lot of systems A1111 will auto-detect it next run and stop complaining.

But it is purely a nice-to-have.
It does not affect GPU rendering or whether SD runs.

---

What I’d do right now

Since you:

• upgraded transformers
• wiped/reinstalled the venv/webui
• are running the installer again

I’d just let this TCMalloc message go, finish the install, then:

1. Start the web UI
2. Make sure v1-5-pruned.safetensors is selected as checkpoint
3. Try a simple prompt again
4. See if the CLIP error still appears or not

If SD generates an image, TCMalloc is officially a non-issue.

If the CLIP tokenizer error shows up again, we’ll treat that as the next (and only) problem to solve and I’ll walk you through a manual “download tokenizer once, drop it in the right place” fix so we can force it to shut up.

For now:
✅ You can safely ignore the TCMalloc warning and keep going.