60 lines
2.4 KiB
Markdown
60 lines
2.4 KiB
Markdown
---
|
|
name: tier2_setup_retro
|
|
description: Tier 2 knowledge database setup retrospective 2026-05-14
|
|
type: project
|
|
---
|
|
|
|
## Setup Session: 2026-05-14
|
|
|
|
**Goal:** Set up Tier 2 local knowledge database (kiwix-serve + zim-llm)
|
|
|
|
### What Worked ✅
|
|
|
|
1. **Docker installation** - Successfully installed Docker Engine on Debian
|
|
2. **User permissions** - Added to docker group, can run without sudo
|
|
3. **zim-llm setup** - Repository cloned, venv created, dependencies installed
|
|
4. **CPU-only PyTorch** - Corrected initial CUDA download issue for ROCm machine
|
|
5. **Zeal installation** - Installed via apt, configured with setup script
|
|
|
|
### What Failed ❌
|
|
|
|
1. **ZIM file downloads** - kiwix.org servers slow/unreachable
|
|
- 134GB Stack Overflow ZIM was corrupted/truncated
|
|
- Smaller files also stalled at 0B download
|
|
- aria2c with 16 connections didn't help
|
|
|
|
2. **External docset hosting** - Kapeli/Dash discontinued public hosting
|
|
- dash-docs.github.io returns 404s
|
|
- kapeli.com/docsets redirects to homepage
|
|
- Zeal-Docs community feeds repo doesn't exist
|
|
|
|
3. **Proxy bypass** - proxychains helped reach GitHub but docset URLs still return HTML
|
|
|
|
### Current State 📊
|
|
|
|
- **Zeal**: Installed, basic config ready, 1 placeholder docset
|
|
- **zim-llm**: Dependencies installed, config points to `/home/tr/kiwix/zim`
|
|
- **kiwix-serve**: docker-compose.yml created, waiting for ZIM files
|
|
- **No ZIM files**: Library directory empty
|
|
|
|
### Lessons Learned 💡
|
|
|
|
1. **Large downloads need integrity verification** - Don't trust files without checksums
|
|
2. **Network issues may require torrents** - Kiwix provides .torrent files
|
|
3. **External hosting can disappear** - Kapeli discontinued docset hosting
|
|
4. **CPU fallback works** - zim-llm runs fine without GPU for embedding generation
|
|
|
|
### Recommended Next Steps
|
|
|
|
1. **Try torrents** - Download ZIM files via torrent (more reliable for large files)
|
|
2. **Start with small ZIM** - Try Project Gutenberg or compact Wikipedia (~1-5GB)
|
|
3. **Use Zeal as primary** - For API docs, Zeal + manual docset addition works
|
|
4. **Defer zim-llm** - Can revisit when network improves or torrents available
|
|
|
|
### Files Created
|
|
|
|
- `/home/tr/kiwix/docker-compose.yml` - kiwix-serve config
|
|
- `/home/tr/zim-llm/config.json` - Points to `/home/tr/kiwix/zim`
|
|
- `/home/tr/zim-llm/requirements.txt` - Updated with CPU PyTorch
|
|
- `/home/tr/.local/bin/setup-zeal-docsets.sh` - Zeal setup automation
|