User Tools

Site Tools


ai-deployment-english

This is an old revision of the document!


Deployment: Local AI Stack (Dental-Billing Assistant) - Part 1: Setup (EN)

Goal: build the full stack (LM Studio + Open WebUI + ComfyUI) from scratch on Nobara (Fedora-based), x86-64. All components run locally - no cloud access.

0. Prerequisites

Requirement Details
—–
GPU NVIDIA, 16-19 GB VRAM (Qwen3 27B)
Driver current NVIDIA driver, CUDA check: ``nvidia-smi''
Podman ``podman –version'' (Open WebUI runs as a container)
User ``rootai``, home ``/home/rootai''
Ports 1234 (LM Studio), 3000 (Open WebUI), 8188 (ComfyUI)

1. Install LM Studio (AppImage)

1. Download the AppImage into ``/home/rootai/Downloads``.

2. Make it executable:

  ``chmod +x ~/Downloads/lm-studio-*.AppImage''
  

3. Nobara/Fedora compatibility (FUSE): if the AppImage does not start:

  ``./lm-studio-*.AppImage --appimage-extract``
  ``./lm-studio-*/AppRun''

4. Load the model: LM Studio → Models → download Qwen3 27B (GGUF) (~16 GB).

  Models and settings live in ``~/.lmstudio`` - the AppImage itself is replaceable, the data stays.

5. Link the lms CLI globally (symlink to the AppImage):

  ``sudo ln -sf /home/rootai/Downloads/lm-studio-*.AppImage /usr/local/bin/lms''
  Note: after every AppImage update with a new filename, recreate the symlink (see troubleshooting).

Test:

  ``lms ls''

2. Start LM Studio Server + test

  # start the server (LM Studio -> Developer -> Start Server, port 1234)
  lms load qwen/qwen3.8-27b
  # API test
  curl -s http://127.0.0.1:1234/v1/models

Expected: JSON list containing ``qwen3.8-27b``.

3. Open WebUI (Podman container)

  # pull the image
  podman pull ghcr.io/open-webui/open-webui
  # start the container (name: open-webui, port 3000, own data volume)
  podman run -d --name open-webui -p 3000:8080 -v open-webui-data:/app/backend/data --restart unless-stopped ghcr.io/open-webui/open-webui

Setup in the browser (``http://<server>:3000``):

1. LLM connection: Settings → Connections → OpenAI API

2. Embedding model: Settings → Models → set ``nomic-embed-text-v1.5'' as embedding model 3. Create user: ``artz'' with read access to the knowledge base 4. Important: disable the built-in tool Ask User - otherwise the model asks follow-up questions

Test:

  podman ps | grep open-webui
  curl -s http://127.0.0.1:3000

4. ComfyUI (Z-Image)

Start (background, log to ~/comfyui.log):

  nohup ~/comfyui-env/bin/python ~/comfyui/main.py --listen --port 8188 >> ~/comfyui.log 2>&1 &

Test:

  curl -s http://127.0.0.1:8188/system_stats

Connect in Open WebUI: Settings → Connections → ComfyUI

Log on problems: ``tail -f ~/comfyui.log''

5. Auto-start on boot

Script: ``/home/rootai/start-ai.sh''

  #!/bin/bash
  # wait for system services and the LM Studio server
  sleep 15
  # automatically load the Qwen model into LM Studio
  lms load qwen/qwen3.8-27b
  # give it a few seconds to fully initialize
  sleep 10
  # restart Open WebUI
  podman restart open-webui

Start order matters: LM Studio → load model → Open WebUI.

6. Final smoke test

  # 1. LM Studio API
  curl -s http://127.0.0.1:1234/v1/models
  # 2. Open WebUI reachable
  curl -sI http://127.0.0.1:3000 | head -1
  # 3. ComfyUI reachable
  curl -s http://127.0.0.1:8188/system_stats | head -c 100

1. Browser → ``http://<server>:3000'' → login ``artz'' 2. Ask a free question to ``qwen3.8-27b'' → answer without RAG citations (RAG = Part 2)

7. Troubleshooting

Symptom Cause Fix
—–
AppImage does not start FUSE missing (Fedora/Nobara) use ``–appimage-extract``, then run ``AppRun''
lms: command not found broken symlink, old filename ``sudo ln -sf /path/to/new/AppImage /usr/local/bin/lms''
WebUI cannot find the model wrong base URL use ``host.docker.internal:1234/v1, not ``localhost
Container will not restart wrong container name check real name with ``podman ps -a''
Port already in use stale process ``sudo lsof -i :1234'' / ``:3000'' / ``:8188''

Deployment: Local AI Stack (Dental-Billing Assistant) - Part 2: RAG Knowledge Base (EN)

Goal: add the 5 billing documents as the knowledge base Zahnarztabrechnung in Open WebUI and create the model Zahn-Abrechnungs-Assistent.

1. Set the embedding model (prerequisite)

RAG does not work without an embedding model.

1. Open WebUI → Settings → Models 2. Embedding model: ``nomic-embed-text-v1.5'' (built-in, no Ollama/LM Studio needed) 3. Verify: the models list shows the embedding model

Fallback if German retrieval is poor: install ``bge-m3'' via Ollama and set it as the embedding model.

2. Prepare the PDFs

# Document Source Note
—–
1 SGB V (full text) gesetze-im-internet.de 577 pages, lots of unrelated material (pharmacy, hospital) - trim to Part 3 (§§ 55-66) if citations show noise
2 KZBV FZ-Kompendium 2025 KZBV statutory co-payments (Festzuschüsse)
3 GOÄ/GOZ pkv.de private fee schedules
4 BEL-Preisliste 2025 kvzbw.de -
5 BEB-Preisliste 2024 itzgmbh.de image-based (scanned) - OCR required, see section 3

Pre-check for each PDF:

  • text can be selected/copied → OK
  • images only → OCR problem (BEB)

3. Enable OCR for image-based PDFs (BEB)

Symptom without OCR: BEB citations missing or nonsense, other PDFs work fine.

Fix: 1. Open WebUI → Admin → Settings → Documents 2. turn ON the option PDF Extract Images (OCR) 3. re-upload / re-index the BEB file (delete the old document first, otherwise the stale empty index stays)

4. Create the knowledge base

1. Open WebUI → Knowledge → Create, Name: ``Zahnarztabrechnung'' 2. Add documents: upload all 5 PDFs 3. wait until every document shows status processed (no errors) 4. Access: user ``artz'' → Read access on the knowledge base

Knowledge-base test (no model prompt yet):

  • new chat session, attach the Zahnarztabrechnung KB
  • test question: “What statutory co-payment (Festzuschuss) applies to a crown?”
  • expected: answer with source citations from the FZ-Kompendium

5. Create the Zahn-Abrechnungs-Assistent model

1. Open WebUI → Settings → Models → Create 2. Name: ``Zahn-Abrechnungs-Assistent'' 3. Base model: ``qwen3.8-27b'' (LM Studio) 4. System prompt:

  You are an expert in German dental and dental-technician billing
  (GOÄ/GOZ, FZ-Kompendium, BEL/BEB, SGB V). Answer strictly on the basis
  of the documents provided. Give concrete position numbers, prices, the
  edition (e.g. BEB 2024, BEL 2025) and the source. If a topic is not
  covered in the documents, say so clearly. This is not legal advice.

5. Important: disable the built-in tool Ask User for this model - otherwise it asks follow-up questions (“Which tooth number?”) instead of answering from the documents

6. End-to-end test

1. Browser → ``http://<server>:3000'' → login ``artz'' 2. Select model Zahn-Abrechnungs-Assistent 3. Sample test questions (German):

  • “Which GOZ position applies for a composite filling on an incisor?” → expect GOZ 2010 with source
  • “What does a full-ceramic crown cost per BEB?” → expect BEB position + price + edition 2024
  • “What is the statutory co-payment for inlays?” → expect FZ-Kompendium 2025

4. Checklist per answer:

  • source citations visible (document + section)
  • concrete position number + price
  • edition stated (BEB 2024 / BEL 2025 / FZ 2025)
  • for missing info: a clear statement instead of a hallucination

7. Troubleshooting

Symptom Cause Fix
—–
Answer without sources, made up RAG not connected check model prompt, attach the KB in chat
BEB gives no sensible citations PDF is image-based enable OCR (section 3), re-index the document
SGB V citations show pharmacy/hospital noise 577-page full text too broad trim SGB V to §§ 55-66, replace the document, re-index
Slow answers on large PDFs index size normal, cache warms up; optionally check chunk size in Settings
Model asks instead of answering Ask User tool active disable it in the model settings
ai-deployment-english.1789566170.txt.gz · Last modified: by misa