How to Set up On-Prem Qsirch RAG

Last modified date: 2025-10-28

Applicable Products

Qsirch 6.0.0 (or later) on QuTS hero platform

On-Premise RAG Search

For advanced users and enterprises who require full data privacy or offline AI, Qsirch runs RAG on locally hosted LLMs. Everything is performed on your NAS or connected hardware - no data leaves your environment.

Hardware and Configuration

Hardware recommendations

- NAS: TS-h1290FX (https://www.qnap.com/zh-tw/product/ts-h1290fx)

- GPUs: RTX 6000 Ada and RTX 4000 Ada

- QuTS hero 5.2.1 or later versions

- LLM Core is installed and enabled

- Go to Control Panel > Hardware > Hardware Resources > Resource Use > Container Station mode

LLM Core (app) - On-Prem RAG supported models

LLM Core brings large language models directly to your NAS, letting Qsirch run RAG privately and efficiently without cloud services. With built-in model management, you can download, organize, and switch between local models, then use them in Qsirch for generation. Everything runs on your hardware for full control over performance and privacy.

Currently supported local models (list updates over time; check in-app for the latest models):

Display name	Params	Quantization	Size
DeepSeek R1 7B	7.628B	Q4_K_M	4.70 GB
DeepSeek R1 14B	14.8B	Q4_K_M	9.00 GB
Gemma3 12B	12.2B	Q4_K_M	8.10 GB
DeepSeek R1 8B	8.03B	Q4_K_M	4.90 GB
DeepSeek R1 32B	32.8B	Q4_K_M	20.00 GB
Gemma3 1B	1.0B	Q4_K_M	0.815 GB
Gemma3 4B	4.38B	Q4_K_M	3.30 GB
Gemma3 27B	27.4B	Q4_K_M	18.00 GB
gpt-oss 20B	20.98B	MXFP4	14.00 GB
Mistral 7B	7.25B	Q4_K_M	4.10 GB
Microsoft Phi-4 14B	14.7B	Q4_K_M	9.10 GB

How to set up On-premise service in Qsirch

Go to Settings > Administrator setting > RAG Search > On-premise Service.
If the LLM Core is not installed/enabled, click Enable now to go to the app center and install/enable LLM Core.
If the GPU card is not set to Container Station mode, click Enable now to go to Control Panel and set the GPU to Container Station mode.