private-gpt

zylon-ai/private-gpt is an open-source project by zylon-ai: Complete API layer for private AI applications on local models: RAG, skills, tools, MCP, text-to-sql, and more. Works with any OpenAI-compatible inference server.. Indexed here so it can be found — not resold.

It is free. Get it from the upstream repository: https://github.com/zylon-ai/private-gpt

From the project's own README (excerpt, reproduced for discovery under its Apache-2.0 license):

PrivateGPT is the open-source API layer that turns local models into production AI applications.

---

Running a model locally is only the first step. To build useful AI applications you need a set of higher-level building blocks. PrivateGPT provides that layer as an open-source API following the Claude API model — so you can build private AI products without rebuilding the same backend primitives from scratch, and without depending on cloud APIs.

Production-tested: PrivateGPT powers Zylon, the on-premise AI platform providing Private AI to enterprises across the globe.

> PrivateGPT does not run models itself. It connects to any OpenAI-compatible inference server via OPENAIAPIBASE. If it implements /v1/chat/completions and /v1/models, it works.

PrivateGPT ships a built-in workbench UI for testing and demos, available at /ui. The API is the actual product.

---

What PrivateGPT gives you • Standard messages API (streaming, async, token counting) • File and artifact ingestion • Retrieval with citations and agentic RAG • Built-in tools mirroring the Claude API (web search, web fetch, code execution) • Custom tools and MCP connectors • Structured access to databases and CSVs • Embeddings and orchestration

---

Quickstart

> For Docker, full installation options, and model configuration see the full Quickstart guide.

Prerequisites: You need a running OpenAI-compatible LLM server. Ollama is the easiest starting point.

1. Install PrivateGPT

2. Start your LLM server

3. Run PrivateGPT

4. Open the UI

Go to http://localhost:8080/ui. The API is at http://localhost:8080 and follows the Anthropic API spec.

The UI is useful for: • Sending messages. • Selecting models from /v1/models. • Uploading documents. • Testing retrieval with…