
A full-stack AI chat client that connects to any OpenAI-compatible local inference server — LM Studio, Ollama, llama.cpp, vLLM — over the standard /v1/chat/completions endpoint, with client-side MCP tool orchestration, AI image generation, image upload, and optional end-to-end AES message encryption.
Breaking change
LM Studio's native /api/v1/chat API and the OpenAI-compatible /v1/responses/create (Responses API) endpoint have been removed — the modules, routes, and UI code that supported them no longer exist in this repo. See Chat Completions API below for why and what replaced them.
This Nx monorepo hosts two applications that act as a single product: an authenticated proxy in front of your local inference server, and the Angular chat interface on top of it. Every chat session is persisted in MongoDB, token usage is tracked per user, and the backend runs its own MCP client that calls tools on-demand as the model requests them — rather than relying on the inference server to orchestrate tool calls itself.
UI
apps/uiAngular 21 single-page application with real-time SSE streaming, image upload/attachment, and a Markdown-rendering message view shared across chat routes.
API
apps/apiNestJS 11 backend acting as an authenticated inference-server proxy, MCP client and server, InvokeAI image-gen integration, and MongoDB persistence layer with JWT auth and token rate limiting.
LM Studio disabled connecting to localhost MCP servers, which broke this project's original architecture: the Responses API worked by handing LM Studio a type: 'mcp' tool pointing at this backend's MCP server, relying on LM Studio itself to connect, list tools, call them, and feed results back to the model. With per-request localhost MCP connections disabled, that flow no longer works — and it was never portable to other backends anyway, since MCP tool passthrough is a Responses-API-only convenience.
The fix: stop depending on the inference server for MCP orchestration entirely, and do it ourselves. The backend now runs its own MCP client (apps/api/src/modules/mcp-client) that connects directly to the MCP tool server, lists available tools, and translates them into plain OpenAI function-tool definitions. Chat requests go out over the standard Chat Completions API (/v1/chat/completions) with those function tools attached. When the model returns tool_calls, the backend executes them itself via the MCP client and loops back into the model until it produces a final answer.
Removed
/api/v1/chat
LM Studio native API
Removed
/v1/responses/create
OpenAI Responses API
Only supported path
/v1/chat/completions
Works with any OpenAI-compatible backend
Known limitation: file attachments are text-only-friendly (images are inlined as vision content; other file types are referenced by ID and fetched on demand via get-content-from-file-ids). Reasoning-effort and AI-decided chat naming are still supported, matching the old Responses-API experience.
A chat generation isn't tied to the HTTP connection that started it. Refreshing the page, closing the tab, or switching to another chat while the AI is still responding doesn't lose or corrupt anything — the backend keeps generating in the background, and the frontend reattaches to it automatically.
Chat ID sent immediately — ChatMetadata is created (and its name decided) before any model output starts, and a created_chat event fires right away so the browser updates its URL well before the first token arrives.
Writes never abort generation — a dead client socket's write errors are swallowed instead of thrown, so a disconnect can't cut the tool-call/completion loop short — the chat only unlocks once the exchange actually finishes.
Per-chat generation buffer — every SSE chunk sent is buffered and broadcast, keyed by chat id — including an echo of the user's own turn, since it isn't in persisted history until the whole exchange finishes.
Resume endpoint — GET /openai/completions-stream/resume replays everything buffered so far for a chat, then streams live chunks until it finishes.
Automatic frontend reattachment — if a chat's metadata comes back locked when opened, the frontend reconnects and renders exactly like a freshly-submitted message — tool calls, reasoning, all of it.
Honest status text — "AI is generating a response…" is shown while watching a resumed/shared generation — not the old "someone else is generating" message, which is now reserved for the brief window before a poll actually attaches.
Sidebar indicator — in-progress chats get an animated wave background and a pulsing dot, backed by a 5-second self-healing poll so it never gets stuck on after you navigate away.
Unlike the old Responses-API flow — where LM Studio itself connected to the MCP server mid-inference — the NestJS backend now acts as the MCP client itself: it lists tools from its own MCP server, attaches them to the Chat Completions request as plain function tools, and executes any tool_calls the model returns before looping back into the model. The inference server never talks to MCP directly, so this works with any backend that supports standard OpenAI function calling.
Angular UI (4200) ──SSE──▶ NestJS API (8888) ──/v1/chat/completions──▶ Inference server
│ ▲
tool_calls loop │ generate-image-tool
▼ │
MCP Client ──▶ MCP Server (@rekog/mcp-nest, self)
│
▼
InvokeAI (9090) — txt2img via REST + Socket.IOFrontend
Angular 21
localhost:4200
Backend + MCP
NestJS 11
localhost:8888
Inference Server
Any OpenAI-compatible
LM Studio, Ollama, llama.cpp, vLLM
Image Gen
InvokeAI
localhost:9090
Angular 21
^21.2.0
NestJS 11
^11.0.1
TypeScript 5.9
~5.9.2
MongoDB / Mongoose
^9.4.1
@rekog/mcp-nest
^1.9.9
InvokeAI (REST + Socket.IO)
local
@nestjs/platform-express (Multer)
memory storage
CryptoJS AES
^4.2.0
TailwindCSS 4
^4.2.2
openai
^6.34.0
JWT + bcrypt
^6.0.0
Nx
22.6.5
OpenAI-compatible Chat Completions
Talks to any backend implementing /v1/chat/completions — LM Studio, Ollama, llama.cpp, vLLM. The only supported chat path.
Custom MCP Servers
Register your own external MCP servers per account, auto-discover their tools, toggle servers/tools on/off, refresh on demand, and opt individual chats out without changing the account default.
Client-side MCP orchestration
The backend runs its own MCP client, translates MCP tools into OpenAI function-tool definitions, and executes tool_calls itself in a loop.
Real-time SSE Streaming
Responses are streamed token-by-token to the browser, including reasoning/"thinking" deltas where the model provides them.
AI Image Generation
The model can call generate-image-tool during inference; the backend submits a txt2img job to InvokeAI, stores the result, and returns it as a chat image.
Image Upload
Attach one or more images before sending; they are stored in MongoDB via the Assets API and forwarded to the model as vision content.
Voice Input
A dedicated mic mode swaps the editor for a recording panel — live bar visualiser, playback, re-record/remove — built on the raw Web Audio API. Sent as an input_audio content part; text is optional when audio is attached.
Voice Transcription
Per-chat opt-in: recorded voice messages are transcribed via a separate, untracked LLM call and swapped for plain text before the main turn runs — tool-calling, reasoning, and token accounting all behave exactly like a typed message.
AES Message Encryption
Per-chat opt-in encryption. Only ciphertext reaches the inference server; the model decrypts via MCP at inference time.
Persistent Chat History
Every exchange is stored in MongoDB as a rolling message array, rehydrated on demand — including tool-call banners and image attachments.
Branch in New Chat
Clone any chat's settings — model, MCP tools, crypto, invoke config — into a brand-new chat seeded with history up to the reply you branched from. Generated and uploaded files are duplicated too, so the branch keeps working after the source chat is deleted.
Resilient Background Generation
A response keeps generating server-side even if you disconnect. Refresh mid-response or switch chats and reattach to the live stream instead of losing it.
Subscription-Aware Token Limiting
Configurable token budgets per subscription tier — not limited to free/basic, new tiers can be created on the fly — with automatic reset intervals and SSE limit notifications.
Admin CMS
Role-gated /admin UI (reactive forms, no ngModel) for managing users — role, subscription, activation, password, token-usage reset — and token-limit configs, including defining brand-new subscription tiers.
JWT Authentication
Login / register with bcrypt-hashed passwords. Tokens expire after 1 hour; Angular auto-redirects on expiry.
Reasoning Mode
Pass reasoning effort (off / low / medium / high) to supported models via the Chat Completions endpoint.
Swagger UI
Optional OpenAPI documentation at /api. Enabled by setting USE_SWAGGER=true in the environment.
Up and running in a few steps.
Install dependencies
From the monorepo root:
npm install
Configure environment
Create apps/api/.env — see the Environment Variables section below.
Start API & UI
Run each in a separate terminal, or both at once:
nx serve api # → http://localhost:8888 nx serve ui # → http://localhost:4200 # Or start both simultaneously: npm start
Register a user
Use Swagger UI at /api or curl:
curl -X POST http://localhost:8888/auth/register \
-H "Content-Type: application/json" \
-d '{"username":"alice","password":"s3cret"}' Create apps/api/.env with the following:
# MongoDB connection URI MONGODB_URI=mongodb://localhost:27017/lmStudioWrapper # OpenAI-compatible inference server (LM Studio, Ollama, llama.cpp, vLLM, ...) LM_STUDIO_BASE_URL=http://localhost:1234 LM_STUDIO_API_TOKEN= # optional # JWT signing secret JWT_SECRET=your-very-secret-key # Backend's own MCP client connects here — safe as localhost/LAN IP SELF_MCP_URL=http://192.168.0.34:8888/tools/mcp # Additional external MCP servers, comma-separated (optional) # MCP_SERVER_URLS=http://example.com/mcp,http://another-host:9000/mcp # Public base URL of this backend — used to build asset URLs (must be browser-reachable) SELF_URL=http://localhost:8888 PORT=8888 USE_SWAGGER=true # enables /api Swagger UI
Since the backend's own McpClientService is now what calls MCP tools, SELF_MCP_URL only needs to be reachable from the backend process itself — it no longer needs to be reachable from LM Studio. SELF_URL must be reachable from the browser, or generated asset links (e.g. AI-generated images) will be broken. The InvokeAI base URL is currently hard-coded to http://127.0.0.1:9090 in app.module.ts.
The NestJS backend plays both MCP roles at once: an MCP server (apps/api/src/tools/api.tools.ts, via @rekog/mcp-nest) exposing Streamable HTTP + SSE transports at /tools/mcp, and an MCP client that connects to that same server (and any others configured via MCP_SERVER_URLS), lists its tools, and calls them on the model's behalf — forwarding the authenticated user's JWT and current chatId so tools have full access to the user's context.
get-token-usage-toolReturns the authenticated user's current token consumption, subscription tier, configured limit, and next reset timestamp.
get-content-from-file-idsReturns the base64 content of previously uploaded or generated files, looked up by file ID.
generate-file-from-content-toolGenerates a downloadable file from provided content and stores it as an asset.
generate-zip-from-file-idsBundles multiple previously generated or uploaded files into a downloadable ZIP archive.
get-image-toolFetches an image from a URL and stores it as an asset.
decrypt-message-toolReceives the full, unmodified ciphertext of the user's message, looks up the per-chat cryptoKey from chat_metadata, and returns the AES-decrypted plaintext to the model.
greeting-toolExample tool that returns a greeting and demonstrates streaming progress reporting via context.reportProgress().
generate-image-toolGenerates an image from a text prompt via InvokeAI, stores it in MongoDB, and returns a chat-renderable image URL.
Beyond the built-in MCP server/client, each user can register their own external MCP servers on their account and control exactly which tools are available — account-wide or per chat. The account-level list, the New Chat dialog, and every chat's settings dialog all read/write the same data, so changes made from any one of them show up in the others immediately.
Register — paste an endpoint URL in the MCP Servers dialog — the backend connects and auto-discovers the server name and full tool list.
Toggle — switch a server on/off, or allow/deny individual tools, account-wide. Saved immediately.
Refresh — re-run discovery any time — new tools are allowed by default, removed tools are dropped, existing choices are preserved.
Per-chat overrides — opt a specific chat out of a server or tool from the New Chat dialog or a chat's settings, without touching the account default.
Request-time merge — on every Chat Completions request, the backend merges each active server's allowed tools (minus this chat's overrides) in alongside the built-in tool set, routing tool calls back to the correct server.
Per-chat opt-in that pauses a tool call for your approval before it runs, instead of executing automatically the moment the model requests it — the same request/approve/deny loop as an agentic coding CLI, applied to this project's own MCP tool orchestration.
Opt in per chat — toggle "Require tool approval" in the New Chat dialog or a chat's settings. Stored as ChatMetadata.toolsRequireApproval — off by default.
Stream pauses, not aborts — when the model requests a tool call, the backend emits a response.tool_approval.required SSE event and awaits your decision — the SSE connection stays open the whole time.
Banner above the input — the pending call (tool name + arguments) renders as a banner above the chat input with three actions: Allow once, Always allow, Deny.
Decision posted back — POST /openai/tool-approval/:requestId resolves the backend's pending promise for that call — Deny skips execution and feeds the model a "User denied this tool call" result instead.
Always allow, persisted per chat — choosing "Always allow" adds the tool name to ChatMetadata.alwaysAllowedTools in MongoDB — later calls to the same tool in the same chat run without prompting again, and the list survives server restarts.
Revoke individually from chat settings — the chat settings dialog lists every always-allowed tool as a chip; clicking one removes just that tool from the list, re-enabling the approval prompt for it.
The "always allow" list is persisted on ChatMetadata.alwaysAllowedTools in MongoDB, keyed per chat — it survives server restarts and redeploys, and any entry can be revoked individually from that chat's settings dialog.
MCP's spec has a standard notifications/progress mechanism, but neither LM Studio nor llama.cpp forward it anywhere the browser can see. Since this backend is its own MCP client, there was no transport carrying a tool's progress back to the chat UI — apps/api/src/tools/tools-helper.service.ts is a custom workaround that fixes that using the same SSE connection already streaming the chat response.
A tool reports progress — any @Tool() method calls this.toolsHelperService.emitApiEvent(request, ApiEvent.MCP_PROGRESS, { progress, total, message }) while it runs (see greeting-tool in api.tools.ts for a working example).
Looked up by request ID — emitApiEvent reads a requestid header off the tool's own incoming request (forwarded by McpClientService on every tool call) and uses it to find the live SSE Response via OpenAiResponseService.get(requestId) — the same registry used for Resilient Background Generation.
Written onto the chat SSE stream — if a matching response is found, an api_report_mcp_progress SSE event is written straight onto it — riding the exact same connection as the chat completion chunks and response.mcp_call.* events, no separate channel.
Frontend consumption — OpenAiStreamService parses api_report_mcp_progress like any other SSE event and forwards it through events$. ChatCompletionsService updates the currently-streaming tool_call bubble's progress/total/progressMessage fields, rendered live by chat-messages.component.ts.
Not part of the MCP spec's own progress-notification flow — it's a bespoke SSE side-channel built specifically because this project's inference-server-agnostic MCP client architecture has no other way to surface a tool's own progress updates to the browser in real time.
The generate-image-tool MCP tool lets the model generate images on demand during a conversation via a locally running InvokeAI instance.
Tool call — the model calls generate-image-tool with a natural-language prompt string.
Model lookup — InvokeService queries InvokeAI's /api/v2/models/ endpoint for the first model matching the requested name (default: "Dreamshaper 8").
Job submission — a txt2img pipeline graph (512×512, 30 steps, dpmpp_3m_k scheduler, CFG 7.5) is submitted via POST /api/v1/queue/default/enqueue_batch.
Socket.IO listener — the service subscribes to the default queue over Socket.IO and waits for an invocation_complete event with the generated image name.
Download & persist — the image is downloaded from /api/v1/images/i/{name}/full and stored as a binary blob via AssetsService.
URL construction — a public asset URL ({SELF_URL}/assets/filequery/{filename}?chatId=...) is returned and rendered as a Markdown image in the chat.
Users can attach one or more images to a chat message before sending. Attached files are listed below the textarea with filename and size, and can be removed before sending. On send, each image is uploaded to POST /assets/:chatId as multipart/form-data, validated for MIME type (max 10 MB), and stored as a binary blob in the image_blobs MongoDB collection, then forwarded to the model as vision content.
Supported formats
Retrieval routes
GET /assets/:chatId/:filename — authenticated, owner or shared-chat access
GET /assets/filequery/:filename?chatId= — authenticated, used for AI-generated image references
A mode toggle next to the chat input swaps the whole composer between typing and recording. By default, no separate speech-to-text step happens in this codebase — the inference server itself (llama.cpp, etc.) handles transcription/understanding via its own audio input support. A text message is optional whenever a recording is attached — you can send audio-only. For models without audio support, see Voice Transcription below.
Switch modes — a mic/pencil toggle in the action row swaps the markdown editor for a voice-recording panel (fade/scale transition) — your typed draft is preserved underneath and restored when you switch back.
Record — tap the mic in the panel to capture microphone audio via the Web Audio API (AudioContext + ScriptProcessorNode) and hand-encode it as 16-bit PCM WAV on stop — MediaRecorder's default webm/opus output isn't decodable by llama.cpp's audio input.
Live visualiser — an AnalyserNode tapped in parallel with the recording processor (no extra dependencies) drives a real-time bar visualiser on a <canvas>, redrawn every animation frame.
Review, re-record, or remove — once stopped, the panel shows the recording in the same audio-player bubble used elsewhere in the chat, with re-record and remove controls before you send. Text is optional — audio-only messages are allowed.
Send — the recording is base64-encoded and sent as a Chat Completions input_audio content part tagged userRecorded: true: { "type": "input_audio", "input_audio": { "data": "<base64 WAV>", "format": "wav" }, "userRecorded": true }.
System prompt injection — whenever a request contains an input_audio part that wasn't transcribed (see Voice Transcription), the backend injects an extra system message instructing the model to treat what was said as the user's actual message.
Playback — recorded voice messages render as a custom audio player bubble (play/pause, seekable progress bar, elapsed/total time) matching the chat UI, both when freshly sent and after reloading chat history.
Requires a model with audio understanding support (e.g. an audio-capable llama.cpp build/model) unless Voice Transcription is enabled for the chat. If the loaded model can't process input_audio and transcription is off, expect it to ignore or error on the audio content.
Per-chat opt-in (ChatMetadata.transcribeAudio) that turns a recorded voice message into an ordinary typed message before the model ever sees audio — useful for models without audio support, or simply to get more reliable tool-calling/reasoning out of a model that technically accepts input_audio but doesn't handle it as well as text.
Opt in per chat — toggle "Transcribe audio" in the chat settings dialog (or at chat-creation time). Stored as ChatMetadata.transcribeAudio — off by default.
Only mic recordings qualify — the backend only transcribes input_audio parts marked userRecorded: true by the client — i.e. captured via the mic panel, not any other audio source.
Separate, untracked LLM call — before the main turn runs, a system message instructs the model to act as a pure transcription engine (never answer, never act on what's said), paired with a user turn containing only the audio. Tokens are never added to the usage counter — same as the "let AI decide chat name" call.
In-place replacement — the transcript replaces the input_audio part with a plain { "type": "text" } part before the main turn is sent — from there it's indistinguishable from a typed message: same tool-calling, reasoning, and history persistence.
Live UI update — an audio_transcript SSE event fires as soon as the transcript is ready, swapping the just-sent audio bubble to a text bubble labeled "transcribed" without waiting for the rest of the response.
Any input_audio part that isn't userRecorded (or transcription is off for the chat) still falls back to the plain "listen to this audio" system prompt from Voice Input — nothing about that path changes.
Why a separate call instead of one combined prompt: an earlier version tried to get a single request to both transcribe and answer via a JSON-envelope system prompt, but small local models frequently either broke tool-calling, leaked the JSON scaffold into the chat, or answered the audio's request directly instead of transcribing it. Splitting transcription into its own untracked, audio-only call sidesteps all three failure modes.
Per-chat AES-256 encryption can be opted into when creating a new chat session. Only ciphertext ever reaches the inference server's own message store/logs — plaintext never leaves the NestJS trust boundary.
Session created
cryptoKey generated & stored in chat_metadata. Never leaves MongoDB.
Encrypt
Backend runs CryptoJS.AES.encrypt() on all message content before forwarding to the inference server.
Prompt inject
Developer-turn instruction injected: always call decrypt-message-tool first.
MCP decrypt
The model calls back — decrypt-message-tool fetches the key from DB and returns plaintext.
Answer
Model answers the decrypted question. The cycle is invisible to the end user.
| What | Where | Plaintext? |
|---|---|---|
| cryptoKey | chat_metadata MongoDB document | ✓ Yes |
| Messages → inference server | Inference server message store | ✗ No |
| Browser → NestJS (HTTP body) | HTTPS in production | ✓ Yes |
| chatId MCP header | MCP request header (key lookup) | ✓ Yes |
The cryptoKey lives in MongoDB — your NestJS API and database are the security boundary. Use HTTPS and restrict DB access in any non-local deployment.
Registration
POST /auth/register creates a user with a bcrypt-hashed password. New accounts are inactive until an activation link is used.
Login
POST /auth/login returns a signed JWT (1-hour expiry). The Angular root component auto-redirects to /login when the token expires.
JWT Guard
JwtAuthGuard is applied globally as an APP_GUARD. Individual routes are opted out with the @Public() decorator.
Role-Based Access
RolesGuard enforces @Roles(Role.Admin) and @Roles(Role.User) decorators on individual endpoints.
Token consumption is tracked per user and enforced against subscription-tier limits configured in the token_limit_configs MongoDB collection.
Free
Configurable
tokens / interval
Basic
Configurable
tokens / interval
Custom
DB-driven
tokens / interval
After each completed inference, TokenLimitService.updateUsedTokens() increments the user's usedTokens counter. If the limit is reached, an api.info SSE event is emitted with the reset timestamp. Limits reset automatically when tokenCountResetDate elapses. Token limits are managed exclusively through the Admin CMS — the TokenLimitModule controller is gated behind @Roles(Role.Admin).
A role-gated /admin route (Angular reactive forms throughout — no ngModel) for managing users and token-limit configs without touching MongoDB by hand. Only visible/reachable for users with role: 'admin' — the link appears in the account info panel only for admins, and is enforced twice: an Angular adminGuard route guard, and @Roles(Role.Admin) on every backend endpoint.
List every user with role, subscription, activation status, and current token usage. Create a user directly (bypassing the normal registration/activation-email flow), edit an existing user's role, subscription, activation status, or password, reset their token-usage counter on demand, or delete them (an admin cannot delete their own account, to avoid accidental lockout).
List, create, edit, and delete token_limit_configs documents. Creating a config with a brand-new tier name is how a new subscription type is defined — there's no separate "add subscription type" step. The tier-name field is free text when creating a config (validated against ^[a-z0-9_-]{2,32}$), and locked once a config exists (one config per tier, enforced by a unique index). The "assign subscription" dropdown in the user-edit dialog is populated from GET /admin/users/subscription-types, which unions the built-in defaults (free, basic), every tier with a config, and any tier already assigned to a user — so a user's tier stays selectable even if its config was later deleted.
| Method | Path |
|---|---|
| GET | /admin/users |
| GET | /admin/users/subscription-types |
| GET | /admin/users/:id |
| POST | /admin/users |
| PATCH | /admin/users/:id |
| DELETE | /admin/users/:id |
| POST | /admin/users/:id/reset-tokens |
| GET/POST | /token-limit-configs |
| GET | /token-limit-configs/:id |
| PUT | /token-limit-configs/:id |
| DELETE | /token-limit-configs/:id |
All admin routes require both a valid JWT and role: 'admin'.
| Method | Path |
|---|---|
| POST | /auth/register |
| POST | /auth/login |
| POST | /auth/mcp-servers |
| PATCH | /auth/mcp-servers/:id |
| POST | /auth/mcp-servers/:id/refresh |
| DELETE | /auth/mcp-servers/:id |
| GET | /openai/models |
| POST | /openai/completions-stream |
| GET | /openai/completions-stream/resume |
| GET | /chat-metadata |
| GET | /chat-metadata/:id |
| POST | /chat-metadata |
| PATCH | /chat-metadata/:id |
| DELETE | /chat-metadata/:id |
| GET | /chats/:chatId |
| POST | /assets/:chatId |
| GET | /assets/:chatId/:filename |
| GET | /assets/filequery/:filename?chatId= |
| GET | /invoke/test |
| GET/POST | /tools/mcp |
| GET/POST/PATCH/DELETE | /admin/users[/...] |
| GET/POST/PUT/DELETE | /token-limit-configs[/...] |
Full interactive docs at http://localhost:8888/api when USE_SWAGGER=true.