Skip to content

README.md

Latest commit

 

History

History
127 lines (95 loc) · 6.21 KB

README.md

File metadata and controls

127 lines (95 loc) · 6.21 KB
Raw

AI chat

Deploy on velixir

Your own private AI chat, in front of the provider you choose. Replies stream in as they are written, conversations stay in your browser, and your API key never leaves the server. One password keeps everyone else out. Plain Node, with no dependencies at all.

Deploy it on velixir.

It talks to Mistral out of the box (an EU provider), and to OpenAI, Groq, OpenRouter or anything else that speaks the OpenAI Chat Completions API by changing two settings.

What it shows

  • Streaming end to end. The app asks the provider for a streamed reply and passes it on to the browser as server-sent events while it is being written. Keep-alives go out while a model is thinking, so a slow first word does not time out on the way.
  • A key that stays on the server. The browser only ever talks to this app. The key is added to the request upstream, and scrubbed from any provider error before it is shown.
  • A password in front of your bill. Signing in issues a signed, HttpOnly session cookie that lasts 30 days, survives restarts and works across replicas, with no database behind it.
  • Safe rendering. Replies are HTML-escaped first, then a small set of markdown (code blocks, inline code, bold, lists, links) is turned back into formatting. A strict Content-Security-Policy refuses to run any script that is not the app's own.
  • Clear errors. A wrong key, an unknown model, a rate limit, an outage or a timeout each show up in the chat as a plain sentence with a Retry button, not a spinner that never stops.
  • A cap on what each message costs. Every message resends the conversation, and the provider bills all of it again. The app sends the newest 40 messages, up to 32,000 characters, and drops the oldest first.
  • No dependencies. node:http, the built-in fetch and node:crypto. Nothing to update, and nothing to break six months from now.

Setting it up on velixir

  1. Deploy it from the gallery. It needs no database and runs on the Free plan.

  2. Set these on the app's Environment tab, marking the first two as secret:

    Variable Required Value
    AI_API_KEY yes Your provider's API key
    ACCESS_PASSWORD yes A long random password, at least 8 characters (openssl rand -base64 24 makes a good one)
    AI_BASE_URL no The provider's API, default https://api.mistral.ai/v1
    AI_MODEL no The model, default mistral-small-latest
    SYSTEM_PROMPT no Instructions added to the start of every conversation

  3. Redeploy, with the Redeploy button on the live release in the Deploys tab.

Until both required values are set, the app serves a page that names what is missing (never the values themselves) instead of a chat.

Providers

Provider AI_BASE_URL AI_MODEL, for example
Mistral (the default, EU) https://api.mistral.ai/v1 mistral-small-latest
OpenAI https://api.openai.com/v1 gpt-6-luna
Groq https://api.groq.com/openai/v1 llama-3.3-70b-versatile
OpenRouter https://openrouter.ai/api/v1 mistralai/mistral-small-2603
Anything OpenAI-compatible its base URL, up to but not including /chat/completions the provider's model name

The app calls AI_BASE_URL + /chat/completions with stream: true. Model names change often, so check your provider's current list. A server you run yourself (Ollama, vLLM, LM Studio) works too, as long as it is reachable over HTTPS on port 443: apps on velixir can make outbound web requests only.

Running it locally

There is nothing to install.

AI_API_KEY=your-key ACCESS_PASSWORD=choose-a-password npm start

Then open http://localhost:8080.

Deploying

velixir deploy

Things to know

The password is the lock. Anyone who has it can spend your API credit, so make it long and random, and share it only with people you would lend your card to. Set a monthly spending limit with your provider as well: it is the backstop if the password ever leaks. Changing ACCESS_PASSWORD signs everyone out at once.

The rate limits are a brake, not a lock. Each client address gets 10 wrong passwords and 20 messages a minute. Many visitors can share one address (an office, a phone network, a proxy), so treat the limits as a cap on runaway use rather than a guarantee about any one visitor: the password is what keeps strangers out. Browsers that are already signed in are never locked out by someone else's wrong guesses.

The provider bills the tokens. The app itself fits the Free plan. Every message is billed by your AI provider, including the history resent with it, so a long conversation costs more per message than a short one. Start a new chat when the subject changes.

Conversations live in your browser. They are kept in the browser's local storage on the device you used. Nothing is stored on the server, so there is nothing to back up or leak, but there is no sync between devices either, and clearing the site's data deletes them. The provider receives each conversation in order to answer it, under its own data policy.

One password means one shared login. There are no accounts. Everyone who knows the password is the same user, with their own conversations in their own browser.

The Free plan sleeps. After 15 minutes without visits the app goes to sleep, and the next visit takes a little longer while it wakes. For a personal chat that is usually fine; pick an always-on plan if it is not.

Limits. A message can be up to 8,000 characters. A reply is cancelled after 60 seconds without a word from the provider, or after 5 minutes in total, and the chat says so.

Routes

Method Path Does
GET / The chat, or the sign-in page
POST /login Sign in with the access password
POST /logout Sign out
POST /api/chat {"messages": [{"role": "user", "content": "..."}]}, answered as server-sent events: delta, then done or error
GET /healthz Always 200, setup mode included: a provider outage is not a reason to restart the app

Licence

MIT.