AI runtime and access model

Everlock embeds a small LLM and exposes it in two places:

as a free-text prompt mode inside the admin SSH session (backend-ai-ssh + backend-admin-ssh + frontend-ssh)
as an image-captioning provider for backend-image-http, sharing the already-loaded model so there is no second copy in memory

The shared runtime lives in its own component, everlock-ai-runtime, which owns the model bytes and the worker thread. Inference runs on Everlock's own gguf-runner engine — a self-developed GGUF runtime that runs in-process on your CPU. Backends use it through small traits (AiTool, ImageInferenceProvider).

How the admin shell behaves

lines starting with / remain admin commands
all other lines are treated as AI prompts
// escapes a leading slash and sends the rest of the line to the model
model output is streamed back into the session as it is generated
the model can call explicit admin tools that map onto the current command surface
a built-in search_docs tool lets the model query the bundled docs for Everlock-specific context

There is no separate /ai ... command family.

Enabling it

The AI backend is always part of the binary; the model is downloaded and embedded during the build. There is no feature flag that removes the backend itself.

Toggle the SSH prompt path from the admin session:

/backends enable ai-ssh
/backends disable ai-ssh

Even when the SSH prompt mode is disabled the runtime still starts so that backend-image-http can use it for captioning.

Access control

Prompting uses the normal Everlock access model on:

ssh/ai/default

To allow a user to prompt the model:

/users grant alice ssh/ai/default writer

The image-captioning path is not gated on ssh/ai/default — access control for that lives entirely in backend-image-http.

Models

The build embeds one of two models, selected by a Cargo feature on everlock-ai-runtime:

Feature	Model	Approx. size
default	SmolVLM-256M-Instruct (Q8_0) + mmproj	~250 MB
`qwen3`	Qwen3.5-2B (Q3_K_M) + mmproj	~1.9 GB

Both are vision-capable, which is what lets the same runtime serve image captioning.

The published Docker images map onto these builds:

the default latest image (built from Dockerfile.smolvlm) embeds SmolVLM-256M;
a separate Qwen3 image (built from Dockerfile.qwen3) embeds Qwen3.5-2B.

Switch to Qwen3 in a local build:

cargo build --release --features qwen3

Models are downloaded into target/models/ on first build. The default SmolVLM build embeds them via include_bytes!; the Qwen3 build uses assembly .incbin to bypass an LLVM rlib size limit at ~2 GB. For the full engine and model picture, see The gguf-runner inference engine.