Choosing the models

SkyKeep runs every AI step against a model runtime inside your own deployment. An administrator chooses which models, in the admin console under Settings → Models. Only models your runtime actually has are offered: a model that has not been pulled cannot answer, so the vault will not pretend it is a choice.

Choosing a model chooses who writes the sentence, never what the sentence may be written from. Your compartment grants decide that, and nothing on this page reaches them. A different model does not widen, narrow, or otherwise change what any question can see.

Two jobs, two choices

Answering a question and reading a document at upload are different jobs with opposite shapes, so they are separate settings. One model can do both — that is the default — but a deployment that ingests overnight and answers all day often wants a careful slow model for the first and a quick one for the second.

Role	When it runs	What that means	If you leave it empty
Answering (query) `think_model`	Once per question, while somebody watches a spinner.	One large prompt built from the passages your grants admitted. Latency-bound: this is the choice a person feels. Answer quality is visible immediately, and so is a slow model.	No fallback. It is unset or it is chosen.
Ingestion (summary and extraction) `ingest_model`	Once per document version, at upload, with nobody waiting.	Many small prompts. Throughput-bound: latency is nearly free here, so a slower, more careful model costs little — but it is paid on every upload, which is what makes a bulk ingest slow.	Leaving it empty uses the answer model instead — and the console says so rather than showing you a blank field.
Embedding (the index) `embed_model`	Every chunk at ingest, and every query. Not a chat model.	A separate KIND of decision, not a preference. Changing the answering model changes the next answer; changing this one invalidates every vector in the vault. Treat it as a kickoff-time decision.	No fallback. It is unset or it is chosen.

Why the embedding model is different

The two chat roles above are preferences: change one and the next answer changes. The embedding model is not a preference. It is what turned every document in this vault into the numbers that make it findable, and changing it does not migrate them.

A model of a different width is refused outright by the database, which is the good outcome. A model of the same width is the dangerous one: the writes succeed, and from then on new documents are ranked against old vectors from a different space — search quietly gets worse and nothing reports an error. Treat it as a decision made once, at the start of an engagement.

Where the descriptions come from

The console shows what each model is good and bad at. Every such claim is labelled, because most deployments will pull a model this project has never characterised, and a page that quietly invented a description would be worse than one that says it does not know.

Label	What it means
`curated`	Characterised by this project, in the vocabulary of the jobs this vault actually does.
`derived`	Not a model we have characterised. Everything below follows arithmetically from what your runtime reported about it — size, parameter count, quantisation — and nothing else is claimed.
`measured`	Observed on THIS deployment. A number you measured beats a number anyone implied.

The models this project has characterised

Matched by name. Anything else your runtime offers still appears in the console, described only by what the runtime reported about it.

Matches	Good at	Watch for	Suited to
`all-minilm`	Very small and very fast — embeds a whole corpus in minutes. 384 dimensions, which is what this vault's vector column is typed for; no migration needed to use it.	Caps at roughly 256 tokens of input. A chunk longer than that is indexed by its opening only — the rest is stored, decryptable and citable, and unreachable by meaning. A general-purpose sentence embedder: it has no special sympathy for legal, clinical or financial vocabulary.	embed
`nomic-embed-text`	A long input window (thousands of tokens), so a whole chunk is actually read rather than its opening. Still small enough to embed a corpus on CPU.	768 dimensions — the vault provisions a `halfvec(768)` column for them (migration 0044, half precision since 0105), so no migration is needed. Changing to it still invalidates every vector already stored, which means a full re-embed of the corpus before anything ranks against it.	embed
`qwen2.5:0.5b`	Answers in a few seconds even on a CPU-only host. A ~400 MB pull, so a clean checkout is quick to stand up.	Answer quality is poor on multi-passage questions — measured on the reference deployment, it answered a ten-chunk question with a sentence not worth reading. Smaller models follow instructions planted in a document more readily than larger ones. The vault's defences are structural and do not depend on any model resisting — both scan gates run before storage, passages reach the model fenced as data with the rule restated after them, and the model-written summary can never re-enter a prompt — but a model that follows text in a document is a poor one to put in front of an audience.	—
`llama3.2`	Answers multi-passage questions coherently and cites the passages it used. 3B parameters and a ~2 GB pull — small enough for a laptop-class host, large enough to answer from several passages at once.	On a CPU-only host a large context is slow: prompt evaluation is roughly linear in context length, so a ~22k-character prompt can outlast the front door's timeout.	answer, ingest
`llama3.1`	Stronger reasoning than the 3B class on questions that need several passages joined.	8B parameters: comfortably too slow for interactive answering on a CPU-only host, and it wants ~5 GB resident.	ingest
`mistral`	Even-handed general prose; long-standing permissive licence.	7B class — the same CPU latency caveat as any 7-8B model.	ingest
`phi4-mini`	The phi line's summarisation-above-its-weight, at 3.8B and a ~2.5 GB pull — the same size class this project already ships as its answering baseline. Small enough to sit resident on a 4 GB card without evicting the embedding model on every query.	Inherits phi3's terseness: on a question needing several passages it can answer from one and stop, which reads as a confident partial answer rather than a visible gap. Not yet measured on this deployment. The size class is characterised; this particular model is not.	ingest
`granite4.1`	3B parameters and a ~2.1 GB pull: the size class that answers coherently and still finishes on a CPU-only host. IBM ships the Granite line under Apache-2.0, which is a standard open-source licence rather than bespoke model terms — one less thing to clear against an engagement.	Not measured on this deployment. Everything above is its size class and its licence, not observed answer quality. Same CPU-latency caveat as any 3B model: a full set of passages is a large prompt, and prompt evaluation is roughly linear in context length.	ingest
`gemma4`	The Gemma family's summarisation strength for its size, which is the ingestion job.	The `e2b` tag is a ~7.2 GB download and `e4b` is ~9.6 GB, far more than the parameter label implies. On a 4 GB card the runtime will offload most of it to the CPU, and Ollama on this host already reports `entering low vram mode`. Licence is Google's Gemma terms, not a standard open-source licence — check it against the engagement before adopting. Not characterised by this project and not measured here. The download sizes above were read from the registry manifest; nothing else about this model is being claimed.	—
`phi3`	Punches above its size on summarisation, which is exactly the ingestion job.	Tends to be terse; on a question needing several passages it can answer from one and stop.	ingest
`gemma`	Strong summarisation for its size.	Licence is Google's Gemma terms, not a standard open-source licence — check it against the engagement before adopting.	ingest
`deepseek-r1`	Explicit step-by-step reasoning on hard questions.	Emits its reasoning before its answer, which is long, slow, and not what a cited answer wants. The reasoning trace is generated text about document content: on a compartmented vault that is more surface, not less.	—

What a model change makes load-bearing

A model choice is not an isolated setting: it changes what other settings must be true. The console evaluates each of these against your configuration and tells you where you stand; the table below is the list of what it checks.

If you change…	…this becomes load-bearing	What goes wrong if it is ignored
Any chat role	The kind of model selected	An embedding model cannot generate text at all: every answer and every summary fails.
Answering model → a larger one	Context size (k × the passage-expansion bound, NOT the chunk window — a matched chunk is widened to its enclosing section at prompt build) and the front door timeout	Prompt evaluation is roughly linear in context on CPU-only inference: the answer never returns and the proxy reports 504.
Passage widening → a larger bound, or the context → smaller	The context window the answering model is given	A prompt built from widened passages that the window cannot hold is truncated to its tail, and this vault puts its best passage first — so the answer is fluent, confident and missing the evidence it was supposed to use.
Embedding model → one with a different input limit	The chunk window	A window the embedder cannot read is indexed as a prefix. Stored, decryptable, citable — and unreachable by meaning.
Embedding model → one whose similarities sit elsewhere	RL-2's relevance floors — `SKYKEEP_SEARCH_RELEVANCE_FLOOR` and `SKYKEEP_SEARCH_RELATIVE_RELEVANCE_FLOOR`	A floor calibrated on another model's distribution removes nothing: the vector leg's k least-bad chunks reach the prompt, and 'nothing here is relevant' stops being an answer the vault can reach.
Embedding model → any change of width	Migration 0011's `vector(384)` column	A dimension mismatch fails the insert closed, by design.
Embedding model → any change at all	Every vector already stored	A same-width change is the dangerous one: it succeeds, and silently ranks new documents against old vectors from a different space.
Selecting several distinct models	Available memory on this host	The runtime evicts and reloads between requests, or the host OOM-kills it mid-answer.
Ingestion model → a slower one	Upload wall-clock and any bulk-ingest window	Uploads that used to return promptly no longer do. Nobody is watching, so it surfaces later as 'the demo is slow'.
Any model → a non-local endpoint	The `SKYKEEP_NONLOCAL_MODEL_OVERRIDE` environment variable	Refused at startup, fail-closed (Gate IF.3). The override is an environment variable and not a console setting because it must exist before the service does.

How much of a document the model is shown

A document is cut into chunks so it can be indexed, and how long a chunk may be is decided by the embedding model's input limit — not by how much an answer needs. Those are different numbers, and left alone the second loses: a heading section longer than one chunk is cut mid-thought, and nothing in the answer says it was cut.

So when a chunk wins the ranking, the prompt receives the whole section it came from rather than the chunk window that matched. Nothing is re-indexed and nothing is re-stored — the vault simply reads more of a document it had already decrypted for you. It reaches documents already in the vault, not just new ones, which is unusual for a setting and worth knowing.

A citation still names the chunk that matched, never the wider passage that was sent. That distinction is the whole point of the guarantee: widening changes what the model reads, and never what the vault claims it read. Follow a citation and you get the passage the vault says you get.

The bound is Passage expansion in Settings → Models. It is checked against the context window — the table above lists that pairing — and a bound the window cannot hold is refused with both numbers named, rather than accepted and silently truncated. Setting it to 0 turns widening off entirely and restores the earlier behaviour exactly.

Choosing a model for one question

Where an administrator has enabled more than one model, the query page offers a choice of who writes the answer. Two things are worth knowing about it.

It changes the writer, never the reading. Whichever model you pick, the passages it is given are exactly the ones your own compartment grants admitted — the vault retrieves first, under your predicates, and hands the model references it re-resolves under those same predicates. A different model cannot reach a document you could not.

The radio is a request, not a decision. The page shows what your administrator enabled; the server checks it again on every question and refuses a model that is not enabled, whatever the page happened to be offering. If the deployment runs a single model there is no choice to make and the control does not appear.

The answer names the model that wrote it. If it also says the passages did not fit the model's context window, take that seriously: it means the model was shown only part of the evidence, and an administrator can raise the window in Settings → Models.

The roads out of this vault

A model runtime is reached over exactly six roads, and they are not interchangeable: different data crosses each one, so each has its own address, its own credential and its own consent. The table is this deployment's own, read from the settings the vault is running with.

A road that was never given an address of its own is not switched off — it runs on the road above it. That is the single fact this table exists to make visible: leaving the generation endpoint empty does not disable generation, it sends every prompt to the endpoint that embeds. Where that is happening the row says so in as many words.

No address, key, token or password appears here, and none can. This page is readable by a visitor who has not signed in, so it reports which setting names each road and what state that setting is in — never what the setting contains. Where a client certificate is in use the row says it is configured and names the two settings; it never prints the file. An operator running the runtime on a shared machine should read the GPU gateway, which is what that host is obliged to do at its end.

Road	Where it goes	What it presents	Model	What crosses it
base	`SKYKEEP_OLLAMA_BASE_URL` named, and reached over TLS. It is outside this deployment, so it runs only with the operator's explicit egress acknowledgment, and every call it carries is audited.	`SKYKEEP_MODEL_AUTH` bearer	`SKYKEEP_EMBED_MODEL` Chosen in Settings → Models, so the live value is in the vault and this page does not read it. The roles table above says what each role does.	Every chunk of every document, as it is embedded — and every generation as well, on a deployment that has named no generation endpoint. It is the widest road this vault has.
generation	`SKYKEEP_GENERATION_BASE_URL` (unset — resolves to base)	`SKYKEEP_GENERATION_AUTH` none, and it cannot be set while this road has no address of its own. What is presented is the base road's credential, because this IS that road.	`SKYKEEP_THINK_MODEL` Chosen in Settings → Models, so the live value is in the vault and this page does not read it. The roles table above says what each role does.	Prompts: a document's text on its way to an upload summary, the entity fallback, the auditor's screen, a translation, and a question with its retrieved passages on the way to an answer.
answer	`SKYKEEP_ANSWER_MODEL_URL` (unset — resolves to generation)	`SKYKEEP_ANSWER_MODEL_AUTH` none, and it cannot be set while this road has no address of its own. What is presented is the generation road's credential, because this IS that road.	`SKYKEEP_THINK_MODEL` Chosen in Settings → Models, so the live value is in the vault and this page does not read it. The roles table above says what each role does.	The final answer-writing call alone, layered over the generation endpoint. Nothing from an upload takes this road.
vision	`SKYKEEP_VISION_BASE_URL` (unset — resolves to base)	`SKYKEEP_VISION_AUTH` none, and it cannot be set while this road has no address of its own. What is presented is the base road's credential, because this IS that road.	`SKYKEEP_VISION_MODEL` not configured — the engine is asked for no model, and refuses rather than answering with whatever it has.	Image bytes, when a picture is read rather than stored.
transcribe	`SKYKEEP_TRANSCRIBE_URL` Not set, and it falls through to nothing: this vault does not transcribe, and a video's reading says the sound was not read.	No credential scheme, by decision. The transcription engine is an unauthenticated service an operator runs beside the vault; a deployment that puts it behind a gateway instead should place the gateway inside the deployment's own network, because this seam has nothing to present to it.	`SKYKEEP_TRANSCRIBE_MODEL` not configured — the engine is asked for no model, and refuses rather than answering with whatever it has.	Audio bytes, when a soundtrack is read.
page-render	`SKYKEEP_PAGE_RENDER_URL` Not set, and it falls through to nothing: this vault draws no pages, and the page view answers 404 rather than guessing.	No credential scheme, by decision. The page renderer is a compartment-blind drawing service inside this deployment's own compose network — it holds no database, no keys and nothing to authenticate to.	No model — this seam runs an engine, not a model.	Document bytes, when a page is drawn for the viewer. No model runs on it.

How many answers at once

Each model road this vault uses has a cap on how many calls may be in flight along it at one time, so a burst of questions cannot leave everybody waiting. Past the cap a request waits a few seconds and is then told the vault is busy — with a Retry-After, and never an error: nothing is broken and the same question a moment later is answered. An upload being read waits instead of being refused, because there is nobody for it to tell.

The counts below are this process's own. If a road was never given an address of its own it shares the one above it, and the table says which — two roads showing the same number are one machine, not a mistake.

Road	At once, most	Whose cap
base `SKYKEEP_BASE_MAX_INFLIGHT`	8	its own
generation `SKYKEEP_GENERATION_MAX_INFLIGHT`	8	shares base
answer `SKYKEEP_ANSWER_MAX_INFLIGHT`	8	shares base
vision `SKYKEEP_VISION_MAX_INFLIGHT`	8	shares base
transcribe `SKYKEEP_TRANSCRIBE_MAX_INFLIGHT`	2	its own
page-render `SKYKEEP_PAGE_RENDER_MAX_INFLIGHT`	4	its own

Models never leave your deployment

The vault refuses to start if its model endpoint is outside the deployment. The only way past that is an explicit acknowledgment set in the environment — not in this console, because it has to exist before the service does — and while it is active every single model call emits its own audit entry. The egress is permitted; it is never silent.