Moor your first model

A mooring is a model tied up and written down

A dock is an inference engine or endpoint. A berth is one spot at it, holding exactly one moored model. A mooring is both halves: the model tied up in that berth, and the named record saying which model, which dock, and what it is for.

With no records saved, the launch flags become one anonymous mooring called default. With nothing saved anywhere, the engine starts unmoored, on a provider that refuses every call by name, while the fleet listing and the mooring form keep working so the model gets chosen from inside the session.

Five docks take a mooring

Startup detects the two engines Shoal can start, mlx first on Apple silicon and then llamacpp on any platform. A server Shoal did not start is the remote dock whether it runs on this machine or another, and you reach it by passing an endpoint, naming the dock, or writing a mooring record for it. The anthropic and openai docks are reached by a mooring record that names them, so a request that leaves your machine is a decision you wrote down. A remote mooring needs an endpoint or a port, and an anthropic or openai one needs a key.

The models command names what fits here

Run shoal models. It prints the hardware it found, then every model in your models directory with its size, and marks the one it recommends: the largest that fits the memory budget and can carry a tool call. The tool-call column comes from each GGUF header, so no weights load. A model whose chat template renders none is named as such, beside the note that a mooring setting tools: text reaches it by describing the tools in the prompt instead.

The recommendation line carries the command that moors it, and --json gives machine-readable output. Any GGUF path or MLX identifier works, listed or not.

The berth ledger refuses what will not fit

Before a local mooring is admitted, the ledger estimates what it would take: weights at their on-disk quantized size, KV cache summed per layer, and overhead. It compares that against live device memory, read from nvidia-smi or rocm-smi under a timeout, or a deterministic budget on Apple silicon.

Then it admits or refuses. A refusal carries the estimate, the budget, and at least one alternative, structured rather than prose: a mooring to cast off and the bytes that frees, or a shorter context that does fit. The failure direction is fixed at refusal rather than an out-of-memory crash. The ledger does not guess: a device it cannot read is admitted with the reason said out loud.

A mooring record is a small JSON object

Records live under moorings in .shoal/config.json for one repository, or in the user-wide config for the ones you keep across projects. A project record wins over a user record of the same name. Names are kebab-case and unique, and one mooring holds the main purpose.

{
  "moorings": [
    { "name": "main", "dock": "llamacpp", "model": "gpt-oss-20b-MXFP4.gguf",
      "purpose": "main", "context": 32768 },
    { "name": "gate", "dock": "llamacpp", "model": "small-3b.gguf",
      "purpose": "gate", "context": 8192 },
    { "name": "shop", "dock": "remote", "model": "qwen2.5-coder-14b",
      "purpose": "reader", "endpoint": "http://host:8000/v1", "key": "shop-token" },
    { "name": "deep", "dock": "anthropic", "model": "claude-opus-5",
      "purpose": "reader", "key": "anthropic", "price_in": 5, "price_out": 25 }
  ]
}

The engine routes on five purposes: main, task, janitor, gate, and reader. Any other is a label nothing routes to. key names a secret and never holds one: store the value with shoal secret set <name>, which reads it from a prompt or stdin. chat_template names a llama.cpp built-in template or a jinja file, for a model whose own header carries none. A form writes the same record and can tie it up in the same move.

Swapping and casting off are two different acts

/moor <name> swaps the active mooring mid-session. The conversation carries over, the prefix cache does not, so the next turn re-reads the whole history at prefill speed and Shoal says so rather than letting a slow turn read as a fault. /moor with no argument lists the fleet, and /moorings lists it with what is tied up. /moor given a model path, or a name from your models directory, builds a mooring on the spot and writes it to no file.

/unmoor <name> casts off. The server stops if this process started it, the berth goes back to the ledger, and the record stays, ready for /moor again. Forgetting a record is the opposite move: it removes the record and leaves the process running. Both are labelled plainly wherever they appear. The active mooring cannot be cast off, since stopping the server that answers the next request would leave the session on a closed port. Moor somewhere else first.