Run the agent as a standalone CLI

Overview

@tetherto/mdk-agent runs as a small library and CLI for local development, talking to a served model and an MCP tool server directly, with no Gateway in front of it.

This is one of two ways to run the agent.

Prerequisites

• Node.js ≥ 24
• npm 11 (< 12)
• GPU:
  • macOS: Metal
  • Linux and Windows: Requires Vulkan 1.4+. On macOS and Linux, a missing GPU backend falls back to CPU (slower but functional). On Windows, the native runtime requires Vulkan 1.4+ even for CPU inference and fails outright if missing or on an older version. Verify with vulkaninfo --summary before running.
• Dependencies installed — see the agent's prerequisites: a single root npm install, not one run inside the package
• Language model: this guide uses QVAC serving an OpenAI-compatible API on port 11500

mkdir -p ~/qvac-runtime && cd ~/qvac-runtime && npm init -y   # on macOS/Linux
# On Windows (PowerShell): md ~/qvac-runtime; cd ~/qvac-runtime; npm init -y
npm i @qvac/cli@^0.12.0                                       # ~6 GB of prebuilt engines

cp backend/core/agent/qvac-runtime/qvac.config.json ~/qvac-runtime/

cd ~/qvac-runtime
npx qvac doctor                                               # confirms GPU backend and host capabilities
# --config pins the model: Qwen3-4B, 4-bit quantized, 16k context
npx qvac serve openai \
  --config qvac.config.json \
  --port 11500 --verbose --no-cancel-load-on-disconnect       # --verbose: load + GPU offload; wait until listening on 11500

cd examples/mvp-site
npm run setup         # first time only. NOT npm install: examples/mvp-site is a root workspace member,
                      # and this also builds the UI toolkit and installs its own nested ui/ app
npm run setup:config  # first time only: writes config/devices.json and config/site.deploy.json
npm start             # kernel + workers + gateway + MCP server on :3101, under PM2

cd backend/core/agent
node bin/mdk-agent.js --model qwen3-4b --base-url http://127.0.0.1:11500/v1 --mcp-url http://127.0.0.1:3101/mcp

you › how many miners are on the site?
  → tool   count_devices({"family":"miner","state":"all"})
  ← data   { "summary": "30 miners.", "count": 30 }
  ▌ There are 30 miners on the site.

you › list the devices that are not ready
you › reboot antminer-3          ← a write: the agent stops and asks you to approve

Troubleshooting

npx qvac fails with a 404

When inside ~/qvac-runtime, npx qvac resolves from the local ./node_modules/.bin/qvac binary. If run outside that directory, npx searches the npm registry for a package named qvac (which does not exist; the package is @qvac/cli), failing with a 404. Run cd ~/qvac-runtime first.

qvac serve openai dies with EADDRINUSE

qvac serve openai defaults to port 11434, the same as Ollama's. Pass --port 11500 (as the command above does) if both run on this machine.

The first request 503s with model_load_failed, blaming a disconnect that didn't happen

serve.load.cancelOnDisconnect defaults to true, which cancels a lazily loaded model's first request. --no-cancel-load-on-disconnect fixes it. Models declared preload: true (like qwen3-4b in qvac.config.json) never hit this, but the flag costs nothing.

The first run looks stalled, or the server "never starts"

The first run fetches ~2.5 GB peer-to-peer. The log says registry://s3/…, but the transfer is actually Hyperswarm (a DHT plus UDP hole-punching), at roughly ~110 MB/min, so ~25 minutes. There's no HTTP fallback, so a network that blocks UDP stalls it, and since preload: true means the port doesn't open until the model is resident, a stalled download presents as a dead server, not a failed fetch. Three things help:

1. Copy qvac.config.json and set preload: false. The port opens in ~5 s and a stalled fetch surfaces as a 503 on the first request instead of a server that looks dead. Switch back once ~/.qvac/models holds the weights.
2. Always pass --verbose; without it there's no progress output at all.
3. Bypass the registry entirely: serve.models accepts an explicit { "src", "type" } entry, and *ModelSrc fields accept URLs and filesystem paths, so a GGUF fetched by hand over HTTPS, or copied from a machine that already has one, works. This is the path for a locked-down site.

Disk usage is larger than expected

Budget ~6 GB for the CLI, ~2.5 GB for the model weights, and a KV cache that grows over time. Of the CLI's ~6 GB, ~4.6 GB consists of native prebuilt binaries across multiple platforms and architectures. Deleting the foreign prebuilds reclaims ~2.5 to 3.4 GB safely — see Free foreign prebuilds in Step 1.1 for the commands.

The fleet has fewer devices than expected

Each Worker seeds its device list once, into a persistent on-disk store, and only when that store is empty. A later npm start reuses whatever's already there rather than reseeding from the current config/devices.json, so a fleet from an earlier run (a different device list, or an interrupted seed) sticks around. Run npm run stop:pm2, then clear examples/mvp-site/.site-data for a fresh fleet on the next start.

Next steps

• Look up a flag, the capability table, or a hosted-model setup: the CLI's full reference
• Deploy the agent behind the Gateway instead, for a chat API an operator UI can call