Introduction

nexo-rs is a Rust framework for building multi-agent LLM systems that live on real messaging channels — WhatsApp, Telegram, email — instead of a chat webapp. Event-driven over NATS, per-agent tool sandboxes, drop-in configuration for private vs. public agents.

One process, many agents, many channels. Kate handles your personal Telegram; Ana works the WhatsApp sales line; a cron-style poller sweeps Gmail for leads — all sharing one broker, one tool registry, and one memory layer.

Single binary, ~34 MB. No Node, no npm, no Docker required. Stripped: 29 MB. Gzipped: 13 MB. Runs on a fresh VPS, on Termux without root, or as a systemd unit. The closest reference point is OpenClaw (TypeScript, Node): nexo-rs trades JS familiarity for a single static binary, a fault-tolerant NATS broker layer, per-agent capability sandboxes, durable workflows, secrets audit, and Termux-first portability — see vs OpenClaw for the full side-by-side.

flowchart LR
    WA[WhatsApp] --> NATS[(NATS broker)]
    TG[Telegram] --> NATS
    MAIL[Email / Gmail poller] --> NATS
    BROWSER[Browser CDP] --> NATS
    NATS --> ANA[Agent: Ana]
    NATS --> KATE[Agent: Kate]
    NATS --> OPS[Agent: ops-bot]
    ANA --> TOOLS[Tools & extensions]
    KATE --> TOOLS
    OPS --> TOOLS
    TOOLS --> MEM[(Memory: SQLite + sqlite-vec)]
    TOOLS --> LLM{{LLM providers}}

Why it exists

Most "agent frameworks" assume one LLM talking to one user through one UI. Real deployments are not shaped that way:

• Several agents with different personas, models, and skills
• Multiple channels (WA + Telegram + mail) feeding the same agents
• Business logic that is not LLM-driven (scheduled tasks, regex email triage, lead notifications) running next to the LLM loop
• Private prompts and pricing tables alongside an open-source core

nexo-rs is opinionated toward that shape.

What's in the box

Who it is for

• Developers who want to run real agents — not a ChatGPT demo with retrieval.
• Multi-tenant single-install — several agents, several channels, isolated by config.
• Fault-tolerance-first teams — disk queue, DLQ, circuit breakers, single-instance lock, no message drop on reconnect.
• Anyone extending with their own stack — stdio extensions in any language, MCP, drop-in private agents.

What it is not

• Not a chatbot, not a webapp. It has no UI of its own.
• Not a replacement for LangChain/LlamaIndex as a "primitives library". It is an operational runtime.
• Not a channel-abstraction layer. WhatsApp behaves like WhatsApp, Telegram like Telegram. The runtime surfaces channels, not uniforms them.

Next

• Installation
• Quick start
• Architecture overview
• API reference (rustdoc) — every public type in the workspace

Install nexo-rs (LLM-friendly guide)

Share this URL with any AI assistant to teach it how to install nexo-rs end-to-end on Linux or Termux: https://lordmacu.github.io/nexo-rs/install-for-ai.html

The page is intentionally linear: copy-paste each block in order. No menus to navigate, no marketing, every command is deterministic.

What you are installing

• nexo-rs — a Rust multi-agent framework. Single agent binary
  • a cloudflared tunnel for the web admin UI.
• Source: https://github.com/lordmacu/nexo-rs
• Docs (full): https://lordmacu.github.io/nexo-rs/
• License: dual MIT / Apache-2.0

Pick your platform

• Linux (Debian / Ubuntu / Arch / Fedora) → §A
• Termux (Android, no root) → §B

Skip the section that doesn't apply.

§A — Linux install

A.1. System packages

Debian / Ubuntu:

sudo apt update
sudo apt install -y build-essential pkg-config libsqlite3-dev git curl

Arch:

sudo pacman -Syu --needed base-devel pkgconf sqlite git curl

Fedora:

sudo dnf install -y @development-tools pkgconf-pkg-config sqlite-devel git curl

A.2. Rust toolchain

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh -s -- -y
source "$HOME/.cargo/env"
rustup component add rustfmt clippy

A.3. Clone + build

git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
cargo build --release --bin agent

The compiled binary is at ./target/release/agent. Copy it into PATH (optional):

sudo install -m 0755 target/release/agent /usr/local/bin/agent

A.4. First-run wizard

agent setup

Follow the interactive prompts. Defaults are sane. The wizard writes config/agents.d/<your-agent>.yaml, IDENTITY.md, SOUL.md, and any channel YAMLs you opt into.

A.5. Run

agent

Or, for the web admin (loopback HTTP + Cloudflare tunnel):

agent admin

The admin command prints a one-time URL + password to stdout. Open the URL, log in, and configure from the browser.

A.6. (Optional) systemd service

sudo useradd -r -s /bin/false -d /srv/nexo-rs nexo
sudo mkdir -p /srv/nexo-rs
sudo cp -r config target/release/agent /srv/nexo-rs/
sudo chown -R nexo:nexo /srv/nexo-rs

sudo tee /etc/systemd/system/nexo-rs.service > /dev/null <<'EOF'
[Unit]
Description=nexo-rs agent
After=network.target

[Service]
Type=simple
User=nexo
WorkingDirectory=/srv/nexo-rs
ExecStart=/srv/nexo-rs/agent --config /srv/nexo-rs/config
Restart=on-failure
RestartSec=5

[Install]
WantedBy=multi-user.target
EOF

sudo systemctl daemon-reload
sudo systemctl enable --now nexo-rs

Logs: journalctl -u nexo-rs -f.

A.7. (Optional) NATS broker

A single-process install does not need NATS — the runtime falls back to in-process channels. Add NATS only when scaling beyond one host:

curl -L -o /tmp/nats.tar.gz \
  https://github.com/nats-io/nats-server/releases/download/v2.10.20/nats-server-v2.10.20-linux-amd64.tar.gz
tar -xzf /tmp/nats.tar.gz -C /tmp
sudo mv /tmp/nats-server-*/nats-server /usr/local/bin/
sudo systemctl enable --now nats-server  # if you have a unit file

Then in config/broker.yaml set kind: nats and url: nats://127.0.0.1:4222.

§B — Termux install (Android, no root)

B.1. Termux from F-Droid

Install Termux from https://f-droid.org/en/packages/com.termux/. Do not install from the Google Play Store — that build is outdated.

Open Termux. Then:

pkg update
pkg upgrade -y

B.2. Build dependencies

pkg install -y rust git curl sqlite openssl clang pkg-config

Optional extras (only the ones you'll use):

# media transcoding + OCR + youtube downloads
pkg install -y ffmpeg tesseract yt-dlp
# tmux for long-running tunnels and ssh
pkg install -y tmux openssh
# headless Chromium for the browser plugin
pkg install -y tur-repo
pkg install -y chromium
# Termux:API for sensors / SMS / clipboard
pkg install -y termux-api
# (also install the Termux:API companion app from F-Droid)

B.3. Clone + build

cd ~
git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
cargo build --release --bin agent

B.4. First-run wizard

./target/release/agent setup

B.5. Run

./target/release/agent

Or with the admin UI (the cloudflared tunnel works on Termux):

./target/release/agent admin

B.6. Keep running with the screen off

Termux apps get killed on doze unless you disable battery optimizations and acquire a wake lock:

1. Disable optimizations: Android Settings → Apps → Termux → Battery → Unrestricted.

2. Wake lock: in Termux, type:

   termux-wake-lock
   

3. (Optional) auto-restart on boot: install Termux:Boot from F-Droid, then create ~/.termux/boot/00-nexo-rs:

   mkdir -p ~/.termux/boot
   cat > ~/.termux/boot/00-nexo-rs <<'EOF'
   #!/data/data/com.termux/files/usr/bin/sh
   termux-wake-lock
   cd ~/nexo-rs
   ./target/release/agent --config ./config >> ~/nexo-rs/agent.log 2>&1
   EOF
   chmod +x ~/.termux/boot/00-nexo-rs
   

B.7. Termux-specific tip — Chromium flags

The browser plugin (plugins: [browser]) needs the right Chromium launch flags on Termux. The defaults already cover Android; nothing extra to set. Just make sure chromium is on PATH (it is, after pkg install chromium).

Config layout (both platforms)

After agent setup runs, the project tree looks like:

nexo-rs/
├── config/
│   ├── agents.yaml          # opt-in dev defaults
│   ├── agents.d/            # your agents land here
│   │   └── <slug>.yaml
│   ├── broker.yaml          # NATS or local
│   ├── llm.yaml             # provider keys + model
│   └── plugins/             # one YAML per channel plugin
├── secrets/                 # mode 0600 token files (gitignored)
├── data/                    # SQLite databases (memory, taskflow, transcripts)
├── target/release/agent     # the built binary
└── agent.log                # if you redirected stdout

Edit YAML by hand or use the web admin (agent admin).

Troubleshooting

• cargo build fails with linker errors on Linux — install build-essential and pkg-config (§A.1).
• cargo build hits out of memory on Termux — close other apps, or build with one job: cargo build --release -j 1.
• agent exits immediately with failed to load config — run agent setup first; the wizard creates the missing files.
• WhatsApp QR pairing fails on Termux — make sure the device is on the same network as your phone, then open the QR pairing URL the daemon prints.
• Admin tunnel URL doesn't respond — Cloudflare's quick tunnel occasionally rotates; restart agent admin and copy the new URL.

Useful commands after install

agent --help                                  # all subcommands
agent doctor capabilities --json              # which env toggles are armed
agent setup doctor                            # audit configured secrets
agent ext doctor --json                       # extension health
agent flow list                               # taskflow admin
agent dlq list                                # dead-letter queue

Full reference: https://lordmacu.github.io/nexo-rs/cli/reference.html

When asking an AI for help

Paste this URL into your prompt:

Install nexo-rs from https://lordmacu.github.io/nexo-rs/install-for-ai.html
on this machine. The OS is <Linux distro / Termux>. Stop after each
section to confirm output looks right.

The page above is the canonical, copy-paste-friendly install path. The full mdBook (https://lordmacu.github.io/nexo-rs/) covers the same ground in more depth — link there once the agent is up.

Installation

Pick the channel that matches your environment. Every channel produces the same nexo binary; the differences are in how it gets onto your machine and which dependencies come bundled.

Channel matrix

Quickest path — Docker

docker pull ghcr.io/lordmacu/nexo-rs:latest
docker run --rm \
  -v $(pwd)/config:/app/config:ro \
  -v $(pwd)/data:/app/data \
  -p 8080:8080 -p 9090:9090 \
  ghcr.io/lordmacu/nexo-rs:latest --help

The image is multi-arch (linux/amd64 + linux/arm64), built fresh on every push to main and every v* tag, with SBOM and SLSA provenance attestations. Full guide: Docker.

Build from source

For contributors and operators who want to track main directly:

git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
cargo build --release --bin nexo
./target/release/nexo --help

The workspace compiles 22 crates and produces the nexo binary plus a few smoke-test bins (browser-test, integration-browser-check, llm_smoke). Toolchain is pinned to Rust 1.80 (MSRV) via rust-toolchain.toml — no manual channel selection needed.

Prerequisites

• Rust 1.80+ (rustup recommended)
• NATS running locally or reachable over the network — for development:

  docker run -p 4222:4222 nats:2.10-alpine
  

  Production setup: see broker.yaml.
• Git (the memory subsystem uses per-agent workspace-git)
• Chrome / Chromium (only if you plan to use the browser plugin)

Verification

./target/release/nexo --version
cargo test --workspace --lib

nexo --version prints the build provenance line (commit + build timestamp) so a bug report carries enough context to reproduce.

Bootstrap script

For native or Termux installs, ./scripts/bootstrap.sh automates the whole process — installs the system deps, downloads NATS if not present, scaffolds config/, and runs the setup wizard.

./scripts/bootstrap.sh           # interactive
./scripts/bootstrap.sh --yes     # accept all defaults

The script auto-detects Termux ($PREFIX set) and switches to pkg install + broker.type: local so you don't need root or NATS on a phone.

Next steps

• Quick start — first agent running in five minutes
• Setup wizard — pair channels and wire secrets
• Docker — compose stack, secrets, GHCR pulls
• Nix flake — nix run, dev shell
• Native install — detailed no-Docker setup
• Termux install — phone-hosted personal agent

Native install (no Docker)

If you'd rather run nexo-rs directly on a Linux / macOS host — development loop, single-machine deploy, restricted container environment — this page walks through every step and names the bootstrap script that automates it.

Fast path

git clone git@github.com:lordmacu/nexo-rs.git
cd nexo-rs
./scripts/bootstrap.sh

scripts/bootstrap.sh verifies prerequisites, installs a local NATS, creates the runtime directories, stages example configs, and builds the agent binary. Re-runnable — each step is idempotent.

Keep reading for what it actually does (and what to do when a step needs manual intervention).

Prerequisites

On Ubuntu / Debian:

sudo apt update
sudo apt install -y build-essential pkg-config libsqlite3-dev git curl

On macOS:

xcode-select --install
brew install sqlite git

Install Rust

curl --proto '=https' --tlsv1.2 -sSf https://sh.rustup.rs | sh
source "$HOME/.cargo/env"
rustup component add rustfmt clippy

The repo's rust-toolchain.toml pins the channel; no manual version pick is needed.

Install NATS

Pick one path:

Option A — native NATS server

# Linux x86_64
curl -L -o /tmp/nats.tar.gz \
  https://github.com/nats-io/nats-server/releases/download/v2.10.20/nats-server-v2.10.20-linux-amd64.tar.gz
tar -xzf /tmp/nats.tar.gz -C /tmp
sudo mv /tmp/nats-server-*/nats-server /usr/local/bin/

For macOS: brew install nats-server.

Start it:

nats-server -js                      # foreground
nats-server -js -D                   # foreground with debug
# or, as a systemd service: see below

Option B — dev throwaway via Docker

Even on a "no-Docker" box, a single short-lived container for the broker is often fine:

docker run -d --name nexo-nats --restart unless-stopped \
  -p 4222:4222 -p 8222:8222 nats:2.10-alpine

This is the same broker the compose stack would use; only the broker itself runs in a container.

Systemd unit (Linux, production)

/etc/systemd/system/nats-server.service:

[Unit]
Description=NATS Server
After=network.target

[Service]
Type=simple
ExecStart=/usr/local/bin/nats-server -js
Restart=on-failure
LimitNOFILE=65536

[Install]
WantedBy=multi-user.target

sudo systemctl daemon-reload
sudo systemctl enable --now nats-server

Build nexo-rs

git clone git@github.com:lordmacu/nexo-rs.git
cd nexo-rs
cargo build --release

The output is ./target/release/agent. Symlink it into $PATH if you want:

sudo ln -sf "$(pwd)/target/release/agent" /usr/local/bin/agent

Prepare runtime directories

mkdir -p ./data/{queue,workspace,media,transcripts}
mkdir -p ./secrets          # gitignored; holds API keys, nkey files, etc.
chmod 700 ./secrets         # restrictive — the credential gauntlet checks this

Stage config

The repo ships config/*.yaml with safe defaults. Override whatever you need:

# Optional: copy the ana sales agent template into the gitignored dir
cp config/agents.d/ana.example.yaml config/agents.d/ana.yaml

# Add an API key:
export MINIMAX_API_KEY=...
export MINIMAX_GROUP_ID=...
# or write to secrets/ files referenced from config/llm.yaml via ${file:...}

See Configuration — layout for the full reference.

Pair channels and set secrets

./target/release/agent setup

The wizard pairs WhatsApp / Telegram / Google / LLM credentials interactively. See Setup wizard.

First run

./target/release/agent --config ./config

Watch the startup summary — it tells you exactly which plugins loaded, which extensions were skipped and why, and whether the broker is reachable. If anything's missing, the log line names the specific file or env var to fix.

Run as a systemd service

/etc/systemd/system/nexo-rs.service:

[Unit]
Description=nexo-rs agent
Requires=nats-server.service
After=nats-server.service

[Service]
Type=simple
User=nexo
Group=nexo
WorkingDirectory=/srv/nexo-rs
Environment=RUST_LOG=info
Environment=AGENT_ENV=production
ExecStart=/usr/local/bin/agent --config /srv/nexo-rs/config
Restart=on-failure
RestartSec=5
# Optional: restrict where the agent can write
ReadWritePaths=/srv/nexo-rs/data /srv/nexo-rs/secrets

[Install]
WantedBy=multi-user.target

sudo useradd -r -s /bin/false -d /srv/nexo-rs nexo
sudo chown -R nexo:nexo /srv/nexo-rs
sudo systemctl daemon-reload
sudo systemctl enable --now nexo-rs

Logs:

journalctl -u nexo-rs -f

macOS launchd

~/Library/LaunchAgents/dev.nexo-rs.agent.plist:

<?xml version="1.0" encoding="UTF-8"?>
<!DOCTYPE plist PUBLIC "-//Apple//DTD PLIST 1.0//EN"
  "http://www.apple.com/DTDs/PropertyList-1.0.dtd">
<plist version="1.0">
<dict>
  <key>Label</key>          <string>dev.nexo-rs.agent</string>
  <key>WorkingDirectory</key><string>/Users/you/nexo-rs</string>
  <key>ProgramArguments</key>
  <array>
    <string>/Users/you/nexo-rs/target/release/agent</string>
    <string>--config</string><string>/Users/you/nexo-rs/config</string>
  </array>
  <key>EnvironmentVariables</key>
  <dict>
    <key>RUST_LOG</key><string>info</string>
  </dict>
  <key>RunAtLoad</key>      <true/>
  <key>KeepAlive</key>      <true/>
</dict>
</plist>

launchctl load -w ~/Library/LaunchAgents/dev.nexo-rs.agent.plist
launchctl start dev.nexo-rs.agent

Verify

agent status                    # lists running agents
curl localhost:8080/ready       # readiness
curl localhost:9090/metrics     # Prometheus metrics

See Metrics + health.

Upgrading

cd nexo-rs
git pull
cargo build --release
sudo systemctl restart nexo-rs      # Linux
# or: launchctl kickstart -k gui/$UID/dev.nexo-rs.agent   # macOS

The graceful shutdown sequence drains in-flight work and persists the disk queue before exit.

Uninstalling

sudo systemctl disable --now nexo-rs nats-server
sudo rm /etc/systemd/system/{nexo-rs,nats-server}.service
sudo rm /usr/local/bin/{agent,nats-server}
sudo userdel nexo
rm -rf /srv/nexo-rs

See also

• Quick start — the five-minute dev loop
• Docker — container path for comparison
• Setup wizard
• Configuration

Termux (Android) install

Run nexo-rs directly on an Android phone under Termux. No Docker, no server — a self-hosted agent in your pocket.

Use this path for a personal agent (one phone, one WhatsApp, one Telegram). For multi-tenant / multi-process deployments the regular Linux setup on a server is the right shape.

Quickest path — pre-built .deb

Once a v* release is published (recipe lives in packaging/termux/build.sh), download the asset and install with one command:

# Inside Termux on the phone:
curl -LO https://github.com/lordmacu/nexo-rs/releases/latest/download/nexo-rs_aarch64.deb
pkg install ./nexo-rs_aarch64.deb

The deb pulls the runtime deps Termux already ships (libsqlite, openssl, ffmpeg, tesseract, python, yt-dlp). Its postinst scaffolds ~/.nexo/{data,secret} and prints the next steps. Skip the build-from-source section below if this works.

Root vs non-root

Everything in this guide runs without root. You do not need to root your phone to self-host nexo-rs on it.

Root only unlocks extras:

Short version: don't root just for nexo-rs. Root if you want the full compose stack in a Linux-Deploy chroot, otherwise skip it.

What works

What needs a tweak

Prerequisites

From a fresh Termux install:

pkg update
pkg install -y rust git curl sqlite openssl clang pkg-config

Optional (enables specific skills):

pkg install -y ffmpeg tesseract yt-dlp tmux openssh

Optional (browser plugin):

pkg install -y tur-repo
pkg install -y chromium

Optional (run in background without the terminal session alive):

pkg install -y termux-services termux-api
# install the companion app "Termux:API" from F-Droid

Fast path — bootstrap script

The repo's scripts/bootstrap.sh auto-detects Termux and picks the right defaults:

git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
./scripts/bootstrap.sh --yes

What it does on Termux:

1. Verifies rust, git, curl, sqlite from pkg
2. Downloads the static nats-server Go binary (arm64), drops it in $PREFIX/bin/ — or skip with --nats=skip to use the local broker
3. Creates ./data/** and ./secrets/ (with Termux-compatible perms)
4. Stages config/agents.d/*.example.yaml → *.yaml if missing
5. Runs cargo build --release (grab a coffee — ~20–40 min on phone hardware)
6. Optionally launches agent setup to pair channels

Expect a ~60–100 MB final binary.

Manual install

1. Install Rust and deps

pkg install -y rust git curl sqlite openssl clang pkg-config

2. Clone and build

git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
cargo build --release --bin agent

3. Broker

Option A — local (simplest):

# config/broker.yaml
broker:
  type: local
  persistence:
    enabled: true
    path: ./data/queue

No NATS binary needed. All pub/sub stays in-process.

Option B — native NATS binary:

curl -L -o /tmp/nats.tar.gz \
  https://github.com/nats-io/nats-server/releases/download/v2.10.20/nats-server-v2.10.20-linux-arm64.tar.gz
tar -xzf /tmp/nats.tar.gz -C /tmp
install -m 0755 "$(find /tmp -name nats-server -type f | head -1)" \
  $PREFIX/bin/nats-server
nats-server -js &

Go binaries are static and work on Termux without libc surprises.

4. Runtime directories and secrets

mkdir -p ./data/{queue,workspace,media,transcripts} ./secrets

Termux stores files under /data/data/com.termux/files/home by default. Avoid pointing config paths at /sdcard — Android's scoped-storage model breaks directory permissions there.

5. Relax the credentials perm check

Android's filesystem doesn't support the same permission bits as Linux in the same way. The credentials gauntlet would refuse to boot with false-positive warnings:

export CHAT_AUTH_SKIP_PERM_CHECK=1

Add it to ~/.termux/termux.properties or a wrapper shell script so it's set every time.

6. Launch the wizard

./target/release/agent setup

For the WhatsApp pairing step, the wizard renders the QR as Unicode blocks directly in the terminal — scan from the phone's WhatsApp app (Settings → Linked Devices). No public tunnel needed.

7. Run the agent

termux-wake-lock                # keep CPU awake even with screen off
./target/release/agent --config ./config

Staying alive in the background

Android's aggressive task killing is the biggest operational surprise. Pick one:

A — termux-wake-lock + foreground notification

termux-wake-lock
# agent in foreground:
./target/release/agent --config ./config

The wake-lock persists until you run termux-wake-unlock or kill the session. Minimum friction, most reliable.

B — termux-services (runit)

pkg install -y termux-services
sv-enable termux-services
mkdir -p ~/.config/service/nexo-rs
cat > ~/.config/service/nexo-rs/run <<'EOF'
#!/data/data/com.termux/files/usr/bin/sh
cd /data/data/com.termux/files/home/nexo-rs
export CHAT_AUTH_SKIP_PERM_CHECK=1
exec ./target/release/agent --config ./config 2>&1
EOF
chmod +x ~/.config/service/nexo-rs/run
sv up nexo-rs
sv status nexo-rs

C — Termux:Boot (start on device boot)

Install the Termux:Boot app from F-Droid, then:

mkdir -p ~/.termux/boot
cat > ~/.termux/boot/start-agent <<'EOF'
#!/data/data/com.termux/files/usr/bin/sh
termux-wake-lock
cd /data/data/com.termux/files/home/nexo-rs
export CHAT_AUTH_SKIP_PERM_CHECK=1
exec ./target/release/agent --config ./config
EOF
chmod +x ~/.termux/boot/start-agent

Disabling the browser plugin

If you don't need headless browser control (most phone-hosted agents don't), drop it from config/extensions.yaml:

extensions:
  disabled: [browser]

Or, if you have tur-repo chromium installed and want nexo-rs to spawn it, use the browser.args field to forward the flags Termux needs:

# config/plugins/browser.yaml
browser:
  headless: true
  executable: /data/data/com.termux/files/usr/bin/chromium
  args:
    - --no-sandbox
    - --disable-dev-shm-usage
    - --disable-gpu

The built-in launch flags still apply; args is appended after them so you can also override any of the built-ins (Chrome's CLI parser uses last-wins).

Alternative: launch chromium yourself and attach via cdp_url:

# config/plugins/browser.yaml
browser:
  # Start chromium yourself with:
  #   chromium --headless --no-sandbox --disable-dev-shm-usage \
  #            --disable-gpu --remote-debugging-port=9222 &
  cdp_url: http://127.0.0.1:9222

When cdp_url is set, args is ignored — nexo-rs doesn't spawn Chrome, only connects to yours.

Verify

curl localhost:8080/ready
curl localhost:9090/metrics
./target/release/agent status

Upgrading

cd ~/nexo-rs
git pull
cargo build --release
# restart under whichever method you picked (wake-lock / runit / Boot)

Android's graceful shutdown still runs on SIGTERM — closing the Termux session or killing the process drains the disk queue cleanly.

See also

• Installation — shared prerequisites
• Native install (no Docker) — the Linux/macOS path the Termux recipe is a sibling of
• Plugins — Browser
• Config — broker.yaml

Install — Nix

Nexo ships a Nix flake that pins the toolchain (Rust 1.80, MSRV) and the native build deps so a contributor or operator can go from clean shell to working binary without touching the host system.

Run without installing

nix run github:lordmacu/nexo-rs -- --help

First invocation builds from source (~3-5 min on cold cache); subsequent runs hit the local Nix store.

Build a local binary

nix build github:lordmacu/nexo-rs
./result/bin/nexo --help

The binary is the same nexo produced by cargo build --release --bin nexo. Outputs a result/ symlink the operator can link into /usr/local/bin/ or copy elsewhere.

Contributor dev shell

git clone https://github.com/lordmacu/nexo-rs
cd nexo-rs
nix develop

Drops you into a shell with:

• rustc 1.80 + cargo + clippy + rustfmt + rust-src
• cargo-edit, cargo-watch, cargo-nextest, cargo-deny
• mdbook + mdbook-mermaid (for mdbook build docs)
• sqlite, pkg-config, openssl, libgit2 (build deps)

RUST_LOG=info is exported by default. The toolchain version is pinned in flake.nix — bump in lockstep with [workspace.package].rust-version in Cargo.toml.

What the flake does NOT install

The nexo binary alone is not enough for full functionality. Runtime tools the channel plugins shell out to live at the system level, not in the flake:

• Chrome / Chromium — required by the browser plugin
• cloudflared — used by the tunnel plugin
• ffmpeg — media transcoding for WhatsApp voice notes
• tesseract-ocr — OCR skill
• yt-dlp — the yt-dlp extension

Operators install these via their distro's package manager. The native install guide lists the apt / pacman / brew commands. The Docker image bundles all of them — that's the path of least friction for a "just works" deploy.

Pinning a release

Once v* tags are published, pin to a specific release:

nix run github:lordmacu/nexo-rs/v0.1.1 -- --help

Or in a flake input:

{
  inputs.nexo-rs.url = "github:lordmacu/nexo-rs/v0.1.1";
}

Verifying the build

nix flake check

Runs nix flake check — verifies the flake metadata, evaluates all outputs (packages, apps, devShells, formatter) without actually building. Useful in CI to catch flake regressions early.

Troubleshooting

• "experimental feature 'flakes' is disabled" — add to ~/.config/nix/nix.conf:

  experimental-features = nix-command flakes
  

• First build is very slow — the build re-fetches and re-compiles every cargo dependency in the sandbox. Subsequent builds are cached. A future Phase 27.x will publish a cachix cache so nix build pulls the binary directly.
• Build fails on macOS arm64 — git2-rs occasionally lags on Apple silicon. Workaround: build the binary inside the Docker image instead (see Docker).

Quick start

Minimum viable agent running in five minutes. Covers: NATS, one agent, one channel, one LLM key.

1. Start NATS

docker run -d --name nexo-nats -p 4222:4222 nats:2.10-alpine

2. Build the binary

git clone git@github.com:lordmacu/nexo-rs.git
cd nexo-rs
cargo build --release

3. Provide an LLM key

Pick one provider to get started. MiniMax M2.5 is the primary:

export MINIMAX_API_KEY=your-key
export MINIMAX_GROUP_ID=your-group-id

Or Anthropic:

export ANTHROPIC_API_KEY=sk-ant-...

The shipped config/llm.yaml reads both via ${ENV_VAR}.

4. Run the setup wizard

./target/release/agent setup

The wizard walks you through:

• Choosing a default LLM provider
• Pairing any channels you want (WhatsApp QR, Telegram bot token, Google OAuth)
• Writing secrets into ./secrets/ (gitignored)

See Setup wizard for the full step-by-step.

5. Run the agent

./target/release/agent --config ./config

First boot emits a startup summary listing:

• which plugins loaded
• which extensions were discovered / skipped (and why)
• which LLM providers are wired
• the NATS connection state

If anything is missing, the log line tells you exactly what to fix.

6. Talk to it

If you paired Telegram, send a message to the bot. If you paired WhatsApp, send a message to the paired number. The agent replies via the same channel.

What you just ran

sequenceDiagram
    participant U as User
    participant CH as Channel plugin
    participant B as NATS
    participant A as Agent runtime
    participant L as LLM provider

    U->>CH: Inbound message
    CH->>B: publish plugin.inbound.<channel>
    B->>A: deliver
    A->>L: chat.completion(tools)
    L-->>A: assistant turn
    A->>B: publish plugin.outbound.<channel>
    B->>CH: deliver
    CH-->>U: Outbound reply

Next

• Setup wizard — every wizard step in detail
• Configuration layout
• Architecture overview

Setup wizard

The setup wizard is the recommended way to configure nexo-rs on a fresh install. It pairs channels, writes secrets, and patches the YAML config files so the runtime boots with everything it needs.

./target/release/agent setup

Run it from the repo root (or wherever your config/ directory lives).

What the wizard does

flowchart TD
    START([agent setup]) --> MENU{Menu}
    MENU --> LLM[LLM provider]
    MENU --> WA[WhatsApp pairing]
    MENU --> TG[Telegram bot]
    MENU --> GOOG[Google OAuth]
    MENU --> MEM[Memory DB location]
    MENU --> INFRA[NATS + runtime]
    MENU --> SKILLS[Enable / disable skills]

    LLM --> WRITE1[Write secrets/<br/>patch llm.yaml]
    WA --> QR[Scan QR<br/>write session dir]
    TG --> TOKEN[Ask bot token<br/>write secret]
    GOOG --> OAUTH[Open browser<br/>PKCE flow]
    MEM --> WRITE2[Patch memory.yaml]
    INFRA --> WRITE3[Patch broker.yaml]
    SKILLS --> WRITE4[Patch extensions.yaml]

    WRITE1 --> DONE([Done])
    QR --> DONE
    TOKEN --> DONE
    OAUTH --> DONE
    WRITE2 --> DONE
    WRITE3 --> DONE
    WRITE4 --> DONE

Every step is optional. You can run setup repeatedly — each section is idempotent.

Steps in detail

LLM provider

Prompts for the default provider (MiniMax, Anthropic, OpenAI-compat, Gemini). Writes the API key to ./secrets/<provider>_api_key.txt and ensures config/llm.yaml references it via ${file:...} or the corresponding env var.

WhatsApp pairing (multi-instance)

Per-agent. Asks which agent you are pairing and which instance label to use (personal, work, …). Each instance gets its own session dir under ./data/workspace/<agent>/whatsapp/<instance> and an allow_agents list (defense-in-depth ACL). The wizard:

1. Normalises config/plugins/whatsapp.yaml to sequence form (legacy single-mapping entries are auto-converted on first edit).
2. Upserts the entry by instance label.
3. Writes credentials.whatsapp: <instance> on the chosen agent's YAML — agents.yaml if the agent lives there, otherwise the matching agents.d/*.yaml.
4. Launches the pairing loop and renders the QR as Unicode blocks. Scan with WhatsApp → Settings → Linked Devices.
5. Runs the credential gauntlet so any drift surfaces immediately.

Re-run the wizard once per number you want to pair; instance labels are append-friendly.

Telegram bot (multi-instance)

Same shape as WhatsApp. Asks for instance label (default <agent>_bot) and bot token from @BotFather. Token lands at ./secrets/<instance>_telegram_token.txt with mode 0o600; the YAML references it via ${file:...} so secrets never live in telegram.yaml directly. Adds credentials.telegram: <instance> on the agent.

Google OAuth

The wizard writes one entry per agent in config/plugins/google-auth.yaml:

google_auth:
  accounts:
    - id: ana@google
      agent_id: ana
      client_id_path:     ./secrets/ana_google_client_id.txt
      client_secret_path: ./secrets/ana_google_client_secret.txt
      token_path:         ./secrets/ana_google_token.json
      scopes: [https://www.googleapis.com/auth/gmail.modify]

Two consent flows are offered after the YAML is written:

• Device-code (default — works headless / over SSH): the wizard prints verification_url + a 6-character user_code. Open the URL on any device, type the code, approve. The wizard polls oauth2.googleapis.com/token until approval and persists the refresh_token at token_path (mode 0o600).
• Skip and consent later via the google_auth_start LLM tool — uses the loopback PKCE flow, requires a local browser.

Scopes are comma-separated at the prompt; defaults to gmail.modify. Re-running with a different id adds a second account; re-running with the same id overwrites in place.

Memory DB location

Lets you pick where the SQLite long-term memory file lives. Default is ./data/memory.db. Per-agent isolation is on by default — each agent gets its own DB file under its workspace.

Infrastructure (NATS + runtime)

Asks for the NATS URL, optional user/password, and timeouts. Patches config/broker.yaml.

Skills on/off

Lets you selectively disable shipped extensions you don't plan to use (reduces tool surface exposed to the LLM).

Files the wizard touches

Every YAML patch preserves existing keys and comments via the yaml_patch module — your hand edits survive.

Re-running

Re-run agent setup as many times as you want. Paired channels are detected and skipped unless you explicitly ask to re-pair. To wipe a paired session:

./target/release/agent setup wipe whatsapp --agent ana

Troubleshooting

• WhatsApp QR expires too fast → the QR refreshes every ~20s; the wizard re-renders. Scan from the phone with a stable network.
• Google OAuth fails with redirect_uri_mismatch → the wizard binds to 127.0.0.1:<port>; make sure your OAuth client allows http://127.0.0.1 as a redirect URI.
• NATS unreachable → the wizard will warn but still write config. The runtime's disk queue will drain once NATS comes back.

Agent-centric setup wizard

The hub menu's Configurar agente (canal, modelo, idioma, skills) entry drops the operator into a per-agent submenu. Where the rest of the wizard groups actions by service (Telegram, OpenAI, the browser plugin), this submenu groups them by agent: pick one agent up front, then mutate its model, language, channels, and skills from a single dashboard. Every action reuses the existing channel / LLM / skill flows underneath, so behavior stays in lockstep with the rest of the wizard.

./target/release/agent setup
# → Configurar agente (canal, modelo, idioma, skills)

Dashboard

Agente: kate
  Modelo:   anthropic / claude-haiku-4-5  [creds ✔]
  Idioma:   es
  Canales:  ✔ telegram:default  (bound)
            ✗ whatsapp:default  (unbound)
  Skills:   8 / 24 attached

The dashboard is recomputed from disk on every loop iteration, so the screen always reflects the most recent YAML state.

Action menu

After the dashboard renders, the operator picks one of:

YAML mutations

All mutations land atomically (tempfile + rename) and are gated by the same process-wide YAML mutex the legacy upsert path uses, so concurrent wizard sessions don't corrupt the file.

Hot-reload

After every successful mutation, the wizard fires a best-effort nexo --config <dir> reload so a running daemon picks up the YAML edit without a manual restart. The call is fire-and-forget: when the binary isn't on PATH or the daemon isn't running, the wizard keeps going silently.

Where the code lives

• crates/setup/src/agent_wizard.rs — submenu + dashboard.
• crates/setup/src/yaml_patch.rs — read_agent_field, upsert_agent_field, remove_agent_field, append_agent_list_item, remove_agent_list_item.
• crates/setup/tests/agent_wizard_yaml.rs — schema-roundtrip tests that re-parse the mutated YAML through nexo_config::AgentsConfig.

Verifying releases

Every Nexo release artifact is signed with Sigstore Cosign using keyless OIDC — no long-lived private key, no PGP key management, no out-of-band trust establishment. The signature is tied to the GitHub Actions workflow run that produced the artifact, and a public record lives in the Rekor transparency log.

Why keyless

Traditional signing requires a long-lived signing key. If it leaks, every past release becomes suspect. Keyless signing instead anchors each signature to:

1. The GitHub Actions OIDC identity of the workflow run (https://token.actions.githubusercontent.com)
2. The specific repo + workflow file that ran (https://github.com/lordmacu/nexo-rs/.github/workflows/...)
3. The commit + ref the workflow built from

A short-lived certificate (10 min validity) is issued by Sigstore's fulcio CA, the artifact is signed with it, and the whole bundle is recorded in rekor (immutable). To forge a signature, an attacker would need to compromise GitHub's OIDC infra and the exact workflow path — and even then the forgery shows up in the public log.

Install Cosign

# macOS:
brew install cosign

# Linux (Debian/Ubuntu):
curl -L "https://github.com/sigstore/cosign/releases/latest/download/cosign-linux-amd64" \
  -o /usr/local/bin/cosign
chmod +x /usr/local/bin/cosign

# Linux (Fedora/RHEL):
sudo dnf install cosign

# Verify the install:
cosign version

Verify a Docker image

Every image at ghcr.io/lordmacu/nexo-rs is cosign-signed by the docker.yml workflow. Verify any tag with:

cosign verify ghcr.io/lordmacu/nexo-rs:latest \
  --certificate-identity-regexp 'https://github.com/lordmacu/nexo-rs/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com

A successful verification prints the full certificate + the Rekor entry URL. Anything else (signature missing, identity mismatch, broken cert chain) means don't trust this image — check the release notes, file an issue.

Verify a downloaded binary / .deb / .rpm / .tar.gz

The sign-artifacts.yml workflow attaches three files next to every release asset:

• <asset>.sig — the raw signature
• <asset>.pem — the leaf certificate
• <asset>.bundle — combined Sigstore bundle (preferred; carries the inclusion proof)

Verify with the bundle (recommended, single command):

cosign verify-blob \
  --bundle nexo-rs_0.1.1_amd64.deb.bundle \
  --certificate-identity-regexp 'https://github.com/lordmacu/nexo-rs/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  nexo-rs_0.1.1_amd64.deb

Or with the standalone .sig + .pem if you prefer:

cosign verify-blob \
  --signature nexo-rs_0.1.1_amd64.deb.sig \
  --certificate nexo-rs_0.1.1_amd64.deb.pem \
  --certificate-identity-regexp 'https://github.com/lordmacu/nexo-rs/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  nexo-rs_0.1.1_amd64.deb

Verify in CI / scripted contexts

Drop this in a deploy pipeline:

#!/usr/bin/env bash
set -euo pipefail

ASSET="${1:?usage: $0 <asset-path>}"
BUNDLE="${ASSET}.bundle"

if [ ! -f "$BUNDLE" ]; then
    echo "ERROR: $BUNDLE missing — refusing to deploy unsigned artifact" >&2
    exit 1
fi

cosign verify-blob \
  --bundle "$BUNDLE" \
  --certificate-identity-regexp 'https://github.com/lordmacu/nexo-rs/.*' \
  --certificate-oidc-issuer https://token.actions.githubusercontent.com \
  "$ASSET" \
  || { echo "ERROR: signature verification failed for $ASSET" >&2; exit 2; }

Inspecting the transparency log

Every signature is searchable on Rekor:

# Search by artifact sha256:
cosign tree ghcr.io/lordmacu/nexo-rs:latest

The output shows every cosign-related artifact attached to the image (signatures, attestations, SBOMs) plus the Rekor log index where each was recorded.

What if verification fails

1. Identity regex doesn't match — the asset may have been built from a fork / unofficial workflow. Re-download from the GitHub release page directly.
2. bundle file missing — older releases (pre-Phase 27.3) don't have signatures. Tag v0.1.1 is the first signed release.
3. Cert chain expired / revoked — Sigstore's fulcio root CA has a long lifespan, but the leaf cert is short-lived. cosign automatically fetches the right TUF root; if you see chain errors run cosign initialize to refresh local trust roots.
4. Network errors talking to Rekor / Fulcio — both have CDN in front. Retry, or use --insecure-ignore-tlog for local verification (drops the transparency log check — only safe in air-gapped trust contexts).

Out of scope (for now)

• Long-lived PGP keys for the apt / yum repos — needs Phase 27.4 signed-repo work to consume them on the user side. Until that ships, .deb / .rpm signatures live in the Cosign world only.
• A Homebrew bottle-signing path that lets brew validate without the OIDC chain — Phase 27.6 follow-up.

Reproducible builds + SBOM

Every Nexo release ships with two artefacts that let an operator verify provenance and exact composition:

1. CycloneDX SBOM (sbom-cyclonedx.json) — every cargo dependency at the exact version + hash that was compiled into the binary.
2. SPDX SBOM (sbom-spdx.json) — full filesystem scan via syft, captures anything that wasn't a cargo dep (bundled binaries, generated assets, vendored data files).

Both SBOMs are Cosign-signed (*.bundle) using the same keyless OIDC chain documented in Verifying releases.

Reading the SBOMs

# Pretty-print the CycloneDX dep tree:
jq '.components | map({name, version, purl})' sbom-cyclonedx.json | less

# Find a specific crate:
jq '.components[] | select(.name == "tokio")' sbom-cyclonedx.json

# Audit the cargo deps with `cargo-audit` (run against the SBOM,
# without rebuilding):
cargo audit --db ~/.cargo/advisory-db --json | \
  jq -r '.vulnerabilities.list[].advisory.id'

Reproducible build claim

The release workflow targets bit-identical binary between two runs given the same git sha + rust-toolchain.toml + Cargo.lock. The pipeline pins:

• Rust toolchain: rust-toolchain.toml fixes the channel + components.
• Dependency versions: Cargo.lock is committed and --locked is used by every release build.
• Build environment: GitHub Actions ubuntu-latest runner + cargo build --release with no RUSTFLAGS overrides.
• Build provenance: SLSA Level 2 attestation generated by actions/attest-build-provenance (Phase 27.2 wiring).
• Cosign signature: each binary + SBOM signed via OIDC (Phase 27.3).

Reproducing a release locally

# 1. Check out the exact tag.
git clone https://github.com/lordmacu/nexo-rs && cd nexo-rs
git checkout v0.1.1

# 2. Build with the locked deps.
rustup show       # confirms the toolchain matches rust-toolchain.toml
cargo build --release --bin nexo --locked

# 3. Compare your binary's sha256 against the release asset:
sha256sum target/release/nexo
# Expected: same hash listed in `sha256sums.txt` on the GitHub release.

If the hashes don't match: the build is not reproducible on your host. Common reasons:

• Different glibc version → embedded __VERSIONED_SYMBOL strings drift. The release workflow runs on ubuntu-latest (currently Ubuntu 24.04, glibc 2.39); building on Debian 12 (glibc 2.36) produces different bytes.
• Different LLVM in your local rustc build (rare, mostly affects Mac users compiling with stable + nightly side-by-side).
• Local ~/.cargo/config.toml injecting RUSTFLAGS.
• Build PROFILE-DEV vs PROFILE-RELEASE.

For a guaranteed bit-identical reproduction, build inside the same container the workflow uses:

docker run --rm -v $(pwd):/src -w /src \
  rust:1.80-bookworm \
  cargo build --release --bin nexo --locked

This reproduces what the GitHub Actions runner would do — same glibc, same toolchain version, same LLVM.

SLSA verification

The workflow attaches an attestation.intoto.jsonl (SLSA Level 2 provenance) per release. Verify with slsa-verifier:

go install github.com/slsa-framework/slsa-verifier/v2/cli/slsa-verifier@latest

slsa-verifier verify-artifact nexo \
  --provenance-path attestation.intoto.jsonl \
  --source-uri github.com/lordmacu/nexo-rs \
  --source-tag v0.1.1

A green verification proves:

• The artefact came from the lordmacu/nexo-rs repo
• It was built by a GitHub-hosted runner (not a fork or local box)
• The build inputs match what's recorded in the provenance

Auditing for known CVEs

The SBOM lets cargo-audit work without rebuilding:

# Convert CycloneDX → cargo-audit's format:
cyclonedx-cli convert --input-format json \
  --output-format json sbom-cyclonedx.json | \
  jq '...' > deps.json

# Or just feed it to grype (broader scope, multi-format):
grype sbom:./sbom-cyclonedx.json
grype sbom:./sbom-spdx.json

Grype catches CVEs across both Rust crates and any system-level deps captured by syft.

Out of scope (deferred)

• apk / pkg SBOM for the Termux deb — Termux's package metadata doesn't speak SPDX yet. The release SBOMs cover the same artifact contents though.
• Reproducible Docker image layers — the current Dockerfile uses apt-get update && apt-get install which pulls whatever's latest at build time. Pinning to specific Debian package versions is a follow-up (Phase 34 hardening).

Architecture overview

nexo-rs is a single-process multi-agent runtime. One binary (agent) hosts every agent, every channel plugin, every extension, and the persistence layer. Coordination between components happens over NATS (with a local tokio-mpsc fallback when NATS is offline).

Why single-process: shared in-memory caches, zero IPC overhead between agent and tool invocations, simpler ops. The broker and disk queue give us the durability a multi-process layout would provide, without the coordination cost.

High-level layout

flowchart TB
    subgraph PROC[agent process]
        direction TB

        subgraph PLUGINS[Channel plugins]
            WA[WhatsApp]
            TG[Telegram]
            MAIL[Email / Gmail poller]
            BR[Browser CDP]
            GOOG[Google APIs]
        end

        subgraph BUS[Event bus]
            NATS[(NATS)]
            LOCAL[(Local mpsc fallback)]
            DQ[(Disk queue + DLQ)]
        end

        subgraph AGENTS[Agent runtimes]
            A1[Agent: ana]
            A2[Agent: kate]
            A3[Agent: ops]
        end

        subgraph STORE[Persistence]
            STM[(Short-term sessions<br/>in-memory)]
            LTM[(Long-term memory<br/>SQLite + sqlite-vec)]
            WS[(Workspace-git<br/>per agent)]
        end

        subgraph TOOLS[Tools & integrations]
            EXT[Extensions<br/>stdio / NATS]
            MCP[MCP client / server]
            LLM[LLM providers]
        end

        PLUGINS --> BUS
        BUS --> AGENTS
        AGENTS --> BUS
        AGENTS --> STORE
        AGENTS --> TOOLS
        TOOLS --> LLM
    end

    USERS[End users] <--> PLUGINS

Workspace crates

The Cargo.toml workspace defines these member crates:

Binaries

Defined in Cargo.toml:

Runtime topology

agent runs a single tokio multi-thread runtime. Work is split into independent tasks:

flowchart LR
    MAIN[main tokio runtime]
    MAIN --> PA[Per-agent runtime task]
    MAIN --> PI[Plugin intake loops]
    MAIN --> HB[Heartbeat scheduler]
    MAIN --> MCP[MCP runtime manager]
    MAIN --> EXT[Extension stdio runtimes]
    MAIN --> MET[Metrics server :9090]
    MAIN --> HEALTH[Health server :8080]
    MAIN --> ADMIN[Admin console :9091]
    MAIN --> LOCK[Single-instance lock watcher]

Each agent runtime owns its own subscription to inbound topics, its own session manager view, its own LLM-loop state. Agents do not share mutable in-memory state — coordination between agents happens over the event bus (agent.route.<target_id>).

What lives where — quick mental model

• A message arrives → lands on plugin.inbound.<channel> (NATS)
• Agent runtime consumes it → SessionManager attaches or creates a session, HookRegistry fires before_message
• LLM loop runs → tools invoked through the registry, which calls into extensions / MCP / built-ins, each wrapped by CircuitBreaker
• Tool result flows back → after_tool_call hooks fire, LLM decides next turn
• Agent emits reply → publishes to plugin.outbound.<channel>
• Channel plugin delivers → physical message goes to the user

Details per subsystem:

• Agent runtime
• Event bus (NATS)
• Fault tolerance

Agent runtime

The agent runtime is the per-agent machinery that consumes inbound events, drives the LLM loop, invokes tools, and emits outbound events. One AgentRuntime is instantiated per configured agent at boot; each runs as its own async task.

Source: crates/core/src/agent/ (behavior.rs, agent.rs, runtime.rs, hook_registry.rs), boot in src/main.rs.

AgentBehavior trait

Every agent implements AgentBehavior (crates/core/src/agent/behavior.rs). The trait is intentionally small — default no-ops let built-in types (like LlmAgentBehavior) override only what they need.

The shipped LlmAgentBehavior implements the full chat-completion loop with tool calls, streaming, rate-limited retry, and hook fan-out.

Boot sequence

sequenceDiagram
    participant Main as src/main.rs
    participant Cfg as AppConfig
    participant Disc as Extension discovery
    participant SM as SessionManager
    participant TR as ToolRegistry
    participant LLM as LLM client
    participant AR as AgentRuntime
    participant Bus as Broker

    Main->>Cfg: load(config_dir)
    Main->>Disc: run_extension_discovery()
    Main->>SM: with_cap(ttl, max_sessions)
    Main->>TR: register built-ins + extensions + MCP
    Main->>LLM: build per provider (w/ CircuitBreaker)
    loop per agent in config
        Main->>AR: new(agent_id, behavior, tools, sm, llm, broker)
        AR->>Bus: subscribe plugin.inbound.<channel>+
        AR->>Bus: subscribe agent.route.<agent_id>
        AR-->>Main: ready
    end
    Main->>Main: install signal handlers
    Main->>Main: serve forever

Request/response lifecycle

A single inbound message drives the following flow inside one agent runtime:

sequenceDiagram
    participant Bus as NATS
    participant AR as AgentRuntime
    participant SM as SessionManager
    participant HR as HookRegistry
    participant LLM as LLM
    participant TR as ToolRegistry
    participant Ext as Extension / MCP / built-in

    Bus->>AR: plugin.inbound.<ch>
    AR->>SM: get_or_create(session_key)
    AR->>HR: fire("before_message")
    loop LLM turn loop
        AR->>LLM: completion(messages, tools)
        LLM-->>AR: assistant turn (text or tool_calls)
        alt tool_calls present
            AR->>HR: fire("before_tool_call", name, args)
            AR->>TR: invoke(tool_name, args)
            TR->>Ext: call
            Ext-->>TR: result
            TR-->>AR: result
            AR->>HR: fire("after_tool_call", name, result)
        else text only
            AR->>Bus: publish plugin.outbound.<ch>
        end
    end
    AR->>HR: fire("after_message")

SessionManager

Defined in crates/core/src/session/manager.rs. Tracks per-user conversational state in memory.

• Key: SessionKey derived from (agent_id, channel, sender_id); group chats get one session per group
• Storage: DashMap<SessionKey, Session> — lock-free concurrent map
• TTL: configured via memory.short_term.session_ttl (default 30 min); each access updates last_access
• Cap: soft limit DEFAULT_MAX_SESSIONS = 10,000; on overflow the oldest-idle session is evicted before insert
• Sweeper: background task scans every 1 s, removes expired entries
• Callbacks: on_expire() fires via tokio::spawn when a session is dropped — used by the MCP runtime to tear down per-session children

stateDiagram-v2
    [*] --> Active: first message
    Active --> Active: on_message / on_event<br/>(last_access updated)
    Active --> Expired: idle > TTL
    Active --> Evicted: cap exceeded,<br/>oldest-idle chosen
    Expired --> [*]: sweeper removes
    Evicted --> [*]: on_expire() fires

HookRegistry

Defined in crates/core/src/agent/hook_registry.rs. Lets extensions inject behavior at well-known points in the lifecycle without patching the runtime.

• Hook names: arbitrary strings. In practice the runtime fires: before_message, after_message, before_tool_call, after_tool_call, on_session_start, on_session_end
• Fan-out: sequential by priority (lower first), insertion order breaks ties
• Cap: 128 handlers per hook name — defensive guard against a buggy extension re-registering on every reload
• Errors: logged, treated as Continue — one misbehaving hook does not cascade into the rest
• Override: a hook may return Override(new_args) to mutate what the next hook (or the runtime itself) sees

Heartbeat

# per-agent config
heartbeat:
  enabled: true
  interval: 30s

• Scheduled per agent if heartbeat.enabled: true
• Interval parsed via humantime — any humantime duration works
• Each tick:
  1. Fires AgentBehavior::on_heartbeat(ctx)
  2. Publishes agent.events.<agent_id>.heartbeat
• Typical uses: proactive messages ("good morning"), reminders, external state syncs (pull Gmail, scan calendar), liveness pings

Graceful shutdown

src/main.rs installs SIGTERM / Ctrl+C handlers. On signal, the process tears down in a specific order so in-flight work finishes cleanly:

flowchart TD
    SIG[SIGTERM / Ctrl+C] --> C1[Cancel dream-sweep loops<br/>5 s grace]
    C1 --> C2[Mark /ready = false<br/>stop new traffic]
    C2 --> C3[Stop plugin intake<br/>no new inbound]
    C3 --> C4[Shutdown MCP runtime manager<br/>5 s clean close]
    C4 --> C5[Shutdown extensions<br/>5 s grace then kill_on_drop]
    C5 --> C6[Stop agent runtimes<br/>drain buffered messages]
    C6 --> C7[Abort metrics + health tasks]
    C7 --> EXIT([exit 0])

This order is enforced in src/main.rs around lines 1389–1458. Extensions get the longest grace period because stdio children can be mid-tool-call; the disk queue absorbs any events that the plugins couldn't finish publishing.

Why this shape

• One tokio runtime, many tasks: lets you run 10 agents on one CPU core when idle, saturates cores under load. No thread-per-agent bloat.
• No shared mutable state across agents: each agent holds its own registry views, its own session map. Cross-agent communication goes over the bus → visible, replayable, testable.
• Hooks instead of inheritance: extensions customize behavior without recompiling the core. Every insertion point is named, sequenced, and capped.

Event bus (NATS)

Every piece of communication between plugins, agents, and the broker layer itself flows over NATS (async-nats = 0.35). When NATS is offline, a local tokio::mpsc bus takes over and a SQLite-backed disk queue holds events until reconnection. No events are lost.

Source: crates/broker/ (nats.rs, local.rs, disk_queue.rs, topic.rs).

Why NATS

• Subject-based routing fits the "N plugins × M agents" fan-out naturally (plugin.inbound.* wildcards)
• Low-latency pub/sub with no broker-side state to manage
• Cluster-ready without rewriting the data plane
• Async-nats is mature, has JetStream if we ever need it

The design doc discusses the alternatives (RabbitMQ, Redis streams) that were rejected; see proyecto/design-agent-framework.md.

Subject namespace

Multi-instance plugins append an .<instance> suffix so two WhatsApp accounts (e.g. Ana's line and Kate's line) can run side by side without subject collisions.

Agent-to-agent routing

sequenceDiagram
    participant Ana
    participant Bus as NATS
    participant Ops

    Ana->>Ana: LLM decides to delegate
    Ana->>Bus: publish agent.route.ops<br/>(correlation_id=X)
    Bus->>Ops: deliver
    Ops->>Ops: on_message handler runs
    Ops->>Bus: publish agent.route.ana<br/>(correlation_id=X)
    Bus->>Ana: deliver
    Ana->>Ana: correlate reply by ID

The sender always includes a correlation_id in the event envelope; the receiver echoes it on the reply. That's how one agent can fan out to several agents and reassemble results.

Broker abstraction

crates/broker exposes a Broker trait implemented by two backends:

• NatsBroker — real NATS connection wrapped in a CircuitBreaker
• LocalBroker — in-process tokio::mpsc for tests and offline mode

Switching between them is driven by config. The local broker matches NATS subject semantics (including . segments and > wildcards), which keeps the test surface identical to production.

Disk queue

When a publish to NATS fails — circuit breaker open, connection lost, transient 5xx — the event is persisted to the disk queue instead of being dropped.

flowchart LR
    PUB[publish] --> OK{NATS up?}
    OK -->|yes| NATS[(NATS)]
    OK -->|no| ENQ[disk_queue.enqueue]
    ENQ --> SQLITE[(pending_events)]
    RECON[NATS reconnect] --> DRAIN[disk_queue.drain]
    SQLITE --> DRAIN
    DRAIN --> NATS
    DRAIN -.->|3 attempts failed| DLQ[(dead_letters)]
    DRAIN -.->|deserialization error| DLQ

Drain on reconnect

When NatsBroker detects reconnection, it calls disk_queue.drain():

1. Read up to 100 oldest events from pending_events
2. Republish each to NATS
3. On success: delete row
4. On failure: increment attempts, leave row in place
5. Once attempts >= 3: move to dead_letters

Dead-letter queue (DLQ)

Events that exhaust retries, or fail to deserialize at all, land in dead_letters. They're not silently discarded — CLI lets you inspect and replay them.

agent dlq list              # show all dead events
agent dlq replay <event_id> # move one back to pending_events
agent dlq purge             # drop the table (destructive!)

Replay moves the entry back to pending_events; the next drain cycle retries it with attempts reset.

Backpressure

Two independent mechanisms:

• Local broker channels are 256-capacity tokio::mpsc per subscriber. If a subscriber is slow, dropped events log a slow consumer warning but the subscription stays alive.
• Disk queue applies proportional sleep at >50% capacity (scaled from 0 ms up to MAX_BACKPRESSURE_MS = 500 ms). At the hard cap it additionally drops the oldest event and sleeps 500 ms — an intentional "shed load, don't block the producer forever" stance.

The disk queue's backpressure only matters when NATS is down for a long time and the producer is faster than real time. In normal operation the disk queue stays near-empty.

Local fallback

When NATS is unreachable or the circuit breaker on the publish path is Open, the runtime degrades gracefully:

• Inbound events from local plugins (e.g. a Telegram webhook fielded in-process) go through LocalBroker and reach agents immediately
• Outbound events that target a plugin hosted in the same process (which is every shipped plugin) also go through LocalBroker
• Anything that would have crossed a real NATS hop sits in the disk queue until reconnection

In practice, single-machine deployments keep working even with no NATS at all — the disk queue and the local broker together are sufficient for one process. NATS starts earning its keep the moment you scale to multiple processes, machines, or regions.

Fault tolerance

Every external call goes through a CircuitBreaker. Every retryable error has a bounded retry policy with jittered exponential backoff. Every event survives a NATS outage. A second process cannot race the first onto the same bus.

This page collects all of those guardrails in one place.

CircuitBreaker

Source: crates/resilience/src/lib.rs.

A three-state machine wrapped around any fallible external call. Once a dependency is failing, the breaker fails fast instead of piling up calls against a dead endpoint; periodic probes let it recover without human intervention.

stateDiagram-v2
    [*] --> Closed
    Closed --> Open: 5 consecutive failures
    Open --> HalfOpen: backoff elapsed
    HalfOpen --> Closed: 2 consecutive successes
    HalfOpen --> Open: any failure<br/>(backoff × 2, capped)

Defaults

Where it wraps

• LLM calls — one circuit per provider (MiniMax, Anthropic, OpenAI-compat, Gemini). A provider outage doesn't cascade to others.
• NATS publish — one circuit over the broker. When it opens the disk queue absorbs writes.
• CDP commands — one circuit per browser session. A dead Chrome doesn't freeze the agent loop.
• Extension stdio — implicit via the StdioRuntime lifecycle (crashed child → respawn, bounded).

Signals

CircuitBreaker exposes the usual methods (allow(), on_success(), on_failure()) plus two explicit overrides:

• trip() — force Open from outside (e.g. a health check decided the dep is down before a call fails)
• reset() — force Closed (e.g. the operator just restored the dep and doesn't want to wait for the probe window)

Retry policies

Retries live at a layer above the circuit breaker — they handle transient failures (429, 5xx, network blips) that don't warrant flipping the breaker. Every retry policy uses jittered exponential backoff to avoid thundering-herd reconnection storms.

These live in crates/llm/src/retry.rs (LLM) and crates/broker/src/disk_queue.rs (NATS drain).

Error classification

Retries only trigger on retryable errors. A 4xx other than 429 — missing key, invalid model, malformed request — fails fast. The rationale: retrying a misconfigured call wastes budget and still fails. Fail loudly, fix the config.

No message drop

The broker layer guarantees at-least-once delivery for publishes that reach the runtime:

flowchart LR
    P[publisher] --> TRY{NATS healthy?}
    TRY -->|yes| NATS[(NATS)]
    TRY -->|no| DQ[(disk queue)]
    DQ --> WAIT{reconnect?}
    WAIT -->|yes| DRAIN[drain FIFO]
    DRAIN --> NATS
    DQ -->|3 failed attempts| DLQ[(dead letters)]
    DLQ --> CLI[agent dlq replay]

In the absolute worst case — NATS down forever, disk full — the disk queue starts shedding oldest events at its hard cap, but the producer never crashes and never silently drops.

Single-instance lockfile

A second agent process pointed at the same data directory would double-subscribe every topic, delivering every message twice. To prevent that, boot acquires a lockfile and kicks out any stale or racing instance.

Source: src/main.rs::acquire_single_instance_lock.

flowchart TD
    START[agent boot] --> READ[read data/agent.lock]
    READ --> EXIST{file exists?}
    EXIST -->|no| WRITE[write our PID]
    EXIST -->|yes| PID[parse PID]
    PID --> ALIVE{/proc/PID/ exists?}
    ALIVE -->|no| WRITE
    ALIVE -->|yes| SIGTERM[send SIGTERM]
    SIGTERM --> WAIT[wait up to 5 s<br/>50 × 100 ms polls]
    WAIT --> DEAD{process gone?}
    DEAD -->|yes| WRITE
    DEAD -->|no| SIGKILL[send SIGKILL]
    SIGKILL --> WRITE
    WRITE --> LOCK[RAII handle alive]

The SingleInstanceLock RAII struct stores our own PID. On drop it only removes the lockfile if the stored PID still matches the current one — so a takeover by a third process doesn't let the original owner wipe the lock on its way out.

Graceful shutdown

See Agent runtime — Graceful shutdown for the ordered teardown sequence. Key points from a fault-tolerance angle:

• Dream-sweep loops and MCP sessions get explicit grace windows so in-flight work doesn't produce partial state
• Plugin intake is stopped before agent runtimes — the runtimes drain anything already in their mailboxes before exiting
• If the disk queue has unflushed events on SIGTERM, they survive to the next boot

Operator guardrails

Beyond the automatic mechanisms:

• Skill gating — an extension declaring requires.env = ["FOO"] is skipped at discovery when FOO is unset, instead of being registered and failing on every invocation. See Extensions — manifest.
• Inbound filter — events with neither text nor media (receipts, typing indicators, reactions-only) are dropped before they reach the LLM, saving cost and avoiding noisy turns.
• Health endpoints — :8080/ready and :8080/live expose lifecycle state for k8s liveness / readiness probes.
• Metrics — :9090/metrics (Prometheus) exposes everything from inbound event counts to circuit breaker state; see Metrics.

Transcripts (FTS + redaction)

Per-session JSONL transcripts under agents.<id>.transcripts_dir are the canonical record of every turn. Two optional layers wrap that record:

• FTS5 index — a SQLite virtual table that mirrors transcript content for MATCH queries. Backs the session_logs tool's search action when present.
• Redaction — a regex pre-processor that rewrites entry content before it ever reaches disk. Patterns target common credentials and home-directory paths.

Source: crates/core/src/agent/transcripts_index.rs, crates/core/src/agent/redaction.rs, crates/core/src/agent/transcripts.rs.

Configuration

config/transcripts.yaml (optional; absent → defaults below):

fts:
  enabled: true                       # default
  db_path: ./data/transcripts.db      # default

redaction:
  enabled: false                      # default — opt in
  use_builtins: true                  # only relevant if enabled
  extra_patterns:
    - { regex: "TENANT-[0-9]+", label: "tenant_id" }

JSONL is the source of truth. The FTS index is derivable; if the DB is corrupted or deleted, agent transcripts reindex (planned) can rebuild it from disk.

FTS schema

CREATE VIRTUAL TABLE transcripts_fts USING fts5(
    content,
    agent_id        UNINDEXED,
    session_id      UNINDEXED,
    timestamp_unix  UNINDEXED,
    role            UNINDEXED,
    source_plugin   UNINDEXED,
    tokenize = 'unicode61 remove_diacritics 2'
);

The DB is shared across agents; isolation is enforced at query time by WHERE agent_id = ?. User queries are escaped as a single FTS5 phrase so operators (OR, NOT, :) in the user input never reach the engine as syntax.

session_logs integration

When the index is available, the search action returns:

{
  "ok": true,
  "query": "reembolso",
  "backend": "fts5",
  "count": 3,
  "hits": [
    {
      "session_id": "…",
      "timestamp": "2026-04-25T18:00:00Z",
      "role": "user",
      "source_plugin": "wa",
      "preview": "...quería un [reembolso] del pedido..."
    }
  ]
}

If the index is None (FTS disabled or init failed), the action falls back to the legacy substring scan over JSONL. The shape is the same minus backend: "fts5".

Redaction patterns

Each match is replaced with [REDACTED:<label>]. Patterns run in the order above, so more specific shapes (Bearer JWT, Anthropic) win over generic catch-alls below.

A 40-char base64 pattern targeting AWS secret keys was deliberately omitted — it produces too many false positives on legitimate hashes and opaque ids. Operators who need it can add it scoped via extra_patterns.

Custom patterns

redaction:
  enabled: true
  extra_patterns:
    - { regex: "TENANT-[0-9]+",   label: "tenant_id" }
    - { regex: "internal\\.acme", label: "internal_host" }

Custom patterns run after built-ins. Invalid regex aborts boot with a message naming the offending index and label.

What redaction does not do

• It does not maintain a reverse map. Once content is redacted on disk the original is gone — by design. A reversible mapping would recreate the leak surface this feature is meant to close.
• It does not rewrite previously-written JSONL files. New entries redact going forward; historical content stays as-is.
• It does not redact tracing logs — that's a separate concern.
• The FTS index stores the redacted text, so search results never surface the original secrets either.

Operational notes

• The FTS index uses WAL journaling and capped pool size of 4 — it shares the same idiom as the long-term memory DB.
• Insert is best-effort. If an FTS write fails (disk full, lock contention) the tool logs at warn and the JSONL append still succeeds. The source of truth is never compromised.
• Boot logs include transcripts FTS index ready (or the warn that it fell back) and transcripts redaction active when the redactor has any rule loaded.

nexo-rs vs OpenClaw

OpenClaw is the closest reference point in the multi-channel-agent-gateway space. nexo-rs mined OpenClaw's plugin SDK, channel boundaries, and skills layout for ideas, then rebuilt the runtime in Rust with stricter operational guarantees. This page lays out the differences honestly — including where OpenClaw still has the edge.

Substrate

The single-binary shape is the reason nexo-rs runs comfortably on a phone (Termux) and on a fresh VPS without a Node ecosystem underneath. cargo build --release and ship target/release/agent — that is the whole deliverable.

Process & messaging

Hot reload

Per-agent capability sandbox

OpenClaw's plugin allowlist is global to the gateway. nexo-rs pushes the allowlist down to the agent and the binding (the inbound channel surface):

agents:
  - id: kate
    plugins: [whatsapp, telegram, browser, taskflow]
    allowed_tools: ["whatsapp_*", "browser_navigate", "memory_*"]
    outbound_allowlist:
      whatsapp: ["+57…"]
      telegram: [123456789]
    skill_overrides:
      ffmpeg-tools: warn
    accept_delegates_from: ["ana"]
    inbound_bindings:
      - plugin: whatsapp
        instance: kate_wa
        # per-binding overrides for the same agent
        allowed_tools: ["whatsapp_*"]
        outbound_allowlist:
          whatsapp: ["+57…"]

What that buys:

• An LLM running under kate cannot send messages to a number not in outbound_allowlist, even if a prompt injection asks it to.
• Two channels exposed to the same agent (sales WA, private TG) carry different capability surfaces — the sales binding doesn't get the private one's tool set.
• Skill modes (strict / warn / disable) are decided per agent, with explicit requires.bin_versions semver constraints (probed at boot, process-cached).

Secrets

Transcripts

OpenClaw stores transcripts as JSONL and greps them. nexo-rs keeps the JSONL (source of truth) and adds:

• SQLite FTS5 index (data/transcripts.db) — write-through from TranscriptWriter::append_entry. The session_logs search agent tool uses MATCH queries with phrase-escaped user input so operator strings can't inject FTS operators.
• Pre-persistence redactor (opt-in) — regex pass over content before write. 6 built-in patterns (Bearer JWT, sk-…, sk-ant-…, AWS access keys, 64+ hex tokens, home paths) plus operator-defined extra_patterns. JSONL and FTS receive the same redacted text.
• Atomic header writes — OpenOptions::create_new(true) so 16 concurrent first-appends to the same session result in exactly one header line.

Durable workflows

OpenClaw doesn't ship a durable-flow primitive. nexo-rs has TaskFlow:

• taskflow LLM tool with actions start | status | advance | wait | finish | fail | cancel | list_mine.
• Three wait conditions: Timer { at }, ExternalEvent { topic, correlation_id }, Manual.
• Single global WaitEngine ticks every 5 s (configurable), resumes flows whose deadlines have passed.
• taskflow.resume NATS subject lets external services wake external_event flows: publish {flow_id, topic, correlation_id, payload} and the bridge calls try_resume_external.
• agent flow list/show/cancel/resume from the CLI.
• Guardrails: timer_max_horizon (default 30 days) blocks unbounded waits; non-empty topic + correlation_id required for external_event.

LLM auth

MCP

OpenClaw supports MCP as a client. nexo-rs is both:

• Client — stdio and HTTP transports, full tool / resource / prompt catalog, tools/list_changed hot-reload.
• Server — agent mcp-server exposes the agent's own tools (filtered by allowlist) over stdio for Claude Desktop / Cursor / any MCP-aware host. Proxy tools (ext_*, mcp_*) are unconditionally hidden so the agent doesn't become an open relay.

Build size

target/release/agent             34 MB
target/release/agent (stripped)  29 MB
target/release/agent (.gz -9)    13 MB

For comparison, an OpenClaw install (Node + node_modules after pnpm install) sits in the hundreds of megabytes — most of it needed at runtime, not just build-time.

Where OpenClaw is still ahead

Honest list:

• Installer & onboarding flow — OpenClaw's openclaw doctor family and the bundled installer give a smoother first-run UX than nexo-rs's agent setup wizard, especially for non-Rust developers.
• TS familiarity — the JS / TS audience for plugin authors is larger than the Rust audience; if your team writes mostly TypeScript, contributing back to OpenClaw is faster.
• Track record — OpenClaw has a longer release history, more maintainers, and more shipped extensions in the wild.
• Apps surface — OpenClaw ships iOS / Android / macOS companion apps; nexo-rs only ships the daemon and the loopback web admin (admin-ui Phase A0–A11 still in progress).

Summary

If you want operational guarantees (single binary, fault-tolerant broker, per-agent sandbox, durable workflows, secrets audit) and you're OK with Rust, nexo-rs.

If you want fast onboarding, a TS plugin ecosystem, and the OpenClaw apps, OpenClaw.

The two projects share enough vocabulary that moving an extension between them is mostly a port, not a rewrite. The plugin SDK shape (stdio-spoken JSON-RPC + a plugin.toml manifest) is deliberately compatible.

Driver subsystem (Phase 67)

The driver subsystem turns the nexo-rs agent runtime into the "human in the loop" for another agent — typically the Claude Code CLI. It runs a goal-bound experiment: spawn the external CLI, watch its tool-use stream, decide allow/deny on every action, feed back acceptance failures, and stop only when the CLI claims "done" AND objective verification passes.

This page describes the architectural shape; concrete impl details live with each sub-phase.

Why

Claude Code (or any other local CLI agent) is excellent at writing code, but it sometimes:

• over-claims completion — says "done" when tests are red;
• proposes destructive shell commands when stuck;
• forgets which approaches it already tried and failed.

A second agent — driven by nexo-rs, backed by a different LLM (MiniMax M2.5), with persistent memory — closes those gaps.

Architecture

nexo-rs daemon
│
├─ "claude-driver" agent
│   ├─ LLM: MiniMax M2.5
│   ├─ memory: short_term + long_term + vector + transcripts
│   └─ skills: claude_cli, git_checkpoint, test_runner,
│              acceptance_eval, escalate
│
└─ MCP server (in-process)
    └─ tool: permission_prompt(tool_name, input) → {allow|deny, message}

claude  (subprocess, one per turn)
└─ claude --resume <id>
          --output-format stream-json
          --permission-prompt-tool mcp__nexo-driver__permission_prompt
          --add-dir <worktree>
          --allowedTools "Read,Grep,Glob,LS,WebFetch"
          -p "<turn prompt>"

Termination model

Claude says "done" — driver does NOT trust it. Driver runs the goal's acceptance criteria (cargo build, cargo test, cargo clippy, PHASES marker, custom verifiers). Only when all pass is the goal declared Done. Otherwise the failures are folded into the next turn's prompt: "you said done, but here's what still fails — fix it".

The driver also stops on budget exhaustion: max turns, wall-time, tokens, or consecutive denies. On exhaustion the driver escalates to the operator (WhatsApp / Telegram via existing channel plugins) with a state dump.

Foundational types — nexo-driver-types

The contract — AgentHarness trait + Goal / Attempt / Decision / AcceptanceCriterion / BudgetGuards types — lives in the leaf crate nexo-driver-types. Every value is serde-serializable so the contract can travel through NATS, get re-imported by extensions, and power admin-ui dashboards without dragging in the daemon.

How a turn flows (Phase 67.1)

#![allow(unused)]
fn main() {
use std::time::Duration;
use nexo_driver_claude::{ClaudeCommand, spawn_turn};
use nexo_driver_types::CancellationToken;

async fn doc(session_id: String) -> anyhow::Result<()> {
let cmd = ClaudeCommand::discover("Implementa Phase 26.z")?
    .resume(session_id)
    .allowed_tools(["Read", "Grep", "Glob", "LS"])
    .permission_prompt_tool("mcp__nexo-driver__permission_prompt")
    .cwd("/tmp/claude-runs/26-z");

let cancel = CancellationToken::new();
let mut turn = spawn_turn(cmd, &cancel, Duration::from_secs(600), Duration::from_secs(1)).await?;

while let Some(ev) = turn.next_event().await? {
    // dispatch on ev (Assistant tool_use → permission_prompt; Result → done check)
    let _ = ev;
}
let _exit = turn.shutdown().await?;
Ok(())
}
}

next_event cooperatively races three signals via tokio::select!: the cancel token, the per-turn deadline, and the JSONL stream. Errors land as Cancelled, Timeout, ParseLine, etc. Cleanup is always shutdown() — ChildHandle::Drop is the panic safety net.

Persistence (Phase 67.2)

SqliteBindingStore keeps (goal_id → claude session_id) plus timestamps in a single claude_session_bindings table. Two filters are applied on get:

• idle TTL — last_active_at must be within idle_ttl of now;
• max age — created_at + max_age must be in the future.

Either filter can be None (no filter) or Duration::ZERO (alias).

Three soft-delete-friendly operations live alongside clear:

• mark_invalid(goal_id) flips last_session_invalid = 1 instead of deleting the row. Phase 67.8 (replay-policy) calls this when Claude rejects a session id mid-turn; the row stays for forensics.
• touch(goal_id) bumps last_active_at only. Driver loop calls it per observed event so the idle filter doesn't need a structural upsert per turn.
• purge_older_than(cutoff) reaps rows the operator no longer cares about. Phase 67.6 (worktree janitor) calls it nightly.

Schema migrations: PRAGMA user_version = 1 is the sentinel; every open() runs CREATE TABLE/INDEX IF NOT EXISTS. Future v2 will extend that helper.

Permission flow (Phase 67.3)

Every Claude tool call that isn't on the static allowlist (Read,Grep,Glob,LS,WebFetch) goes through the MCP server before execution:

Claude Code ─── tools/call mcp__nexo-driver__permission_prompt ───▶
                                                                    │
                                                          stdio JSON-RPC
                                                                    │
                                                                    ▼
                                              nexo-driver-permission-mcp (child)
                                                                    │
                                                            calls PermissionDecider
                                                                    │
                                                                    ▼
                                                     {behavior: allow|deny, ...}

PermissionMcpServer exposes one tool, permission_prompt. The in-process AllowSession cache keyed on (tool_name, hash(input)) short-circuits repeat calls (a Claude turn that re-reads the same file pays the decider once).

Outcomes Claude receives are always one of two shapes:

{ "behavior": "allow" }                   // optional updatedInput
{ "behavior": "deny", "message": "..." }

Internally the driver tracks five outcomes — AllowOnce, AllowSession{scope}, Deny, Unavailable, Cancelled — collapsing the last three to deny on the wire. Unavailable (timeout) is fail-closed by design.

Phase 67.3 ships the bin in placeholder modes (--allow-all for dev, --deny-all <reason> for shadow). Phase 67.4 will swap those flags for --socket <path> so the bin asks the daemon's LlmDecider (MiniMax + memory) for each decision.

Goal lifecycle (Phase 67.4)

nexo-driver run goal.yaml
        │
        ▼
DriverOrchestrator::run_goal
        │
        ├─ workspace_manager.ensure(&goal)        ─┐
        │                                          │
        ├─ write_mcp_config(workspace,             ├─ side-effects in
        │     bin_path, socket_path)               │   <workspace>/
        │                                          │
        ├─ DriverSocketServer (already running) ──┘
        │     spawned by builder, owned via JoinHandle
        │
        └─ for each turn:
             ├─ budget.is_exhausted? → BudgetExhausted{axis}
             ├─ AttemptStarted event
             ├─ run_attempt(ctx, params)
             │     spawn `claude --resume <id> ... --mcp-config ...`
             │     event-loop on stream-json
             │     binding_store.upsert(session_id)
             │     acceptance.evaluate(criteria, workspace)
             │     return AttemptResult { outcome }
             ├─ AttemptCompleted event
             └─ match outcome:
                Done            → break, GoalCompleted{Done}
                NeedsRetry{f}   → next turn with prior_failures
                Continue{...}   → next turn (e.g. session-invalid retry)
                Cancelled       → break
                BudgetExhausted → break
                Escalate{r}     → emit Escalate event, break

AttemptOutcome::Continue covers two cases the loop treats the same: the stream ended without Result::Success (Claude crashed early), and a session not found reply that triggered binding_store.mark_invalid so the next turn starts fresh.

NATS subjects emitted (when feature = "nats" and emit_nats_events: true):

• agent.driver.goal.{started,completed}
• agent.driver.attempt.{started,completed}
• agent.driver.decision (Phase 67.7 will populate when LlmDecider records its rationale)
• agent.driver.acceptance
• agent.driver.budget.exhausted
• agent.driver.escalate
• agent.driver.replay (Phase 67.8 — replay-policy verdict)
• agent.driver.compact (Phase 67.9 — compact-policy scheduled a /compact <focus> turn)

Compact policy (Phase 67.9)

Long agentic runs let Claude's context grow without bound. The orchestrator runs a CompactPolicy after every successful work turn: when running tokens cross threshold * context_window, the next iteration is rewritten as a /compact <focus> slash command turn so Claude Code shrinks its own context before the next work turn. Compact turns absorb token usage but do not bump the goal's turn counter, so they don't burn the budget. min_turns_between_compacts prevents back-to-back compacts. Set context_window: 0 (or enabled: false) in compact_policy: to disable.

Sub-phases

See also

• crates/driver-types/README.md — contract surface and layering
• proyecto/PHASES.md — Phase 67 sub-phase status of record
• OpenClaw reference: research/src/agents/harness/types.ts
• OpenClaw subprocess pattern: research/extensions/codex/src/app-server/transport-stdio.ts

Project tracker + multi-agent dispatch (Phase 67.A–H)

The project-tracker subsystem lets a nexo-rs agent answer "qué fase va el desarrollo" through Telegram / WhatsApp / a shell, and lets it dispatch async programmer agents that ship phases on its behalf.

The implementation is layered:

Project tracker (Phase 67.A)

FsProjectTracker reads <root>/PHASES.md (required) and <root>/FOLLOWUPS.md (optional) at startup, caches parsed state behind a parking-lot RwLock with a 60 s TTL, and starts a notify watcher on the parent directory that invalidates the cache on Modify | Create | Remove events.

Read tools register through nexo_dispatch_tools::READ_TOOL_NAMES (project_status, project_phases_list, followup_detail, git_log_for_phase).

Set ${NEXO_PROJECT_ROOT} to point at a workspace other than the daemon's cwd.

Multi-agent registry (Phase 67.B)

AgentRegistry is the single source of truth for every goal the driver has admitted. Each entry holds an ArcSwap<AgentSnapshot> (turn N/M, last acceptance, last decision summary, diff_stat) so list_agents / agent_status readers never block writers.

• admit(handle, enqueue) enforces the global cap. Beyond the cap, enqueue=true parks the goal as Queued; enqueue=false rejects.
• release(goal_id, terminal) returns the next-up queued goal so the orchestrator can promote it via promote_queued once the worktree / binding is ready.
• apply_attempt(AttemptResult) refreshes the live snapshot. Idempotent against out-of-order replay (lower turn_index ignored).
• Reattach (Phase 67.B.4) walks the SQLite store at boot and rehydrates Running rows. With resume_running=false they flip to LostOnRestart and surface to the operator.

LogBuffer keeps a per-goal ring of recent driver events for the agent_logs_tail tool — bounded so a chatty goal cannot OOM the process.

Persistence wiring (Phase 71)

The bin reads agent_registry.store from config/project-tracker/project_tracker.yaml and opens SqliteAgentRegistryStore when the resolved path is non-empty. Env placeholders (${NEXO_AGENT_REGISTRY_DB:-./data/agents.db}) are expanded before the open. Path open failures fall back to MemoryAgentRegistryStore with a warn so a corrupt sqlite file never bricks boot.

When the registry is sqlite-backed and reattach_on_boot: true, the bin runs the reattach sweep with resume_running=false. Every prior-run Running row flips to LostOnRestart, and any notify_origin / notify_channel hook attached to that goal fires once with an [abandoned] summary so the originating chat learns the goal could not be resumed. Subprocess respawn is intentionally not attempted — restoring a Claude Code worktree the daemon no longer owns is unsafe to do silently and lives under Phase 67.C.1.

Shutdown drain (Phase 71.3)

On SIGTERM the bin runs nexo_dispatch_tools::drain_running_goals before plugin teardown so notify_origin reaches WhatsApp / Telegram while their adapters are still alive. Each Running goal's Cancelled hooks fire with a [shutdown] summary; per-hook dispatch is bounded by a 2 s timeout so a stuck publish cannot hold shutdown hostage. The row then flips to LostOnRestart so the next boot's reattach sweep does not re-fire the same notification.

[shutdown] daemon stopping — goal `<id>` was running and has
been marked abandoned. Re-dispatch with `program_phase
phase_id=<phase>` if you still need it.

SIGKILL still bypasses this — the boot-time reattach sweep is the safety net for that case.

Turn-level audit log (Phase 72)

Live state (AgentSnapshot) only carries the latest decision / diff / acceptance per goal. Once a turn rolls forward the previous turn's data is gone. To answer "what did the agent actually do across its 40 turns?" the runtime now writes a durable row per turn into a goal_turns table on the same agents.db:

goal_turns(
    goal_id      TEXT,
    turn_index   INTEGER,
    recorded_at  INTEGER,
    outcome      TEXT,        -- done | continue | needs_retry | …
    decision     TEXT,        -- last Decision rendered as
                              --   "<tool> (allow|deny:msg|observe:note) — rationale"
    summary      TEXT,        -- mirror of AgentSnapshot.last_progress_text
    diff_stat    TEXT,
    error        TEXT,        -- pre-rendered for needs_retry / escalate / budget
    raw_json     TEXT,        -- full AttemptResult payload
    PRIMARY KEY (goal_id, turn_index)
);

EventForwarder writes a row on every AttemptResult event, upsert-on-conflict so a replay can't dup history. The new chat tool agent_turns_tail goal_id=<uuid> [n=20] returns a markdown table of the last N rows (default 20, capped at 1000):

showing 20 of 40 turn(s) for `…`

| turn | outcome | decision | summary | error |
|---|---|---|---|---|
| 21 | continue | Edit (allow) — patch crate slack | wired Plugin trait | - |
| 22 | needs_retry | Bash (allow) — cargo build | … | E0432 in slack/src/lib.rs |
…

Best-effort writes: an append failure logs a warn but never blocks the driver loop. When the registry isn't sqlite-backed (memory fallback), the tool reports "set agent_registry.store in project_tracker.yaml" rather than silently returning empty.

Async dispatch (Phase 67.C + 67.E)

DriverOrchestrator::spawn_goal(self: Arc<Self>, goal) returns a tokio::task::JoinHandle so the calling tool returns the goal id instantly without waiting for the run to finish. Per-goal pause / cancel signals (watch<bool> and CancellationToken::child_token) let pause_agent / cancel_agent target one goal without taking down the rest of the orchestrator.

program_phase_dispatch is the heart of the dispatch surface: it reads the sub-phase out of PHASES.md, runs DispatchGate::check, constructs a Goal with the dispatcher / origin metadata, asks the registry for a slot, and either spawns the goal or returns Queued / Forbidden / NotFound. dispatch_followup is the mirror that pulls the description from a FOLLOWUPS.md item.

Capability gate (Phase 67.D)

DispatchPolicy { mode, max_concurrent_per_dispatcher, allowed_phase_ids, forbidden_phase_ids } lives on AgentConfig and (as Option<DispatchPolicy>) on InboundBinding. The per-binding override fully replaces the agent-level value so an operator can be precise per channel ("asistente is none everywhere except this Telegram chat where it is full").

DispatchGate::check short-circuits in this order:

1. capability None → CapabilityNone (every kind).
2. ReadOnly capability + write kind → CapabilityReadOnly.
3. write + require_trusted + !sender_trusted → SenderNotTrusted. Read tools bypass the trust gate so list_agents stays open for unpaired senders.
4. forbidden_phase_ids match → PhaseForbidden.
5. non-empty allowed_phase_ids + no match → PhaseNotAllowed.
6. dispatcher / sender / global caps. Global cap with queue_when_full=true is admitted; the orchestrator queues it. Without queue → GlobalCapReached.

ToolRegistry::apply_dispatch_capability(policy, is_admin) prunes the registry of dispatch tool names not allowed by the resolved policy. ToolRegistryCache::get_or_build_with_dispatch builds the per-binding filtered registry that respects both allowed_tools and dispatch_policy. Hot reload (Phase 18) constructs a fresh ToolRegistryCache per snapshot, so a new dispatch_policy lands on the next intake without restart; in-flight goals keep their pre-reload tool surface so a hot reload never preempts.

Completion hooks (Phase 67.F)

Each hook is (on: HookTrigger, action: HookAction, id). Triggers fire on Done | Failed | Cancelled | Progress { every_turns }. Actions:

• notify_origin — publish a markdown summary to the chat that triggered the goal. No-op when origin.plugin == "console".
• notify_channel { plugin, instance, recipient } — publish to an explicit channel different from the origin (escalate to ops).
• dispatch_phase { phase_id, only_if } — chain another goal when only_if matches the firing transition. Implemented via a pluggable DispatchPhaseChainer so the runtime owns program_phase_dispatch plumbing.
• nats_publish { subject } — JSON payload to a custom subject.
• shell { cmd, timeout } — opt-in via allow_shell_hooks. Capability PROGRAM_PHASE_ALLOW_SHELL_HOOKS registered with the setup inventory so agent doctor capabilities flags it the moment the operator exports the env var. Receives NEXO_HOOK_GOAL_ID / PHASE_ID / TRANSITION / PAYLOAD_JSON env vars.

HookIdempotencyStore (SQLite) keeps (goal_id, transition, action_kind, action_id) UNIQUE so at-least-once NATS replay or a mid-hook restart cannot fire a hook twice.

HookRegistry (in-memory DashMap<GoalId, Vec<CompletionHook>>) backs add_hook / remove_hook / agent_hooks_list.

NATS subjects (Phase 67.H.2)

Plus the existing Phase 67.0–67.9 subjects: agent.driver.{goal,attempt}.{started,completed}, agent.driver.{decision,acceptance,budget.exhausted,escalate,replay,compact}.

CLI (Phase 67.H.1)

nexo-driver-tools mirrors the chat tool surface for shell use:

nexo-driver-tools status [--phase <id> | --followups]
nexo-driver-tools dispatch <phase_id>
nexo-driver-tools agents list [--filter running|queued|...]
nexo-driver-tools agents show <goal_id>
nexo-driver-tools agents cancel <goal_id> [--reason "…"]

origin.plugin = "console" so notify_origin is a no-op (the operator sees stdout, not a chat reply).

Built-in registration (nexo daemon)

The default nexo agent binary registers every dispatch tool definition at boot via nexo_core::agent::dispatch_handlers::register_dispatch_tools_into. The LLM sees program_phase, list_agents, agent_status, etc. in its toolset; per-binding dispatch_capability (config/agents.yaml) prunes the write tools for bindings that opted out.

What's NOT bundled by default is the runtime DispatchToolContext — the orchestrator + registry + tracker references the handlers consult. Without it, a tool call returns a clean dispatch tools require AgentContext.dispatch to be set at boot error instead of pretending success. Two integration paths from there:

• In-process orchestrator — boot a DriverOrchestrator alongside the agents, share one AgentRegistry. See the next section for the wiring sample.
• NATS-based dispatch — agent bin publishes a message to agent.driver.dispatch.request that a separate nexo-driver daemon consumes. This is the topology to use when the Claude subprocess needs hardware (GPU box) the agent daemon doesn't have. The dispatch tool surface only changes in the registry it consults; operators can swap the in- process AgentRegistry for one that mirrors a NATS-backed registry without touching the handlers.

Boot wiring (B8)

The integrator's main.rs ties everything together. Minimal shape:

use std::sync::Arc;
use nexo_agent_registry::{AgentRegistry, MemoryAgentRegistryStore, LogBuffer};
use nexo_core::agent::{
    dispatch_handlers::{register_dispatch_tools_into, DispatchToolContext},
    tool_registry::ToolRegistry,
};
use nexo_dispatch_tools::{
    event_forwarder::EventForwarder,
    hooks::{DefaultHookDispatcher, HookRegistry, NoopNatsHookPublisher},
    policy_gate::CapSnapshot,
    NoopTelemetry,
};
use nexo_pairing::PairingAdapterRegistry;
use nexo_project_tracker::FsProjectTracker;

// 1. Project tracker.
let tracker: Arc<dyn nexo_project_tracker::ProjectTracker> =
    Arc::new(FsProjectTracker::open(std::env::current_dir().unwrap())?);

// 2. Agent registry + log buffer.
let registry = Arc::new(AgentRegistry::new(
    Arc::new(MemoryAgentRegistryStore::default()),
    4,
));
let log_buffer = Arc::new(LogBuffer::new(200));
let hook_registry = Arc::new(HookRegistry::new());

// 3. Hook dispatcher with the channel adapters that Phase 26
//    registered (whatsapp / telegram).
let pairing = PairingAdapterRegistry::new();
// pairing.register(WhatsappPairingAdapter::new(...));
// pairing.register(TelegramPairingAdapter::new(...));
let hook_dispatcher = Arc::new(DefaultHookDispatcher::new(
    pairing,
    Arc::new(NoopNatsHookPublisher),
));

// 4. Orchestrator with EventForwarder so registry / log_buffer /
//    hooks see every driver event.
let inner_sink: Arc<dyn nexo_driver_loop::DriverEventSink> =
    Arc::new(nexo_driver_loop::NoopEventSink);
let event_sink: Arc<dyn nexo_driver_loop::DriverEventSink> =
    Arc::new(EventForwarder::new(
        registry.clone(),
        log_buffer.clone(),
        hook_registry.clone(),
        hook_dispatcher.clone(),
        inner_sink,
    ));
// (orchestrator builder consumes event_sink)

// 5. Bundle for AgentContext.dispatch.
let dispatch_ctx = Arc::new(DispatchToolContext {
    tracker,
    orchestrator: orch.clone(),
    registry,
    hooks: hook_registry,
    log_buffer,
    default_caps: CapSnapshot {
        queue_when_full: true,
        ..Default::default()
    },
    require_trusted: true,
    telemetry: Arc::new(NoopTelemetry),
});

// 6. Register the handlers into the base ToolRegistry. The
//    per-binding cache prunes write tools when capability=None
//    or read_only.
let base = ToolRegistry::new();
register_dispatch_tools_into(&base);

// 7. Per-session AgentContext.with_dispatch(dispatch_ctx)
//    + .with_sender_trusted(true) + .with_inbound_origin(plugin,
//    instance, sender).

Without step 6 the handlers exist but aren't reachable by the LLM. Without step 4 the registry / log_buffer / hooks stay inert. Without step 5 the handlers return MissingDispatchCtx.

See also

• proyecto/PHASES.md — Phase 67.A–H sub-phase status of record.
• architecture/driver-subsystem.md — Phase 67.0–67.9 driver loop
  • replay + compact policies.

Configuration layout

nexo-rs loads configuration from a single directory (passed via --config <path>, default ./config). The runtime reads a small set of required YAML files and a handful of optional ones.

Source: crates/config/src/lib.rs::AppConfig::load.

Directory tree

config/
├── agents.yaml              # required — base agent catalog
├── agents.d/                # optional — drop-in agents, merged in alpha order
│   ├── ana.example.yaml     # template (committed)
│   └── *.yaml               # real definitions (gitignored)
├── broker.yaml              # required — NATS / local broker + disk queue
├── llm.yaml                 # required — LLM providers
├── memory.yaml              # required — short-term + long-term + vector
├── extensions.yaml          # optional — extension search paths, toggles
├── mcp.yaml                 # optional — MCP servers the agent consumes
├── mcp_server.yaml          # optional — expose this agent as an MCP server
├── tool_policy.yaml         # optional — per-tool / per-agent policy
├── runtime.yaml             # optional — hot-reload watcher settings
├── plugins/
│   ├── whatsapp.yaml
│   ├── telegram.yaml
│   ├── email.yaml
│   ├── browser.yaml
│   ├── google.yaml
│   └── gmail-poller.yaml
└── docker/                  # optional — overrides for containerized runs
    ├── agents.yaml
    ├── llm.yaml
    └── …

Required vs optional

The loader fails startup if any required file is missing or malformed. Optional files return None when absent and unlock related features only if present.

Drop-in agents

Files under config/agents.d/*.yaml are merged into the base agents.yaml in lexicographic filename order. Each file has the same top-level shape (agents: [...]); entries append to the base list.

Common patterns:

• 00-dev.yaml / 10-prod.yaml — control override order by numeric prefix
• Keep agents.yaml public-safe and drop sensitive business content (sales prompts, pricing, phone numbers) into gitignored config/agents.d/ana.yaml
• Ship config/agents.d/<name>.example.yaml as a template so the shape stays discoverable

Details in Drop-in agents.

Docker layout

config/docker/ mirrors the main layout and is consumed when the compose file mounts it at /app/config/docker:

# docker-compose.yml
command: ["agent", "--config", "/app/config/docker"]

Secrets inside Docker containers live at /run/secrets/<name> — the compose definitions use ${file:/run/secrets/...} references. See LLM config — auth for the full secret resolution rules.

Env vars and secrets in YAML

YAML values can reference env vars and files:

Path-traversal rules for ${file:...}:

• Relative paths are rooted at the current working directory
• .. segments are rejected outright
• Absolute paths must sit under one of these whitelisted roots:
  • /run/secrets/ (Docker secrets)
  • /var/run/secrets/ (Kubernetes projected volumes)
  • ./secrets/ (project-local)
  • the directory pointed at by $CONFIG_SECRETS_DIR (operator-defined)

Everything else is refused at parse time with an explicit error naming the invalid path and the allowed roots.

Validation

All config structs deserialize with #[serde(deny_unknown_fields)], so typos fail fast:

unknown field `modl`, expected `model`
at line 4, column 5 in config/agents.yaml

Missing required fields produce the same kind of message:

missing field `model`
at line 5, column 3 in config/agents.yaml

Env / file resolution errors identify the placeholder and the file:

env var MINIMAX_API_KEY not set (referenced in llm.yaml)

${file:../etc/passwd}: `..` not allowed in file reference (in broker.yaml)

Boot sequence

flowchart TD
    START([agent --config path]) --> LOAD[AppConfig::load]
    LOAD --> REQ{required files<br/>present & parseable?}
    REQ -->|no| FAIL([fail fast, exit 1])
    REQ -->|yes| OPT[read optional files]
    OPT --> DROP[merge config/agents.d/]
    DROP --> RESOLVE[resolve env / file placeholders]
    RESOLVE --> VAL[struct-level validation<br/>deny_unknown_fields]
    VAL --> SEM[semantic validation<br/>validate_agents, MCP headers]
    SEM --> READY([AppConfig ready])

Next

• agents.yaml — full agent schema
• llm.yaml — LLM provider schema + auth modes
• broker.yaml — NATS + disk queue
• memory.yaml — short/long/vector
• Drop-in agents — merge order and patterns

agents.yaml

The agent catalog. One entry per agent; each entry declares the model, channels, tools, sandboxing, and behavioral knobs for that agent.

Source: crates/config/src/types/agents.rs.

Top-level shape

agents:
  - id: ana
    model:
      provider: minimax
      model: MiniMax-M2.5
    plugins: [whatsapp]
    inbound_bindings:
      - plugin: whatsapp
    allowed_tools:
      - whatsapp_send_message
    outbound_allowlist:
      whatsapp:
        - "573000000000"
    system_prompt: |
      You are Ana, …

Full field reference

All fields use #[serde(deny_unknown_fields)] — typos fail fast.

Identity & model

Channels

Each inbound_bindings[] entry can override the agent-level defaults for that channel: allowed_tools, outbound_allowlist, skills, model, system_prompt_extra, sender_rate_limit, allowed_delegates. Useful for running the same agent on two channels with different rules. See Per-binding capability override below for the full override surface and merge rules.

Tool sandboxing

allowed_tools semantics:

• For legacy agents (no inbound_bindings) the allowlist is applied at registry-build time — tools not matching the patterns are removed from the registry before the LLM sees them.
• For agents with inbound_bindings the base registry keeps every tool and enforcement happens per-binding at turn time (see Per-binding capability override) so a binding's override can both narrow AND expand within the registry. Defense-in-depth: the LLM only receives tools allowed by the matched binding, and the tool-call execution path rejects any hallucinated name outside the same allowlist.

In both modes the LLM never receives disallowed tool definitions; the difference is where the filter is applied.

System prompt & workspace

Heartbeat

heartbeat:
  enabled: true
  interval: 30s

See Agent runtime — Heartbeat.

Runtime knobs

config:
  debounce_ms: 2000
  queue_cap: 32

Agent-to-agent delegation

Routing uses agent.route.<target_id> over NATS with a correlation_id. See Event bus — Agent-to-agent routing.

Dreaming (memory consolidation)

dreaming:
  enabled: false
  interval_secs: 86400
  min_score: 0.35
  min_recall_count: 3
  min_unique_queries: 2
  max_promotions_per_sweep: 20
  weights:
    frequency: 0.24
    relevance: 0.30
    recency: 0.15
    diversity: 0.15
    consolidation: 0.10

Defaults shown. See Soul — Dreaming.

Workspace-git

workspace_git:
  enabled: false
  author_name: "agent"
  author_email: "agent@localhost"

When enabled, the agent's workspace directory is a git repo that the runtime commits to after dream sweeps, forge_memory_checkpoint, and session close. Good for forensic replay.

Google auth (per-agent OAuth)

google_auth:
  client_id: ${GOOGLE_CLIENT_ID}
  client_secret: ${file:./secrets/google_secret.txt}
  scopes:
    - https://www.googleapis.com/auth/gmail.readonly
  token_file: ./data/workspace/ana/google_token.json
  redirect_port: 17653

Used by crates/plugins/google to run OAuth PKCE per agent.

Deprecated in Phase 17 — prefer declaring Google accounts in a dedicated config/plugins/google-auth.yaml and binding them from credentials.google (see next section). Inline google_auth still boots with a warn so existing deployments keep working; it is auto-migrated into the credential store at startup.

Credentials (per-agent WhatsApp / Telegram / Google)

Pins each agent to the plugin instance / Google account it may use for outbound traffic. The runtime resolves the target at publish time from the agent id — the LLM cannot pick the instance via tool args, closing the prompt-injection vector.

credentials:
  whatsapp: personal          # must match whatsapp.yaml instance label
  telegram: ana_bot           # must match telegram.yaml instance label
  google:   ana@gmail.com     # must match google-auth.yaml accounts[].id
  # Silence the "inbound ≠ outbound" warning when intentional:
  # telegram_asymmetric: true

Validated at boot by the gauntlet (agent --check-config runs the same checks without starting the daemon). Omitting credentials: keeps the legacy single-account behavior for back-compat.

Full schema + migration guide: config/credentials.md.

Relationship diagram

flowchart LR
    AG[agent entry] --> MOD[model provider]
    AG --> PL[plugins list]
    AG --> IB[inbound_bindings]
    AG --> AT[allowed_tools]
    AG --> OA[outbound_allowlist]
    AG --> WS[workspace]
    AG --> HB[heartbeat]
    AG --> DEL[delegation gates]
    IB -->|per-binding override| AT
    IB -->|per-binding override| OA
    MOD -->|resolved from| LLM[llm.yaml]
    PL -->|tools from| PLUG[plugins/*.yaml]
    WS -->|files| SOUL[SOUL.md /<br/>IDENTITY.md /<br/>MEMORY.md]

Per-binding capability override

A single agent can expose distinct capability surfaces per InboundBinding without running two agent processes. Typical use: the same Ana agent answers WhatsApp with a narrow sales-only surface and Telegram with the full catalogue.

Schema

Every inbound_bindings[] entry accepts the following optional overrides. Unset fields inherit the agent-level value.

Anything else (workspace, transcripts_dir, heartbeat, memory, workspace_git, google_auth) stays at the agent level — identity and persistent state do not change per channel.

Example

agents:
  - id: ana
    model: { provider: anthropic, model: claude-haiku-4-5 }
    plugins: [whatsapp, telegram]
    workspace: ./data/workspace/ana
    skills_dir: ./skills
    system_prompt: |
      You are Ana.
    allowed_tools: []            # agent-level = permissive; bindings narrow
    outbound_allowlist: {}
    inbound_bindings:
      - plugin: whatsapp
        allowed_tools: [whatsapp_send_message]
        outbound_allowlist:
          whatsapp: ["573115728852"]
        skills: []
        sender_rate_limit: { rps: 0.5, burst: 3 }
        system_prompt_extra: |
          Channel: WhatsApp sales. Follow the ETB/Claro lead flow.
      - plugin: telegram
        instance: ana_tg
        allowed_tools: ["*"]
        outbound_allowlist:
          telegram: [1194292426]
        skills: [browser, github, openstreetmap]
        model: { provider: anthropic, model: claude-sonnet-4-5 }
        allowed_delegates: ["*"]
        sender_rate_limit: disable
        system_prompt_extra: |
          Channel: private Telegram. Full tool access allowed.

Boot-time validation

The runtime rejects configs with:

• Duplicate (plugin, instance) tuples in the same agent.
• Telegram instance referenced by a binding but not declared in config/plugins/telegram.yaml.
• Binding model.provider different from the agent-level provider (the LLM client is wired once per agent).
• Skills listed in a binding whose directory does not exist under skills_dir.

A binding that sets no overrides is allowed but logs a warn.

Matching order

Bindings are evaluated top-to-bottom; the first match wins. If you have both {plugin: telegram, instance: None} (wildcard) and {plugin: telegram, instance: "admin"}, declare the specific entry first — otherwise the wildcard consumes every Telegram event.

Runtime isolation

• Tool list shown to the LLM is filtered through the binding's allowed_tools; tools hidden on WhatsApp remain invisible even if the LLM hallucinates the name.
• Tool-call execution re-checks the allowlist and returns not_allowed for anything outside — stops hallucination loops without executing the forbidden tool.
• Outbound tools (whatsapp_send_message, telegram_send_message) read outbound_allowlist from the matched binding, so WhatsApp sends on the sales channel cannot reach numbers that only the private channel allows.
• Sender rate limit buckets are keyed per binding; flood on one channel cannot drain the quota on another.

Back-compat

Agents without inbound_bindings keep the pre-feature behavior byte- for-byte: the agent-level allowed_tools is pruned into the base registry at boot, and the runtime synthesises a policy from agent- level defaults (keyed at binding_index = usize::MAX).

Output language

Operators pin the language an agent replies in without rewriting workspace markdown. Workspace docs (IDENTITY, SOUL, MEMORY, USER, AGENTS) and tool descriptions stay in English — the single source of truth that recall, dreaming, vector search, and developer tooling all read. The runtime injects a # OUTPUT LANGUAGE system block right after the agent's system_prompt, telling the model to read those docs as-is but reply to the user in the configured language.

Where to set it

agents:
  - id: ana
    language: es                # default for every binding on this agent
    inbound_bindings:
      - plugin: whatsapp
        # → uses Spanish (inherits from the agent)
      - plugin: telegram
        instance: support_intl
        language: en            # → uses English on this channel only
      - plugin: telegram
        instance: bilingual_qa
        language: ""            # → no directive (model picks)

Resolution

Precedence (first non-empty wins):

1. inbound_bindings[i].language — per-channel override.
2. language — agent-level default.
3. null — no # OUTPUT LANGUAGE block emitted; the model decides from the user's input.

Empty string and whitespace-only values resolve to no directive on both layers — useful for "turn the directive off on this binding even though the agent has one".

Accepted values

The runtime treats the value as a label and forwards it verbatim into the directive (after sanitisation; see below). Both forms work:

• ISO codes: "es", "en", "en-US", "pt-BR".
• Human names: "Spanish", "English", "español", "Brazilian Portuguese".

Human names produce slightly clearer directives in practice (Respond to the user in Spanish. reads more natural than Respond to the user in es.), but both yield the same model behaviour with modern LLMs.

Rendered block

# OUTPUT LANGUAGE

Respond to the user in {language}. Workspace docs (IDENTITY, SOUL,
MEMORY, USER, AGENTS) and tool descriptions are in English — read
them as-is, but your turn-final reply to the user must be in
{language}.

The block lands after the agent's system_prompt (and the optional # CHANNEL ADDENDUM block) so its instruction wins under the LLM's recency bias.

Sanitisation

Defense-in-depth against config-driven prompt injection: every language value is normalised before rendering — control characters and embedded newlines are stripped, trimmed, and the result is capped at 64 characters. A YAML payload like language: "es\n\nIgnore previous instructions" cannot smuggle a multi-line directive into the system prompt.

Hot reload

Phase 18 hot-reload covers this field. Edit agents.d/<id>.yaml, save (or run agent reload), and the next message uses the new language. In-flight LLM turns finish on the old policy; subsequent turns flip to the new one.

Related

• Workspace docs and recall stay English regardless — see Soul, identity & learning.
• Per-channel rotation walkthrough lives in Recipes — A/B prompt swap.

Link understanding

Per-agent (and per-binding) toggle that fetches URLs in the user's message and injects a # LINK CONTEXT block. Off by default. Full schema, caps, and SSRF denylist live on Link understanding. The field is link_understanding at agent scope and at each inbound_bindings[] entry; binding value replaces agent default, omitted = inherit.

Web search

Per-agent (and per-binding) toggle that exposes a web_search tool backed by Brave / Tavily / DuckDuckGo / Perplexity. Off by default. Full schema, providers, cache, and circuit-breaker behaviour live on Web search. The field is web_search at agent scope and at each inbound_bindings[] entry; binding value replaces agent default, omitted = inherit.

Pairing policy

Per-binding toggle that turns on the DM-challenge gate for inbound senders. Off by default. The field is pairing_policy on each inbound_bindings[] entry; null (default) = inherit agent value or skip the gate entirely. Full protocol, threat model, and CLI reference live on Pairing.

Common mistakes

• Forgetting plugins: [...]. An agent without plugins has no inbound channel and no outbound tools. It is inert.
• Setting allowed_tools without a wildcard. ["memory_*"] allows the full memory_* family; ["memory_store"] allows only one. Check the glob before assuming.
• Large system_prompt duplication across agents. Use inbound_bindings[].system_prompt_extra to add per-channel content without duplicating the whole prompt.
• Sharing a WhatsApp session across agents. Each agent's workspace should contain its own whatsapp/default session; the wizard does this automatically, but pointing two agents at the same session dir will cause message cross-delivery.
• Translating the workspace markdown to match language. Don't. Workspace docs are the single source of truth read by recall, dreaming, and developer tooling — keep them in English. The # OUTPUT LANGUAGE block tells the model to translate the reply on its way out.

Next

• Drop-in agents — merging multiple agent files
• llm.yaml — where model.provider is resolved
• Skills catalog — names that go in allowed_tools

llm.yaml

LLM provider registry. Each agent's model.provider must resolve to a key in this file.

Source: crates/config/src/types/llm.rs.

Shape

providers:
  minimax:
    api_key: ${MINIMAX_API_KEY:-}
    group_id: ${MINIMAX_GROUP_ID:-}
    base_url: https://api.minimax.io
    rate_limit:
      requests_per_second: 2.0
      quota_alert_threshold: 100000
  anthropic:
    api_key: ${ANTHROPIC_API_KEY:-}
    base_url: https://api.anthropic.com
    rate_limit:
      requests_per_second: 2.0
    auth:
      mode: oauth_bundle
      bundle: ./secrets/anthropic_oauth.json
retry:
  max_attempts: 5
  initial_backoff_ms: 1000
  max_backoff_ms: 60000
  backoff_multiplier: 2.0

Per-provider fields

Top-level retry block

Applies to every provider that doesn't define its own:

Retries are jittered to avoid thundering-herd reconnects. See Fault tolerance — Retry policies.

Auth modes

auth:
  mode: auto | static | token_plan | oauth_bundle
  bundle: ./secrets/anthropic_oauth.json
  setup_token_file: ./secrets/anthropic_setup.json
  refresh_endpoint: https://auth.example.com/refresh
  client_id: your-oauth-client

Supported providers

Provider-specific docs

• MiniMax M2.5
• Anthropic / Claude
• OpenAI-compatible
• Rate limiting & retry

Common mistakes

• api_key: sk-… committed to git. Use ${ENV_VAR} or ${file:./secrets/…}; the secrets/ directory is gitignored.
• Mismatched embedding_model dimensions. The vector store asserts embedding.dimensions matches the model output. A mismatch aborts startup with an explicit message.
• Setting both api_key and auth.mode: oauth_bundle. The auth mode wins. The api_key is kept as a fallback for tools that bypass the OAuth path.

Input-token reduction (context_optimization)

Four independent kill switches for prompt caching, online history compaction, pre-flight token counting, and the workspace bundle cache. Full schema, defaults, and rollout guidance in Operations → Context optimization.

broker.yaml

Broker topology, disk persistence, and fallback behavior.

Source: crates/config/src/types/broker.rs.

Shape

broker:
  type: nats          # nats | local
  url: nats://localhost:4222
  auth:
    enabled: false
    nkey_file: ./secrets/nats.nkey
  persistence:
    enabled: true
    path: ./data/queue
  limits:
    max_payload: 4MB
    max_pending: 10000
  fallback:
    mode: local_queue
    drain_on_reconnect: true

Fields

Operational notes

• type: local for single-machine dev. You don't need NATS running just to try the agent. The local broker matches NATS subject semantics, so everything works the same.
• Disk queue always on in production. Even on a single machine. It's the guarantee against losing events on a NATS blip.
• drain_on_reconnect: true is FIFO. See Event bus — Disk queue.

See also:

• Fault tolerance
• DLQ operations

memory.yaml

Short-term sessions, long-term SQLite storage, and optional vector search.

Source: crates/config/src/types/memory.rs.

Shape

short_term:
  max_history_turns: 50
  session_ttl: 24h
  max_sessions: 10000

long_term:
  backend: sqlite
  sqlite:
    path: ./data/memory.db

vector:
  enabled: false
  backend: sqlite-vec
  default_recall_mode: hybrid
  embedding:
    provider: http
    base_url: https://api.openai.com/v1
    model: text-embedding-3-small
    api_key: ${OPENAI_API_KEY}
    dimensions: 1536
    timeout_secs: 30

Short-term

Per-session conversation buffer held in memory by SessionManager.

Long-term

Persisted memory, durable across restarts.

Vector

Opt-in semantic memory.

Memory layers

flowchart LR
    MSG[incoming message] --> STM[short-term<br/>in-memory buffer]
    STM -->|turns exceed max| PRUNE[prune]
    PRUNE --> LTM[(long-term<br/>SQLite)]
    LTM --> EMB{vector<br/>enabled?}
    EMB -->|yes| VEC[(sqlite-vec index)]
    TOOL[memory tool] --> RECALL{recall mode}
    RECALL -->|keyword| LTM
    RECALL -->|vector| VEC
    RECALL -->|hybrid| LTM
    RECALL -->|hybrid| VEC

Per-agent isolation

Each agent's memory DB lives under its workspace when workspace_git is enabled — keeps memories forensically reviewable and prevents one agent from reading another's history.

See also:

• Memory — long-term
• Memory — vector search
• Soul, identity & learning

Drop-in agents

config/agents.d/*.yaml is a merge-directory for agent definitions that should not live in agents.yaml — typically anything with business content (sales prompts, pricing tables, internal phone numbers, customer-facing identities).

Source: crates/config/src/lib.rs (merge logic).

Why it exists

• Keep agents.yaml public-safe and checked into git
• Keep sensitive content gitignored and loaded at runtime
• Compose layered configs (00-dev.yaml, 10-prod.yaml) without editing a single monolithic file
• Ship .example.yaml templates so the shape stays discoverable

.gitignore rules include:

config/agents.d/*.yaml
!config/agents.d/*.example.yaml

The .example.yaml files are committed and serve as templates; the real .yaml files are not.

Merge order

Files are loaded in lexicographic filename order and their agents arrays are concatenated to the base agents.yaml:

flowchart TD
    BASE[agents.yaml] --> MERGE[merged catalog]
    D1[agents.d/00-shared.yaml] --> MERGE
    D2[agents.d/10-ana.yaml] --> MERGE
    D3[agents.d/20-kate.yaml] --> MERGE
    EX[agents.d/ana.example.yaml] -.->|gitignored template<br/>usually not loaded| MERGE

Every file must have the top-level agents: [...] shape:

# config/agents.d/10-ana.yaml
agents:
  - id: ana
    model:
      provider: minimax
      model: MiniMax-M2.5
    plugins: [whatsapp]
    inbound_bindings:
      - plugin: whatsapp
    system_prompt: |
      …private content…

Agent id collisions

Two files cannot define the same agent.id. On collision the loader fails fast with a clear message. If you want to override an agent, either:

• Replace the entry (rename or remove the original)
• Use inbound_bindings[] per-binding overrides inside a single entry

Common patterns

Public vs. private split

config/agents.yaml                  # committed, only support/ops agents
config/agents.d/ana.yaml            # gitignored, full sales prompt
config/agents.d/kate.yaml           # gitignored, personal assistant
config/agents.d/ana.example.yaml    # committed, empty template

Environment layering

config/agents.d/00-common.yaml      # shared defaults
config/agents.d/10-dev.yaml         # dev-only overrides (loaded only on dev box)

Swap the 10-*.yaml file per environment. Docker compose can mount the right one from a secret volume.

Validation

• #[serde(deny_unknown_fields)] still applies to every file
• validate_agents() runs after the merge — checks duplicate ids, missing plugin references, invalid skill directories
• Errors name the file and the offending agent id

Per-agent credentials

Bind each agent to specific WhatsApp / Telegram / Google accounts so outbound traffic originates from the right number, bot, or mailbox — never from a shared pool.

Mental model

Three layers:

1. Plugin instance — a labelled WhatsApp session or Telegram bot in config/plugins/{whatsapp,telegram}.yaml. Each instance owns its own token / session_dir and an optional allow_agents list.
2. Google account — an entry in the optional config/plugins/google-auth.yaml. Each account is 1:1 with an agent_id.
3. Agent binding — in config/agents.d/<agent>.yaml, the credentials: block pins the agent to the instance / account it may use for outbound tool calls.

The runtime runs a boot-time gauntlet that cross-checks all three layers before any plugin boots. Every invariant violation surfaces in a single report so you can fix the full YAML in one edit.

Config schemas

config/agents.d/ana.yaml

agents:
  - id: ana
    credentials:
      whatsapp: personal        # must match whatsapp.yaml instance
      telegram: ana_bot         # must match telegram.yaml instance
      google:   ana@gmail.com   # must match google-auth.yaml accounts[].id
      # Opt-out for the symmetric-binding warning when inbound bot and
      # outbound bot are intentionally different:
      # telegram_asymmetric: true
    inbound_bindings:
      - { plugin: whatsapp, instance: personal }
      - { plugin: telegram, instance: ana_bot }

config/plugins/whatsapp.yaml

whatsapp:
  - instance: personal
    session_dir: ./data/workspace/ana/whatsapp/personal
    media_dir:   ./data/media/whatsapp/personal
    allow_agents: [ana]           # defense-in-depth ACL
  - instance: work
    session_dir: ./data/workspace/kate/whatsapp/work
    media_dir:   ./data/media/whatsapp/work
    allow_agents: [kate]

config/plugins/telegram.yaml

telegram:
  - instance: ana_bot
    token: ${file:./secrets/telegram/ana_token.txt}
    allow_agents: [ana]
    allowlist:
      chat_ids: [1194292426]
  - instance: kate_bot
    token: ${file:./secrets/telegram/kate_token.txt}
    allow_agents: [kate]

config/plugins/google-auth.yaml

google_auth:
  accounts:
    - id: ana@gmail.com
      agent_id: ana                       # 1:1 — the gauntlet enforces it
      client_id_path:     ./secrets/google/ana_client_id.txt
      client_secret_path: ./secrets/google/ana_client_secret.txt
      token_path:         ./secrets/google/ana_token.json
      scopes:
        - https://www.googleapis.com/auth/gmail.modify

Agents that still declare the legacy inline google_auth block are auto-migrated into this store on boot (a warning tells you to migrate).

What the gauntlet validates

Linux permission check is skipped for /run/secrets/* (Docker secrets) and can be disabled entirely with CHAT_AUTH_SKIP_PERM_CHECK=1.

Topics

Outbound tool calls land on instance-suffixed topics when the resolver has a binding:

plugin.outbound.whatsapp.<instance>
plugin.outbound.telegram.<instance>

Unlabelled (instance: None) plugin entries keep publishing to the legacy bare topic plugin.outbound.whatsapp / plugin.outbound.telegram for full back-compat.

CLI gate

# Run the full gauntlet without booting the daemon. Exits 0 clean,
# 1 on errors, 2 on warnings-only.
agent --config ./config --check-config

# Promote warnings to errors (CI lane).
agent --config ./config --check-config --strict

The gate scans agents.yaml, every agents.d/*.yaml, whatsapp.yaml, telegram.yaml, and google-auth.yaml. Sample failure:

credentials: FAILED with 1 error(s):
   1. agent 'ana_per_binding_example' binds credentials.telegram='ana_tg' but no such telegram instance exists (available: [])

Secrets in logs

The credential layer never logs a raw account id. Every reference is via an 8-byte sha256(account_id) fingerprint rendered as hex:

2025-04-24T16:03:42Z INFO credentials.audit agent="ana" channel="whatsapp" fp=a3f2…7c direction=outbound

The fingerprint is pinned — switching the algorithm is an explicit breaking change tracked by crates/auth/tests/fingerprint_stability.rs.

Observability

Nine Prometheus series land at /metrics:

Back-compat

• Configs without a credentials: block keep working — the resolver infers outbound from the single inbound_bindings entry when it is unambiguous; otherwise outbound tools are marked unbound and fall back to the legacy bare topic.
• Plugin entries with instance: None stay on the legacy bare topic.
• agents.<id>.google_auth still registers google_* tools for that agent; google-auth.yaml is preferred going forward.

Hot-reload (no daemon restart)

Edit agents.d/*.yaml, plugins/whatsapp.yaml, plugins/telegram.yaml, or plugins/google-auth.yaml, then trigger a reload via the loopback admin endpoint:

curl -fsSX POST http://127.0.0.1:9091/admin/credentials/reload | jq

{
  "accounts_wa": 2,
  "accounts_tg": 2,
  "accounts_google": 1,
  "warnings": [],
  "version": 4
}

The resolver runs the gauntlet against the fresh files, then atomically swaps bindings in place. Plugin tools holding Arc<…> references see the new state on their next call. Failure mode: gauntlet errors return HTTP 400 with the error list; the previous bindings stay active so a typo in YAML does not knock out the runtime.

CredentialHandles already issued to in-flight tool calls keep working — handles are by-value clones; the resolver only mediates lookup of future calls.

What the reload does NOT cover

• Adding a brand-new WhatsApp / Telegram instance still requires a restart for the plugin (each instance owns its own session_dir
  • websocket). The resolver picks up the new account but the plugin side stays as-was until next boot.
• Removing an account leaks its breaker entry in BreakerRegistry until restart. No correctness impact.

Google client_id / client_secret rotation

Rewriting the secret files (./secrets/<agent>_google_client_id.txt, ..._client_secret.txt) is picked up automatically on the next google_* tool call — GoogleAuthClient checks file mtime before each network hop and re-reads when it advanced. No reload call required for that case. Audit log line:

INFO credentials.audit event="google_secrets_refreshed" \
  google_*: re-read client_id/client_secret after on-disk rotation

Strict mode

agent --check-config --strict promotes warnings to errors. Two checks behave differently under strict:

Run --strict in CI to gate PRs that touch credential YAML.

Migrating

1. Add instance: + allow_agents: to each entry in whatsapp.yaml / telegram.yaml.
2. Create config/plugins/google-auth.yaml with one accounts[] per agent that needs Gmail.
3. Add credentials: to each agents.d/*.yaml.
4. Run agent --check-config --strict. Fix every listed error.
5. Commit.

pollers.yaml

The Phase 19 generic poller subsystem. One runner orchestrates N modules — each module is an impl Poller (gmail, rss, calendar, webhook_poll, or anything you write yourself) — and every module shares the same scheduler, lease, breaker, cursor persistence, and outbound dispatch via Phase 17 credentials.

Source: crates/poller/, crates/config/src/types/pollers.rs.

Top-level shape

pollers:
  enabled: true
  state_db: ./data/poller.db
  default_jitter_ms: 5000
  lease_ttl_factor: 2.0
  failure_alert_cooldown_secs: 3600
  breaker_threshold: 5
  jobs:
    - id: ana_leads
      kind: gmail
      agent: ana
      schedule: { every_secs: 60 }
      config:
        query: "is:unread subject:lead"
        deliver: { channel: whatsapp, to: "57300...@s.whatsapp.net" }
        message_template: |
          New lead 🚨
          {snippet}

Absent file → subsystem off (no jobs spawn, no admin endpoint).

Top-level fields

Per-job fields

Schedules

# Repeat every N seconds. Most common.
schedule: { every_secs: 60 }

# 6-field cron: sec min hour dom mon dow.
schedule:
  cron: "0 */5 * * * *"          # every 5 minutes on the boundary
  tz: "America/Bogota"           # accepted; evaluated in UTC unless cron-tz feature on
  stagger_jitter_ms: 2000        # local override for this job

# One-shot at an RFC3339 instant. After it fires the job stays paused.
schedule: { at: "2026-04-26T15:00:00Z" }

Built-ins

gmail

- id: ana_leads
  kind: gmail
  agent: ana
  schedule: { every_secs: 60 }
  config:
    query: "is:unread subject:(lead OR interesado)"
    newer_than: "1d"             # avoids back-filling years on first deploy
    max_per_tick: 20
    dispatch_delay_ms: 1000      # throttle between dispatches in same tick
    sender_allowlist: ["@mycompany.com"]
    extract:
      name: "Nombre:\\s*(.+)"
      phone: "Tel:\\s*(\\+?\\d+)"
    require_fields: [name, phone]
    message_template: |
      New lead 🚨 {name} — {phone}
      {snippet}
    mark_read_on_dispatch: true
    deliver: { channel: whatsapp, to: "57300...@s.whatsapp.net" }

Multiple gmail jobs for the same agent share a cached GoogleAuthClient — token refreshes happen once across all jobs.

google_* errors are classified: 401 / invalid_grant / revoked → Permanent (auto-pause), 5xx / network → Transient (backoff).

rss

- id: ana_blog_watch
  kind: rss
  agent: ana
  schedule: { every_secs: 600 }
  config:
    feed_url: https://example.com/feed.xml
    max_per_tick: 5
    message_template: "{title}\n{link}"
    deliver: { channel: telegram, to: "1194292426" }

ETag from the previous response is sent as If-None-Match. 304 Not Modified produces a zero-cost tick.

webhook_poll

- id: ana_jira_assigned
  kind: webhook_poll
  agent: ana
  schedule: { every_secs: 300 }
  config:
    url: https://company.atlassian.net/rest/api/3/search
    method: GET
    headers:
      Authorization: "Bearer ${JIRA_TOKEN}"
      Accept: "application/json"
    items_path: "issues"        # dotted path to the array; "" for root
    id_field: "id"              # field used for dedup
    max_per_tick: 10
    message_template: "[{key}] {fields}"
    deliver: { channel: telegram, to: "1194292426" }
    # SSRF guard — must opt in to hit private / loopback hosts:
    # allow_private_networks: true

401 / 403 → Permanent. Any other 4xx → Permanent. 5xx → Transient.

google_calendar

- id: ana_calendar_sync
  kind: google_calendar
  agent: ana
  schedule: { every_secs: 300 }
  config:
    calendar_id: primary
    skip_cancelled: true
    message_template: "📅 {summary} — {start}\n{html_link}"
    deliver: { channel: telegram, to: "1194292426" }

First tick captures nextSyncToken and dispatches nothing (baseline). Subsequent ticks use syncToken=... and dispatch the diff. 410 Gone (token expired) is classified Permanent — operator runs agent pollers reset <id> to re-baseline.

Multi-job per built-in

Same agent + same kind, multiple jobs — completely independent. The runner gives each its own cursor, breaker, schedule, metrics, and pause/resume controls. The GoogleAuthClient is the only thing shared (intentional, so quota and refresh costs aren't multiplied).

# Three Gmail polls for Ana, all independent
- id: ana_leads
  kind: gmail
  agent: ana
  schedule: { every_secs: 60 }
  config:
    query: "is:unread label:lead"
    deliver: { channel: whatsapp, to: "57300...@s.whatsapp.net" }
    # …

- id: ana_invoices
  kind: gmail
  agent: ana
  schedule: { every_secs: 600 }
  config:
    query: "is:unread label:invoice"
    deliver: { channel: telegram, to: "1194292426" }
    # …

- id: ana_alerts
  kind: gmail
  agent: ana
  schedule: { cron: "0 */15 * * * *" }
  config:
    query: "is:unread from:monitor@infra.com"
    deliver: { channel: telegram, to: "9876543210" }
    # …

Pause ana_invoices independently with agent pollers pause ana_invoices.

CLI

agent pollers list                 # plain table; --json for machine output
agent pollers show ana_leads      # detail of one job
agent pollers run ana_leads       # manual tick (bypasses schedule + lease)
agent pollers pause ana_invoices  # paused = 1
agent pollers resume ana_invoices
agent pollers reset ana_calendar_sync --yes  # destructive; clears cursor
agent pollers reload              # re-read pollers.yaml + diff

The daemon must be running (CLI hits the loopback admin server at 127.0.0.1:9091).

Admin endpoints

GET  /admin/pollers
GET  /admin/pollers/<id>
POST /admin/pollers/<id>/run
POST /admin/pollers/<id>/pause
POST /admin/pollers/<id>/resume
POST /admin/pollers/<id>/reset
POST /admin/pollers/reload

reload returns a ReloadPlan JSON: { add, replace, remove, keep }. Validation runs across every job in the new file before any task is touched — a typo never knocks healthy siblings offline.

Agent tools

When the poller subsystem is up, every agent gets six LLM-callable tools registered on its ToolRegistry:

Each registered Poller impl can also expose per-kind custom tools via Poller::custom_tools() — gmail ships gmail_count_unread out of the box. See Build a poller.

Create / delete are intentionally not exposed: prompt-injection could plant a webhook_poll aimed at internal infra. Operators own pollers.yaml + agent pollers reload.

Failure-destination

- id: ana_leads
  kind: gmail
  # …
  failure_to:
    channel: telegram
    to: "1194292426"     # alerts on the operator's chat

When the per-job circuit breaker trips (consecutive_errors >= breaker_threshold), the runner publishes a text message to the configured channel (resolved via Phase 17 just like the happy path) and records the timestamp for cooldown gating. Cooldown is failure_alert_cooldown_secs global default, overridable per job in a future revision.

Observability

Seven Prometheus series exposed under /metrics:

Migrating from gmail-poller.yaml

The legacy crate nexo-plugin-gmail-poller keeps its YAML schema but no longer drives its own loop. On boot the wizard auto-translates every legacy job into a kind: gmail entry, folds it into cfg.pollers.jobs, and logs a deprecation warn. Explicit entries in pollers.yaml win on id collision so a manual migration is never clobbered.

To migrate cleanly:

1. Run agent --check-config to print every translated id.
2. Copy each into config/pollers.yaml under pollers.jobs, adjusting the agent: field if the legacy agent_id was inferred.
3. Delete config/plugins/gmail-poller.yaml.

MiniMax M2.5

MiniMax M2.5 is the primary LLM provider for nexo-rs. It's the first provider implemented and the recommended default for new agents.

Source: crates/llm/src/minimax.rs, crates/llm/src/minimax_auth.rs.

Why it's primary

• Strong tool-calling support on both the OpenAI-compat wire and the Anthropic Messages wire
• Token Plan auth lets you run agents on a subscription without per-request billing headaches
• Aggressive price/performance for multi-agent deployments

If you don't have a specific reason to pick another provider, start with MiniMax.

Configuration

# config/llm.yaml
providers:
  minimax:
    api_key: ${MINIMAX_API_KEY:-}
    group_id: ${MINIMAX_GROUP_ID:-}
    base_url: https://api.minimax.io
    rate_limit:
      requests_per_second: 2.0
      quota_alert_threshold: 100000

Per-agent selection:

# config/agents.d/ana.yaml
agents:
  - id: ana
    model:
      provider: minimax
      model: MiniMax-M2.5

Wire formats (api_flavor)

MiniMax exposes two HTTP shapes. The client auto-detects from base_url but can be overridden via api_flavor.

Auto-detection: if base_url ends in /anthropic, the client picks anthropic_messages automatically.

Authentication

Static API key

Simple path: put the key in env or a secrets file.

Env var precedence (first wins):

1. MINIMAX_CODE_PLAN_KEY
2. MINIMAX_CODING_API_KEY
3. ./secrets/minimax_code_plan_key.txt
4. api_key field in llm.yaml

Token Plan OAuth bundle

For subscription-based access. The wizard writes a bundle to ./secrets/minimax_token_plan.json:

{
  "access_token": "...",
  "refresh_token": "...",
  "expires_at": "2026-05-01T12:00:00Z",
  "region": "https://api.minimax.io"
}

Auto-refresh: 60 seconds before expires_at, a background task POSTs to {region}/oauth/token with grant_type=refresh_token and rewrites the bundle atomically. Concurrent refreshes are serialized behind a mutex — you never get two refresh calls in flight.

Mid-flight 401: if an API call returns 401 while holding what we thought was a valid token (clock skew, revocation), the client force-refreshes once and retries the request. A second 401 is surfaced as a credential error.

Shared OAuth client id for the MiniMax Portal flow: 78257093-7e40-4613-99e0-527b14b39113.

Request / response flow

sequenceDiagram
    participant A as Agent loop
    participant RL as RateLimiter
    participant C as MiniMaxClient
    participant AU as AuthSource
    participant MX as MiniMax API

    A->>C: chat(ChatRequest)
    C->>RL: acquire()
    C->>AU: fresh_bearer()
    AU->>AU: refresh if <60s to expiry
    AU-->>C: access_token
    C->>MX: POST chatcompletion_v2 / v1/messages
    alt 200
        MX-->>C: ChatResponse
    else 401
        C->>AU: force_refresh()
        C->>MX: retry once
    else 429
        MX-->>C: Retry-After
        C-->>A: LlmError::RateLimit
    else 5xx
        MX-->>C: error body
        C-->>A: LlmError::ServerError
    end

Supported features

Rate limiting

Per-provider token bucket. requests_per_second: 2.0 refills one slot every 500 ms. Acquired before every request.

An optional quota_alert_threshold emits a structured warn log when the remaining quota (if the provider reports it) crosses the threshold. Useful for Prometheus alerting.

Error classification

See Retry & rate limiting.

Common mistakes

• Forgetting group_id. MiniMax requires a group id alongside the key for most endpoints. The wizard sets this; manual configs often miss it.
• Pointing base_url at /anthropic with a regular API key. That endpoint is for Token Plan / Coding keys only — regular keys will 401. Leave base_url at https://api.minimax.io.
• Refreshing the bundle manually mid-flight. The client already serializes refreshes. Editing the file while the agent runs can lead to an atomic write race — stop the agent, edit, restart.

Anthropic / Claude

Native Anthropic client with multiple authentication paths: static API key, setup tokens, full OAuth PKCE subscription flow, or automatic import from the local Claude Code CLI.

Source: crates/llm/src/anthropic.rs, crates/llm/src/anthropic_auth.rs. Phase 15 added the subscription flow end-to-end.

Configuration

# config/llm.yaml
providers:
  anthropic:
    api_key: ${ANTHROPIC_API_KEY:-}
    base_url: https://api.anthropic.com
    rate_limit:
      requests_per_second: 2.0
    auth:
      mode: oauth_bundle
      bundle: ./secrets/anthropic_oauth.json

Per-agent selection:

model:
  provider: anthropic
  model: claude-haiku-4-5

Authentication modes

auto resolution order

Used when auth.mode: auto or omitted:

flowchart TD
    START[anthropic client build] --> B1{oauth_bundle<br/>file exists?}
    B1 -->|yes| USE1[use OAuth bundle]
    B1 -->|no| B2{Claude Code CLI<br/>credentials found?}
    B2 -->|yes| USE2[import from<br/>~/.claude/.credentials.json]
    B2 -->|no| B3{setup_token<br/>file exists?}
    B3 -->|yes| USE3[use setup token]
    B3 -->|no| B4{api_key<br/>set?}
    B4 -->|yes| USE4[use static key]
    B4 -->|no| FAIL([fail: no credentials])

OAuth bundle

The wizard runs a PKCE flow in the browser and writes the bundle to ./secrets/anthropic_oauth.json:

{
  "access_token": "...",
  "refresh_token": "...",
  "expires_at": "2026-05-01T12:00:00Z"
}

• Refresh endpoint: https://console.anthropic.com/v1/oauth/token
• Refresh cadence: 60 seconds before expires_at, background task POSTs grant_type=refresh_token
• Concurrency: all refreshes serialize behind a mutex
• Shared OAuth client id: 9d1c250a-e61b-44d9-88ed-5944d1962f5e
• Stale-token handling: a 401 mid-flight marks the token stale so the next refresh fires immediately instead of waiting for the expiry window

CLI credentials import

If you're already running Claude Code CLI on the same host, the client auto-detects and imports ~/.claude/.credentials.json. Zero config — if it exists and is valid, it's used.

Tool calling

Native Anthropic shape:

• Tool definitions: {name, description, input_schema}
• Tool invocation: tool_use blocks with id, name, input
• Tool result: tool_result blocks correlated via tool_use_id

Streaming uses native SSE; a dedicated parser in crates/llm/src/stream.rs handles message_start, content_block_*, and message_delta events.

Error classification

Supported features

• Chat completions ✅
• Tool calling ✅
• Streaming (SSE) ✅
• Multimodal (images) ✅
• Prompt caching ✅ (via Anthropic beta headers)
• Extended thinking ✅ (model-dependent)

Common mistakes

• Setup-token string under 80 chars. The setup-token validator refuses it at parse time. Make sure you pasted the full string.
• api_key + oauth_bundle both set. The auth mode wins. The static key is kept only as a fallback the auto-resolver may pick up if the bundle is missing.
• Claude Code CLI credentials being used unintentionally. If auto mode is on and you installed CLI on the host, that path wins before api_key. Set auth.mode: static to pin the static key.

OpenAI-compatible

Client for OpenAI itself and for any upstream that speaks the same wire: Ollama, Groq, OpenRouter, LM Studio, vLLM, Azure OpenAI, or your own proxy.

Source: crates/llm/src/openai_compat.rs.

Configuration

# config/llm.yaml
providers:
  openai:
    api_key: ${OPENAI_API_KEY:-}
    base_url: https://api.openai.com/v1
    rate_limit:
      requests_per_second: 2.0

Per-agent:

model:
  provider: openai
  model: gpt-4o

Known-working upstreams

Point base_url at any of these and it works out of the box:

Authentication

Single mode: static API key sent as Authorization: Bearer <key>. Some upstreams ignore the key entirely (Ollama, local vLLM) — supply any non-empty string to satisfy the config validator.

Features & gaps

Feature gating when the upstream lacks support: we do not pre-probe features — a call that requires a feature the upstream doesn't speak will fail with the upstream's own error (typically a 400). The error bubbles up as LlmError::Other and does not retry, so you notice quickly.

Error classification

Common mistakes

• Trailing slash in base_url. Some upstreams are lenient, some are not. Stick to the form shown in the table.
• Using Azure OpenAI without the deployment path. Azure requires an extra segment (/openai/deployments/<name>/chat/completions) that the vanilla OpenAI path doesn't. Currently not supported out of the box; use a proxy or a custom provider if you need Azure.
• Relying on JSON mode everywhere. Many local servers don't enforce schemas. Validate the response yourself when using Ollama / LM Studio for critical tool args.

DeepSeek

Connector for DeepSeek's hosted models. The API is OpenAI-compatible end to end (same /v1/chat/completions shape, same SSE streaming, same Bearer auth) so the connector is a thin factory that wraps OpenAiClient with DeepSeek's default endpoint.

Source: crates/llm/src/deepseek.rs.

Configuration

# config/llm.yaml
providers:
  deepseek:
    api_key: ${DEEPSEEK_API_KEY}
    # base_url defaults to https://api.deepseek.com/v1 when blank.
    # Override only for self-hosted gateways or testing fixtures.
    base_url: ""
    rate_limit:
      requests_per_second: 2.0
      quota_alert_threshold: 100000

Pin the agent to it:

agents:
  - id: ana
    model:
      provider: deepseek
      model: deepseek-chat

Models

deepseek-reasoner agents must therefore leave allowed_tools empty (or list only tools the agent never plans to invoke). Tool calls fired against the reasoner endpoint return an error from upstream.

Streaming

Identical to OpenAI's SSE format, so OpenAiClient::chat_stream parses it without per-provider code. nexo_llm_stream_ttft_seconds and nexo_llm_stream_chunks_total Prometheus series labelled with provider="deepseek" show up automatically.

Tool calling

deepseek-chat follows OpenAI's tool-calling spec verbatim. JSON arguments deserialise the same way; parallel_tool_calls is honoured.

Rate limits

DeepSeek returns standard 429 with a retry-after header. The existing retry plumbing (crates/llm/src/retry.rs) consumes that header so 429s back off cleanly without touching the connector.

Quota / cost

DeepSeek's pricing is per-1M-tokens; the TokenUsage returned by each ChatResponse is forwarded to the standard agent_llm_tokens_total counter (labels: provider="deepseek", model, usage_kind).

Known limitations

• No native embeddings client — DeepSeek does not currently publish an embeddings endpoint. Use a different provider for embedding_model if your agent needs vector search.
• Reasoner tool-call gap — see Models. Validate at boot by leaving allowed_tools: [] on agents pinned to deepseek-reasoner.
• Cache awareness — DeepSeek's KV-cache hit information is surfaced through the same cache_usage field as the OpenAI client reports it.

See also

• OpenAI-compatible — same wire format, full notes on the underlying client.
• Rate limiting & retry — backoff policy.

Rate limiting & retry

Every LLM provider client sits behind a token bucket and a bounded retry policy with decorrelated jittered exponential backoff. This page is the definitive reference for those two mechanisms.

Source: crates/llm/src/retry.rs, crates/llm/src/rate_limiter.rs, crates/llm/src/quota_tracker.rs.

Rate limiter

Token bucket, acquired before every outbound request.

• interval = 1 / requests_per_second
• One token per request
• Bucket fully refills after interval per slot
• Per-provider, per-agent — each client has its own bucket, so one noisy agent can't starve another even when they share a provider

rate_limit:
  requests_per_second: 2.0
  quota_alert_threshold: 100000   # optional

At 2.0 rps, the bucket tops up a slot every 500 ms. A burst of 3 requests will wait briefly on the third.

Quota tracker

Optional. When a provider returns remaining-quota info (header, response body), quota_tracker records it via record_usage() on the token response. If the remaining crosses quota_alert_threshold, a structured warn log is emitted:

WARN quota threshold crossed  provider=minimax remaining=99500 threshold=100000

Pair with a Prometheus log-scraping rule for an alert.

Retry policy

Retries live above the circuit breaker. They handle transient failures that don't warrant flipping the breaker.

Decorrelated jittered backoff

Not simple exponential — the next backoff is a uniform random draw in a growing range:

next = uniform(base, max(base, last × multiplier))

Defaults from llm.yaml retry block:

Why decorrelated jitter: multiple clients hitting the same 429 don't re-fire in lockstep. Desynchronization is built-in.

flowchart LR
    REQ[request] --> API{API response}
    API -->|200| OK[return ChatResponse]
    API -->|429| RL[RateLimit]
    API -->|5xx| SE[ServerError]
    API -->|401| AU[CredentialInvalid]
    API -->|4xx| F[Other fail fast]

    RL --> D1{attempts<br/>< 5?}
    SE --> D2{attempts<br/>< 3?}
    AU --> REF[auth refresh<br/>+ single retry]
    D1 -->|yes| BO1[wait max(retry_after,<br/>jittered_backoff)]
    D1 -->|no| F
    D2 -->|yes| BO2[wait jittered_backoff]
    D2 -->|no| F
    BO1 --> REQ
    BO2 --> REQ
    REF --> REQ

Error classification per provider

The providers classify HTTP responses into a shared LlmError so the retry layer can be common code:

Tuning

• Bursty workloads: bump requests_per_second cautiously; the upstream's own rate limits won't move, so you'll just pay more 429s to find the ceiling.
• Flaky networks: raise max_attempts for 5xx; keep max_backoff_ms bounded so slow agents don't spiral.
• Subscription plans: lower requests_per_second to keep daily usage under caps; pair with quota_alert_threshold.

See also

• Fault tolerance — CircuitBreaker
• Operations — Metrics

WhatsApp

End-to-end WhatsApp channel: Signal Protocol pairing, inbound message bridge, outbound send/reply/reaction/media tools, optional voice transcription.

Source: crates/plugins/whatsapp/ (thin wrapper over the whatsapp-rs crate).

Topics

During pairing the plugin also publishes qr lifecycle events on the inbound topic so the wizard can render the QR.

Config

# config/plugins/whatsapp.yaml
whatsapp:
  enabled: true
  session_dir: ""            # empty → per-agent default
  media_dir: ./data/media/whatsapp
  instance: default
  acl:
    allow_list: []           # empty + empty env = open ACL
    from_env: WA_AGENT_ALLOW
  behavior:
    ignore_chat_meta: true
    ignore_from_me: true
    ignore_groups: false
  bridge:
    response_timeout_ms: 30000
    on_timeout: noop         # noop | apology_text
  transcriber:
    enabled: false
    skill: whisper
  public_tunnel:
    enabled: false
    only_until_paired: true

Key fields:

Pairing

Pairing is setup-time only. The runtime refuses to start without paired credentials.

sequenceDiagram
    participant U as Operator
    participant W as agent setup
    participant WA as whatsapp-rs Client
    participant P as Phone

    U->>W: setup pair whatsapp --agent ana
    W->>WA: new_in_dir(session_dir)
    WA-->>W: QR image
    W-->>U: render QR (Unicode blocks)
    U->>P: Settings → Linked Devices → scan
    P->>WA: pair
    WA-->>W: Connected
    W->>W: persist creds to session_dir/.whatsapp-rs/creds.json

• Credentials at <session_dir>/.whatsapp-rs/creds.json
• Daemon-collision check at <session_dir>/.whatsapp-rs/daemon.json blocks a second process on the same account
• Multi-account via Client::new_in_dir() — no XDG_DATA_HOME mutation
• Credential expiry mid-run (401 loop) → operator must re-pair; no runtime QR fallback

Tools exposed to the LLM

All tools honor the per-binding outbound_allowlist.whatsapp — empty list = unrestricted, populated = hard allowlist.

Event shapes

Inbound payloads (on plugin.inbound.whatsapp[.<instance>]):

// message
{
  "kind": "message",
  "from": "573000000000@s.whatsapp.net",
  "chat": "573000000000@s.whatsapp.net",
  "text": "hi",
  "reply_to": null,
  "is_group": false,
  "timestamp": 1714000000,
  "msg_id": "3EB0..."
}

// media_received
{
  "kind": "media_received",
  "from": "...",
  "chat": "...",
  "msg_id": "...",
  "local_path": "./data/media/whatsapp/abc.jpg",
  "mime": "image/jpeg",
  "caption": null
}

// qr  (pairing only)
{"kind": "qr", "ascii": "...", "png_base64": "...", "expires_at": ...}

// lifecycle
{"kind": "connected" | "disconnected" | "reconnecting" | "credentials_expired"}

// observability
{"kind": "bridge_timeout", "msg_id": "...", "waited_ms": 30000}

Gotchas

• Shared session_dir across agents = cross-delivery. Each agent should point at its own <workspace>/whatsapp/default. The wizard does this automatically; manual configs need care.
• ignore_chat_meta: true silently skips muted/archived chats. If a user archives a chat on the phone, the agent never sees it again until they unarchive.
• Credential expiry is irreversible without re-pair. whatsapp-rs will loop on 401. Watch for credentials_expired lifecycle events and alert.

See Setup wizard — WhatsApp pairing.

Telegram

Bot API channel with long-polling intake, multi-bot routing, full send/reply/reaction/edit/location/media tool surface, and optional voice auto-transcription.

Source: crates/plugins/telegram/.

Topics

Each instance subscribes only to its own outbound topic, so two bots in the same process don't cross-wire.

Config

# config/plugins/telegram.yaml
telegram:
  token: ${file:./secrets/telegram_token.txt}
  instance: sales_bot
  polling:
    enabled: true
    interval_ms: 25000
    offset_path: ./data/media/telegram/sales_bot.offset
  allowlist:
    chat_ids: []        # empty = accept all
  auto_transcribe:
    enabled: false
    command: ./extensions/openai-whisper/target/release/openai-whisper
    language: es
  bridge_timeout_ms: 120000

Key fields:

Auth

Single mode: static bot token. No OAuth. Store it under ./secrets/ and reference via ${file:...}.

flowchart LR
    SETUP[agent setup] --> ASK[ask for bot token]
    ASK --> F[./secrets/telegram_token.txt]
    F -.->|${file:...}| CFG[config/plugins/telegram.yaml]
    CFG --> RUN[runtime: HTTP Bot API with long-poll]

Tools exposed to the LLM

All tools enforce outbound_allowlist.telegram per binding.

Event shapes

// message
{
  "kind": "message",
  "from": "12345",
  "chat": "12345",
  "chat_type": "private",
  "text": "hi",
  "reply_to": null,
  "is_group": false,
  "timestamp": 1714000000,
  "msg_id": "42",
  "username": "jdoe",
  "media": [],
  "latitude": null,
  "longitude": null,
  "forward": null
}

// media item (inside `media`)
{
  "kind": "voice" | "photo" | "video" | "document" | "audio",
  "local_path": "./data/media/telegram/....ogg",
  "file_id": "AgACAgEA...",
  "mime_type": "audio/ogg",
  "duration_s": 4,
  "width": null,
  "height": null,
  "file_name": null
}

// callback_query  (inline-keyboard button press, auto-ACKed)
{"kind": "callback_query", "from": "...", "chat": "...", "data": "buy"}

// chat_membership
{"kind": "chat_membership", "chat": "...", "status": "added" | "kicked" | ...}

// lifecycle
{"kind": "connected" | "disconnected"}
{"kind": "bridge_timeout", "msg_id": "...", "waited_ms": ...}

Forwarded messages include a forward object:

"forward": {
  "source": "user" | "channel" | "chat",
  "from_user_id": 12345,
  "from_chat_id": null,
  "date": 1714000000
}

Gotchas

• Webhook mode is not supported yet. Long-polling only.
• polling.interval_ms is clamped by Telegram. Values outside [1000, 50000] get capped by the server side; default 25000 is a good middle ground.
• Negative chat ids are groups/channels. Telegram uses negative ids for group chats; positive for private. Don't strip the sign.
• Auto-transcribe requires the whisper skill extension. The command path must point at a working binary, otherwise inbound voice messages arrive without text.

Email

Generic SMTP/IMAP plugin. Scaffolded but not yet wired — config shape is defined, but no tool surface or inbound bridge ships today. For a working Gmail → agent pipeline today, use gmail-poller.

Source: crates/plugins/email/ (empty lib.rs), config in crates/config/src/types/plugins.rs.

Config

# config/plugins/email.yaml
email:
  smtp:
    host: smtp.example.com
    port: 587
    username: agent@example.com
    password: ${file:./secrets/email_password.txt}
  imap:
    host: imap.example.com
    port: 993

Status

• No NATS topics active
• No tools exposed to the LLM
• No inbound bridge
• Config schema reserved so future phases can land incrementally

What to use instead

For inbound triage:

• gmail-poller — cron-style Gmail polling with regex capture groups and template-based dispatch to any plugin.outbound.* topic. Production-ready.

For outbound notifications:

• Delegate to a send agent wired to a transactional-email provider via a custom extension, until this plugin lands.

Track progress under the future Phase 17 in ../PHASES.md.

Browser (Chrome DevTools Protocol)

Drives a real Chrome/Chromium instance via CDP. Agents can navigate, click, fill, screenshot, and run JS — with stable element refs that work across DOM mutations within a single turn.

Source: crates/plugins/browser/.

Topics

Browser is an outbound-only plugin — there is no unsolicited inbound event from a web page to the agent.

Config

# config/plugins/browser.yaml
browser:
  headless: false
  executable: ""                     # empty → search PATH
  cdp_url: ""                        # empty → launch new Chrome
  user_data_dir: ./data/browser/profile
  window_width: 1280
  window_height: 800
  connect_timeout_ms: 10000
  command_timeout_ms: 15000
  args: []                           # extra CLI flags for Chrome

Auth

None. CDP is an unauthenticated protocol — use cdp_url only with a loopback / firewalled Chrome.

Tools exposed to the LLM

All tools are prefixed browser_* for glob filtering in allowed_tools.

Element refs

browser_snapshot emits a text tree where every actionable element has a ref like @e12. Those refs are stable within the snapshot turn but invalidated by any subsequent DOM mutation:

sequenceDiagram
    participant A as Agent
    participant B as Browser plugin
    participant C as Chrome

    A->>B: browser_snapshot
    B->>C: DOM.describeNode(..)
    C-->>B: tree
    B-->>A: "Login @e12\nEmail @e13\n..."
    A->>B: browser_fill(@e13, "user@…")
    B->>C: DOM.focus + Input.dispatch
    A->>B: browser_click(@e12)
    Note over A,B: refs still valid<br/>(same snapshot turn)
    A->>B: browser_snapshot
    Note over B: refs from prior snapshot<br/>now INVALID

Rule: take a snapshot, act on refs from that snapshot, take a new snapshot before acting again.

Gotchas

• browser_fill replaces content. No append mode. To add text to existing content, read the current value first (via evaluate) then send the merged string.
• Connecting to an existing Chrome (cdp_url) skips the profile setup. Any login state is whatever that Chrome already has.
• Element refs expire on DOM mutation. The plugin does not auto-refresh — refs from a stale snapshot will error or misfire.
• Headless sites break. Some sites detect headless Chrome and behave differently. Use headless: false for those.

Google (OAuth, Gmail, Calendar, Drive) + gmail-poller

Two related subsystems:

• google plugin — per-agent OAuth client plus a generic google_call tool that lets an agent hit any Google API the granted scopes allow
• gmail-poller plugin — cron-style scheduler that polls Gmail, matches subjects/bodies with regex, and dispatches results to any outbound topic (WhatsApp, Telegram, another agent)

Sources: crates/plugins/google/ and crates/plugins/gmail-poller/.

google — per-agent OAuth

Config

Two shapes supported:

Preferred (Phase 17) — declare accounts in a dedicated store and bind them from the agent via credentials.google:

# config/plugins/google-auth.yaml
google_auth:
  accounts:
    - id: ana@gmail.com
      agent_id: ana                     # 1:1; gauntlet enforces the binding
      client_id_path:     ./secrets/google/ana_client_id.txt
      client_secret_path: ./secrets/google/ana_client_secret.txt
      token_path:         ./secrets/google/ana_token.json
      scopes:
        - https://www.googleapis.com/auth/gmail.modify

Gmail-poller picks these up automatically; agents see google_* tools when the store has an entry matching their agent_id.

Legacy inline (still works, logs a migration warn):

# agents.yaml
google_auth:
  client_id: ${GOOGLE_CLIENT_ID}
  client_secret: ${file:./secrets/google_secret.txt}
  scopes:
    - gmail.readonly
    - gmail.send
    - calendar
    - drive.file
  token_file: ./data/workspace/ana/google_token.json
  redirect_port: 17653

Pairing flow

sequenceDiagram
    participant A as Agent LLM
    participant T as google_auth_start
    participant B as Browser
    participant L as Loopback listener<br/>127.0.0.1:<port>/callback
    participant G as Google OAuth

    A->>T: invoke
    T->>L: start listener
    T-->>A: return consent URL
    A->>B: ask user to open URL
    B->>G: consent flow
    G->>L: redirect w/ code
    L->>G: exchange code → tokens
    L->>L: persist refresh_token<br/>(mode 0o600)
    L-->>A: success

The wizard wraps this as a one-shot step, but runtime tools expose the same primitives for re-auth.

Device-code flow (headless setup)

agent setup google offers a second consent path that does not require a local browser — useful for servers, CI, and SSH-only environments. The wizard:

1. POSTs to oauth2.googleapis.com/device/code with the account's client_id and scopes.
2. Prints a 6-character user_code + a verification_url to the terminal.
3. Polls oauth2.googleapis.com/token (default every 5 s) until the operator approves on any device.
4. Persists the resulting refresh_token at token_path with mode 0o600.

╭─ Device-code OAuth ───────────────────────────────────────
│  Open in any browser:          https://www.google.com/device
│  Code to enter:                HBQM-WLNF
│  (valid for 1800s)
╰───────────────────────────────────────────────────────────

Waiting for approval...
✔ Tokens persisted at ./secrets/ana_google_token.json.

The Google Cloud Console OAuth client must be type "TVs and Limited Input devices" for this flow — Desktop/Web clients reject device-code with client_type_disabled.

Lazy-refresh of client_id / client_secret

GoogleAuthClient.config is ArcSwap<GoogleAuthConfig>. Every network call (exchange_code, request_device_code, poll_device_token, refresh_token) first invokes refresh_secrets_if_changed, which compares mtime on client_id_path and client_secret_path and re-reads them when they advance. Rotating the secret files (e.g. quarterly key rotation in Google Cloud Console) takes effect on the next tool call without a daemon restart.

Steady-state cost: one fs::metadata call per outbound request. Audit trail (target credentials.audit):

INFO event="google_secrets_refreshed" \
  google_*: re-read client_id/client_secret after on-disk rotation

Tools exposed

Supported APIs

Anything under *.googleapis.com that the granted scopes permit. Common call shapes:

• Gmail v1 — https://gmail.googleapis.com/gmail/v1/users/me/messages?q=is:unread
• Calendar v3 — https://www.googleapis.com/calendar/v3/calendars/primary/events
• Drive v3 — https://www.googleapis.com/drive/v3/files?q=mimeType='application/pdf'
• Sheets v4 — https://sheets.googleapis.com/v4/spreadsheets/<id>/values/A1:D10

Gotchas

• 401 means the refresh token was revoked. Re-auth via google_auth_start.
• 403 means a scope wasn't granted. Add the scope, revoke, re-auth.
• Token file leaks → revoke immediately. The file holds a refresh token with the granted scopes.

gmail-poller — cron-style Gmail bridge

Poll Gmail, extract fields via regex, render a template, dispatch to any outbound topic. Multi-account, allowlisted by sender substring, rate-limited per dispatch.

Config

# config/plugins/gmail-poller.yaml
gmail_poller:
  enabled: true
  interval_secs: 60
  accounts:
    - id: default
      agent_id: ana           # Phase 17 — binds the account to an agent; defaults to `id` when omitted
      token_path: ./data/workspace/ana/google_token.json
      client_id_path: ./secrets/google_client_id.txt
      client_secret_path: ./secrets/google_client_secret.txt
  jobs:
    - name: lead_forward
      account: default
      query: "is:unread subject:(lead OR interesado)"
      newer_than: 1d
      interval_secs: 120
      forward_to_subject: plugin.outbound.whatsapp.default
      forward_to: "573000000000@s.whatsapp.net"
      extract:
        name: "Nombre:\\s*(.+)"
        phone: "Tel:\\s*(\\+?\\d+)"
      require_fields: [name, phone]
      message_template: |
        New lead 🚨
        {name} — {phone}
        Subject: {subject}
        {snippet}
      mark_read_on_dispatch: true
      max_per_tick: 20
      dispatch_delay_ms: 1000
      sender_allowlist: ["@mycompany.com", "partners@"]

Per-job fields

Event shape

{
  "to": "<forward_to>",
  "kind": "text",
  "text": "<rendered message_template>",
  "subject": "<email subject>",
  "<extract key>": "<captured group>"
}

Published to <forward_to_subject>.

Error backoff

Sustained errors are backed off: [0, 0, 0, 30, 60, 120, 300] seconds (caps at 300). Transient failures don't stop the poll loop.

Gotchas

• Gmail API only — no IMAP. This plugin is Google-specific. For generic IMAP triage, use a custom extension.
• sender_allowlist is substring, not regex. Simpler to read, simpler to get wrong. Quote boundary characters explicitly.
• extract regex must compile. Invalid regex fails the whole job at boot with an error naming the field.

See also

• Setup wizard — Google OAuth
• Config — agents.yaml (per-agent google_auth)

Short-term memory

Per-session conversational buffer held entirely in memory. Tracks the turns of the ongoing conversation so the LLM has context on every completion request.

Source: crates/core/src/session/ (types.rs, manager.rs) — the Session struct owns the short-term buffer.

What lives in a session

Each Session stores:

An Interaction is {role: User | Assistant | Tool, content, timestamp}.

Sliding window — max_history_turns

short_term:
  max_history_turns: 50

Hard cap, sliding FIFO. When history.len() > max_history_turns, the oldest entry is removed on the next push:

flowchart LR
    MSG[new turn] --> PUSH[history.push]
    PUSH --> CHECK{len > max?}
    CHECK -->|no| DONE[done]
    CHECK -->|yes| DROP[history.remove(0)]
    DROP --> DONE

Old content is lost, not promoted. If you need long-term persistence, the agent must explicitly call the memory tool with action remember. See Long-term memory.

Session cap and eviction

short_term:
  max_sessions: 10000

Soft cap across the whole process. On overflow, the oldest-idle session (lowest last_access) is evicted to make room. Eviction fires the on_expire callbacks — used by workspace-git to checkpoint before tearing down the session.

max_sessions: 0 disables the cap (unbounded). Leave it at the default unless you have a specific reason — the cap is DoS protection against a spammer rotating chat_ids.

TTL sweeper

short_term:
  session_ttl: 24h

Sessions expire after session_ttl of inactivity. The sweeper runs every ttl / 4 (so every 6 h with the default 24 h TTL) and drops expired sessions.

stateDiagram-v2
    [*] --> Active: first message
    Active --> Active: message / event<br/>(last_access updated)
    Active --> Expired: idle > session_ttl
    Active --> Evicted: cap exceeded
    Expired --> [*]: sweeper
    Evicted --> [*]: on_expire callbacks fire

Expiry also fires on_expire — good place to hook session-close commits to a workspace-git repo.

Relationship to other memory layers

flowchart LR
    STM[short-term<br/>in-memory Vec] -.->|tool call:<br/>memory.remember| LTM[(long-term<br/>SQLite)]
    LTM -.->|vector enabled| VEC[(sqlite-vec)]
    STM -.->|transcripts_dir| TR[(JSONL transcripts)]
    STM -.->|session close| WSG[(workspace-git)]

STM does not auto-promote to LTM. Promotion happens via:

• Explicit memory.remember tool call from the agent
• Dream sweeps (Phase 10.6) that scan recall-event signals and promote hot memories
• Session-close commits to workspace-git if enabled

Gotchas

• Lost turns are gone. Once a turn falls off the sliding window it is not recoverable. If it mattered, save it to LTM before the next turn.
• max_sessions: 0 has no DoS guard. Only do this in single-tenant setups where you control the sender id space.
• last_access updates on any access. That includes heartbeat ticks if they read the session — effectively keeping a session alive past its TTL as long as the agent is alive.

Long-term memory (SQLite)

Durable memory shared by every agent in the process. One SQLite file, multi-tenant via an agent_id column on every row. Survives restarts.

Source: crates/memory/src/long_term.rs.

Storage location

long_term:
  backend: sqlite
  sqlite:
    path: ./data/memory.db

One file for all agents. Per-agent isolation is enforced by WHERE agent_id = ? on every query — not by separate DB files. An idx_memories_agent(agent_id, created_at DESC) index keeps those queries fast.

If you want per-agent file separation, override sqlite.path per agent via an inbound_bindings[] override or a per-agent config directory.

Schema

The runtime creates these tables at boot if they don't exist.

memories — atomic facts

CREATE TABLE memories (
  id            TEXT PRIMARY KEY,  -- UUID
  agent_id      TEXT NOT NULL,
  content       TEXT NOT NULL,
  tags          TEXT DEFAULT '[]', -- JSON array
  concept_tags  TEXT DEFAULT '[]', -- auto-derived (phase 10.7)
  created_at    INTEGER NOT NULL   -- ms since epoch
);
CREATE INDEX idx_memories_agent ON memories(agent_id, created_at DESC);

memories_fts — full-text search (FTS5)

CREATE VIRTUAL TABLE memories_fts USING fts5(
  content,
  id        UNINDEXED,
  agent_id  UNINDEXED
);

Powers the keyword recall mode with BM25 ranking.

interactions — conversation archive

CREATE TABLE interactions (
  id          TEXT PRIMARY KEY,
  session_id  TEXT NOT NULL,
  agent_id    TEXT NOT NULL,
  role        TEXT,
  content     TEXT,
  created_at  INTEGER
);
CREATE INDEX idx_interactions_session ON interactions(session_id, created_at DESC);

reminders — phase 7 heartbeat reminders

CREATE TABLE reminders (
  id            TEXT PRIMARY KEY,
  agent_id      TEXT NOT NULL,
  session_id    TEXT NOT NULL,
  plugin        TEXT,
  recipient     TEXT,
  message       TEXT,
  due_at        INTEGER,
  claimed_at    INTEGER,
  delivered_at  INTEGER,
  created_at    INTEGER
);
CREATE INDEX idx_reminders_due
  ON reminders(agent_id, delivered_at, due_at ASC);

recall_events — signal tracking (phase 10.5)

CREATE TABLE recall_events (
  id         INTEGER PRIMARY KEY AUTOINCREMENT,
  agent_id   TEXT,
  memory_id  TEXT,
  query      TEXT,
  score      REAL,
  ts_ms      INTEGER
);

Every recall() hit records a row. Dream sweeps read this to decide what to promote.

memory_promotions — dreaming ledger (phase 10.6)

CREATE TABLE memory_promotions (
  memory_id    TEXT PRIMARY KEY,
  agent_id     TEXT,
  promoted_at  INTEGER,
  score        REAL,
  phase        TEXT
);

Prevents double-promotion across sweeps.

vec_memories — vector index (phase 5.4, optional)

Created on demand when vector.enabled: true. See Vector search.

What gets written when

Memory tool

Single unified tool with three actions, visible to the LLM as memory:

Results do not include similarity scores — only content and tags. Scores are used internally for dreaming signal tracking but aren't surfaced to the LLM to avoid encouraging score-gaming prompts.

Other memory-related tools:

• forge_memory_checkpoint — snapshot the workspace-git repo (phase 10.9)
• memory_history — git log + optional unified diff (phase 10.9)

Per-agent isolation

flowchart TB
    subgraph PROC[agent process]
        DB[(./data/memory.db<br/>single SQLite file)]
    end
    A1[agent: ana] -->|WHERE agent_id = 'ana'| DB
    A2[agent: kate] -->|WHERE agent_id = 'kate'| DB
    A3[agent: ops] -->|WHERE agent_id = 'ops'| DB

One LongTermMemory instance per process, shared across agents via Arc. The MemoryTool attached to each agent passes ctx.agent_id to every query.

Workspace-git (phase 10.9)

A separate per-agent git repo lives in the agent's workspace directory (not inside the memory DB). When workspace_git.enabled: true, the runtime commits after:

• Dream sweeps (Phase 10.6)
• forge_memory_checkpoint tool calls
• Session close (on_expire)

Good for forensic replay — you can git log to see the memory state at any point. See Soul — MEMORY.md.

Gotchas

• One DB, multi-tenant. A query missing its agent_id filter would leak across agents. All runtime code goes through the LongTermMemory API which injects it automatically.
• Vacuum is manual. SQLite does not auto-compact after deletes. Run VACUUM; periodically (or PRAGMA auto_vacuum=incremental from day one).
• recall_events grows unboundedly. Dream sweeps periodically prune, but a dreaming-disabled agent's table will grow forever. Add a retention job if you run without dreaming.

Vector search

Optional semantic memory via sqlite-vec — a virtual table inside the same SQLite file used for long-term memory. No separate service, no extra process, no migration.

Source: crates/memory/src/vector.rs, crates/memory/src/embedding/.

Turning it on

vector:
  enabled: true
  backend: sqlite-vec
  default_recall_mode: hybrid
  embedding:
    provider: http
    base_url: https://api.openai.com/v1
    model: text-embedding-3-small
    api_key: ${OPENAI_API_KEY}
    dimensions: 1536
    timeout_secs: 30

Dimension must match the model output:

A mismatch aborts startup with an explicit error. If you already have vectors at a different dimension, you must delete the DB (or the vector table) and rebuild the index.

Storage

CREATE VIRTUAL TABLE vec_memories USING vec0(
  memory_id TEXT PRIMARY KEY,
  embedding FLOAT[<dimensions>]
);

The virtual table lives in the same SQLite file as memories. A join on memory_id brings you back the content and tags.

Embedding provider

#![allow(unused)]
fn main() {
trait EmbeddingProvider {
    fn dimension(&self) -> usize;
    async fn embed(&self, texts: &[String]) -> Result<Vec<Vec<f32>>>;
}
}

Phase 5.4 ships one provider: http — any OpenAI-compatible /embeddings endpoint. That covers OpenAI, Gemini (via its API), Ollama, LM Studio, and self-hosted inference.

Local-only providers (fastembed, candle) are intentional follow-ups — the HTTP provider is enough to unblock everything downstream.

Recall modes

Set the default in memory.yaml and override per tool call with the mode argument.

keyword — FTS5 + concept expansion

flowchart LR
    Q[query] --> CT[derive 3 concept tags]
    Q --> M[FTS5 MATCH<br/>query OR tag1 OR tag2 OR tag3]
    CT --> M
    M --> R[rank by BM25]
    R --> RES[top N]

• Fast, no embedding cost
• Misses semantic neighbors that don't share vocabulary
• The extra concept tags are auto-derived from the query and help narrow down concept matches

vector — nearest-neighbor

flowchart LR
    Q[query] --> EMB[embed]
    EMB --> VEC[vec_memories<br/>MATCH k=N*2]
    VEC --> JOIN[join memories<br/>filter by agent_id]
    JOIN --> RES[top N by distance]

• Catches paraphrases and cross-vocabulary matches
• Embedding request on every call — watch costs and latency
• Falls back to keyword on provider error (via hybrid) — not on pure vector mode, where errors surface

hybrid — Reciprocal Rank Fusion

The default recommendation. Runs both keyword and vector, then fuses ranks with the RRF formula 1 / (K + rank + 1) where K = 60:

flowchart LR
    Q[query] --> K[keyword search]
    Q --> V[vector search]
    K --> RRF[RRF fusion<br/>K=60]
    V --> RRF
    RRF --> RES[top N by fused score]

Vector errors degrade gracefully to keyword-only without raising.

Tool interaction

The memory tool takes an optional mode param:

{
  "action": "recall",
  "query": "what's the client's address?",
  "limit": 5,
  "mode": "hybrid"
}

If omitted, default_recall_mode is used.

Cost and latency profile

For high-throughput agents that recall on every turn, start with keyword and upgrade to hybrid only where you see miss rate matter.

Gotchas

• Changing embedding model = full reindex. The dimension check catches the obvious case, but even same-dimension model swaps produce semantically different vectors; the old index becomes stale.
• sqlite3_auto_extension registers once per process. Not a problem in production, but test suites that instantiate multiple SQLite connections across tests may hit edge cases.
• Vector returns distance, not similarity. Lower is closer. Hybrid fusion normalizes across both, so callers don't see this directly unless they bypass the tool.

Manifest (plugin.toml)

Every extension ships a plugin.toml at its root. It declares identity, transport, capabilities, runtime requirements, and any bundled MCP servers. The runtime parses and validates the manifest before spawning anything.

Source: crates/extensions/src/manifest.rs.

Minimal example

[plugin]
id = "weather"
version = "0.1.0"
name = "Weather"
description = "Fetch weather by city name."
min_agent_version = "0.1.0"
priority = 0

[capabilities]
tools = ["get_weather"]
hooks = []

[transport]
type = "stdio"
command = "./weather"
args = []

[requires]
bins = ["curl"]
env = ["WEATHER_API_KEY"]

[context]
passthrough = false

[meta]
author = "you"
license = "MIT OR Apache-2.0"

Sections

[plugin]

Reserved ids: agent, browser, core, email, heartbeat, memory, telegram, whatsapp. The host may register more via register_reserved_ids().

[capabilities]

[capabilities]
tools = ["get_weather", "get_forecast"]
hooks = ["before_message", "after_tool_call"]
channels = []
providers = []

At least one capability list must be non-empty. Names match ^[a-z][a-z0-9_]*$, ≤ 64 chars, no duplicates.

[transport]

One of three forms:

# stdio — spawn a child process
[transport]
type = "stdio"
command = "./my-extension"
args = ["--verbose"]

# nats — talk over a NATS subject prefix
[transport]
type = "nats"
subject_prefix = "ext.myext"

# http — call over HTTP
[transport]
type = "http"
url = "https://localhost:8080"

Validation: command, subject_prefix, url non-empty; url must be http(s)://.

[requires]

[requires]
bins = ["ffmpeg", "imagemagick"]
env  = ["OPENAI_API_KEY"]

Declarative preconditions used for gating: when the runtime discovers the extension, it calls Requires::missing(). If any bins is not on $PATH or any env is unset, the extension is skipped (warn, not fail) and its tools are not registered.

See Stdio runtime — Gating.

[context]

[context]
passthrough = true

When true, every tool call sent to this extension has _meta = { agent_id, session_id } injected into the JSON args. Lets the extension tell calls apart per-agent without the runtime having to encode the split into every tool signature.

[mcp_servers] (phase 12.7)

Inline MCP server declarations bundled with the extension:

[mcp_servers.gmail]
type = "stdio"
command = "./gmail-mcp"
args = []

[mcp_servers.calendar]
type = "streamable_http"
url = "https://mcp.example.com/calendar"

Each server name must match ^[a-z][a-z0-9_-]*$, ≤ 32 chars. Alternatively, drop a sidecar .mcp.json next to plugin.toml if the manifest has no [mcp_servers] section.

Validation at a glance

flowchart TD
    READ[read plugin.toml] --> PARSE[parse TOML]
    PARSE --> ID{id valid?<br/>regex + length<br/>+ not reserved}
    ID --> VER{version<br/>valid semver?}
    VER --> MIN{min_agent_version<br/>satisfied?}
    MIN --> CAPS{at least one<br/>capability declared?}
    CAPS --> NAMES{capability names<br/>valid + unique?}
    NAMES --> TRANS{transport<br/>non-empty +<br/>http scheme valid?}
    TRANS --> MCP{mcp_server names<br/>valid?}
    MCP --> OK([Manifest accepted])
    ID --> FAIL([Diagnostic: Error])
    VER --> FAIL
    MIN --> FAIL
    CAPS --> FAIL
    NAMES --> FAIL
    TRANS --> FAIL
    MCP --> FAIL

Any failure produces a DiagnosticLevel::Error in the discovery report — the candidate is dropped but scanning continues so an operator sees every broken manifest at once.

Agent-version gating

[plugin]
min_agent_version = "0.2.0"

On load the runtime compares against the agent build version. A mismatch logs a diagnostic and drops the candidate. Useful for shipping a manifest that relies on a newer host API without crash-looping older deployments. The host can override the reported version for tests via set_agent_version().

Next

• Discovery and NATS runtime — how the manifest drives spawn
• CLI — agent ext validate <path> checks a manifest without touching the registry
• Templates — prebuilt skeletons to copy

Stdio runtime + Discovery

The stdio runtime is the default way extensions run: a child process speaking line-delimited JSON-RPC over stdin/stdout. This page covers how the runtime discovers, spawns, supervises, and registers tools from a stdio extension.

Source: crates/extensions/src/discovery.rs, crates/extensions/src/runtime/stdio.rs.

Discovery

# config/extensions.yaml
extensions:
  enabled: true
  search_paths: [./extensions]
  ignore_dirs: [node_modules, .git, target]
  disabled: []
  allowlist: []            # empty = all allowed
  max_depth: 4
  follow_links: false
  watch:
    enabled: false
    debounce_ms: 500

ExtensionDiscovery walks each search path, looking for plugin.toml files:

flowchart TD
    ROOT[search_paths root] --> WALK[walkdir max_depth]
    WALK --> IGNORE{dir in<br/>ignore_dirs?}
    IGNORE -->|yes| SKIP[skip]
    IGNORE -->|no| FIND[find plugin.toml]
    FIND --> PARSE[parse + validate manifest]
    PARSE --> SIDE[sidecar .mcp.json if manifest<br/>has no mcp_servers]
    SIDE --> PRUNE[prune nested candidates]
    PRUNE --> DEDUP[dedupe by id]
    DEDUP --> DIS[apply disabled filter]
    DIS --> ALLOW[apply allowlist filter]
    ALLOW --> SORT[sort by root_index, id]
    SORT --> CANDS[DiscoveryReport<br/>candidates + diagnostics]

Prune-nested removes any candidate whose root_dir is a strict descendant of another — avoids registering an extension twice if it happens to live inside another extension's tree. Algorithm is O(N × depth).

follow_links = false is the default (monorepo-safe). When enabled, symlink escapes out of the root raise DiagnosticLevel::Error.

Gating

Before spawn, Requires::missing() runs:

flowchart LR
    CAND[candidate] --> REQ[requires.bins<br/>+ requires.env]
    REQ --> BINS{all on $PATH?}
    BINS -->|no| SKIP1[warn + skip]
    BINS -->|yes| ENV{all env set?}
    ENV -->|no| SKIP2[warn + skip]
    ENV -->|yes| SPAWN[spawn runtime]

A skipped extension does not register any tools. The warn log names exactly which bin or env var was missing.

Spawn model

sequenceDiagram
    participant H as Host (agent)
    participant S as StdioRuntime
    participant C as Child process

    H->>S: spawn(manifest, cwd)
    S->>C: tokio::process::Command
    S->>C: {"jsonrpc":"2.0","method":"initialize",<br/>"params":{"agent_version","extension_id"},"id":0}
    C-->>S: {"result":{"server_version","tools":[...],"hooks":[...]}}
    S-->>H: HandshakeInfo
    H->>H: register each tool as ExtensionTool
    H->>H: register each hook as ExtensionHook

• Child is spawned with the extension's directory as cwd
• stdin + stdout is the RPC channel (line-delimited JSON)
• stderr is routed to the agent's tracing output
• Handshake timeout: default 10 s

Tool descriptors

{
  "name": "get_weather",
  "description": "Look up weather by city.",
  "input_schema": { "type": "object", "properties": { "city": { "type": "string" } }, "required": ["city"] }
}

The host wraps each descriptor in an ExtensionTool:

• Registered name: ext_{plugin_id}_{tool_name} (truncated with hash suffix if it exceeds 64 chars)
• Description prefixed with [ext:{id}] so the LLM knows the origin
• input_schema copied to the registered tool

Context passthrough

If the manifest sets context.passthrough = true, every call() injects:

{ "_meta": { "agent_id": "...", "session_id": "..." }, ...user_args }

The extension can decide how to split state per agent or session.

Env injection

The host passes through most env vars to the child, but blocks secret-like names via substring/suffix rules:

• Suffixes: _TOKEN, _KEY, _SECRET, _PASSWORD, _CREDENTIAL, _PAT, _AUTH, _APIKEY, _BEARER, _SESSION
• Substrings: PASSWORD, SECRET, CREDENTIAL, PRIVATE_KEY

Extensions that need a secret should read it from a file path the host passes by argument, or have the secret baked into their own requires.env entry (which the operator whitelists consciously).

Supervision

stateDiagram-v2
    [*] --> Spawning
    Spawning --> Ready: handshake ok
    Ready --> Restarting: child crash
    Restarting --> Ready: handshake ok again
    Restarting --> Failed: max attempts<br/>in restart_window
    Ready --> Shutdown: graceful signal
    Failed --> Shutdown
    Shutdown --> [*]

Supervisor policy:

• Max restart attempts within a sliding restart_window
• Exponential backoff base_backoff → max_backoff
• Each transport is wrapped in a CircuitBreaker named ext:stdio:{id} so hung children don't freeze the agent loop

Graceful shutdown sends an empty message, waits shutdown_grace (default 3 s), then kills the child.

Watcher (phase 11.2 follow-up)

With extensions.watch.enabled: true the runtime watches search_paths for changes to any plugin.toml. Change-set is debounced (debounce_ms) and compared by SHA-256 of the file to squash spurious writes.

On change the runtime logs — it does not auto-reload. The operator restarts the agent to pick up the new manifest. Hot reload is a future phase.

Gotchas

• Blocked env vars surprise extensions. If an extension expected OPENAI_API_KEY to come through and it wasn't declared in requires.env, the name-based block may silently strip it. Declare the env you need — that whitelists it.
• follow_links: true + symlinked monorepo layouts can cause discovery to traverse out of the search root. Keep follow_links: false unless you know the layout is bounded.
• Children crashing during handshake. You get a single DiagnosticLevel::Error per candidate, not a retry loop. Fix the binary, restart the host.

NATS runtime

For extensions that run out-of-process and manage their own lifecycle — a long-lived service on another machine, a container in an orchestrator, an operator-maintained daemon. The agent talks to them over NATS RPC instead of stdin/stdout.

Source: crates/extensions/src/runtime/nats.rs.

When to pick NATS over stdio

Stdio is the default. Reach for NATS when the extension's failure domain must be separated from the agent's.

Manifest

[plugin]
id = "heavy-compute"
version = "0.3.0"

[capabilities]
tools = ["long_running_job"]

[transport]
type = "nats"
subject_prefix = "ext.heavy-compute"

Wire shape

Single request/reply subject:

{subject_prefix}.{extension_id}.rpc

sequenceDiagram
    participant A as Agent
    participant N as NATS
    participant E as Extension service

    A->>N: publish ext.heavy-compute.rpc<br/>{method:"initialize", ...}
    N->>E: deliver
    E->>N: reply HandshakeInfo
    N-->>A: tools + hooks
    A->>A: register ExtensionTool per tool
    Note over A,E: steady state
    loop tool call
        A->>N: {method:"tools/long_running_job", params, id}
        N->>E: deliver
        E-->>N: result
        N-->>A: reply
    end

The JSON-RPC shape is identical to stdio — only the transport changes. Extensions don't need to know which form the host chose.

Liveness

Instead of supervising a child process, the NATS runtime uses heartbeats:

A failed extension logs a warn and is marked unavailable. Tools stay registered in the registry but calls error out immediately. When the extension starts beaconing again, it's automatically marked available.

Circuit breaker

Same pattern as stdio: one CircuitBreaker per extension, ext:nats:{id}, wrapping every RPC. Prevents a flapping extension from piling up outstanding calls against it.

Deployment recipes

Docker compose side service

services:
  agent:
    image: nexo-rs:latest
    depends_on: [nats, heavy-compute]
  nats:
    image: nats:2.10-alpine
  heavy-compute:
    image: my-ext:0.3.0
    command: ["--nats-url", "nats://nats:4222",
              "--subject-prefix", "ext.heavy-compute"]

Kubernetes

Run the extension as its own Deployment with its own resource limits, rollouts, and observability. Share the NATS cluster via a Service. Scale extensions independently of agents.

Gotchas

• subject_prefix collisions. Two extensions with the same prefix will step on each other. Enforce uniqueness in your ops convention.
• Latency. NATS over LAN is sub-millisecond, but any network hop is orders of magnitude slower than stdio's pipe. Don't pick NATS for a 1 kHz tool call pattern.
• Auth on the broker. NATS auth applies to extensions too — if you turn on NKey mTLS, every extension service must be enrolled.

CLI (agent ext)

Operator-facing commands for discovering, installing, validating, and toggling extensions. Every subcommand accepts --json for scripting.

Source: crates/extensions/src/cli/.

Subcommands

agent ext list                           [--json]
agent ext info <id>                      [--json]
agent ext enable <id>
agent ext disable <id>
agent ext validate <path>
agent ext doctor                         [--runtime] [--json]
agent ext install <path>                 [--update] [--enable] [--dry-run] [--link] [--json]
agent ext uninstall <id> --yes           [--json]

list — discovered extensions

Walks the configured search_paths, prints each candidate, its transport, and its enabled/disabled state.

info <id> — manifest + status

Prints the full parsed manifest, the runtime state if the agent is currently running, and any diagnostics attached to the candidate.

enable / disable — toggle in extensions.yaml

Rewrites the disabled list in config/extensions.yaml:

extensions:
  disabled: [weather]

No runtime side effect; operator must restart the agent to apply.

validate <path> — manifest check without registering

Parses and validates a plugin.toml at <path>. Good for CI checks on an extension's manifest before shipping.

doctor — preflight checks

Runs the same Requires::missing() logic as discovery, plus transport-specific checks:

flowchart TB
    START([agent ext doctor]) --> DISC[discover candidates]
    DISC --> REQ[check requires.bins + requires.env]
    REQ --> RUNT{--runtime?}
    RUNT -->|yes| SPAWN[spawn each stdio extension<br/>and handshake]
    RUNT -->|no| DONE([report table])
    SPAWN --> DONE

--runtime actually spawns each stdio extension and runs the handshake — useful to catch a broken binary before production boot.

install <path> — copy or symlink

Adds an extension to the active search_paths:

agent ext install ./extensions/weather
agent ext install /abs/path/to/my-ext --link --enable

• --update replaces an existing extension with the same id
• --enable adds it to extensions.yaml enabled (default: disabled until you enable)
• --dry-run prints what would happen without writing
• --link creates a symlink instead of copying — requires an absolute source path. Good for dev loops.

uninstall <id> --yes

Removes the extension's directory from the active search path (or the symlink, in --link installs). --yes is mandatory — no accidental destruction.

Exit codes

Non-zero codes are stable for scripting.

JSON mode

Every subcommand that produces human output also supports --json for machine consumption. Fields are stable per code-phase; schema is not officially frozen yet — pin to a specific agent version in CI.

Common ops flows

Ship an extension to staging

agent ext validate ./my-ext/plugin.toml
agent ext install ./my-ext --link --enable
agent ext doctor --runtime

Disable a flapping extension without redeploying

agent ext disable weather   # writes to extensions.yaml
systemctl reload agent       # or restart, depending on deployment

CI gate

# .github/workflows/extension.yml
- run: cargo build --release
- run: agent ext validate ./plugin.toml

Templates

The repo ships two extension templates as starting points. Copy one, rename it, fill in the tools, done.

Location: extensions/template-rust/ and extensions/template-python/.

What's shared

Both templates follow the same wire protocol and directory shape:

<your-ext>/
├── plugin.toml        # manifest (see ./manifest.md)
├── README.md          # what the extension does
├── <binary or script> # stdio-RPC entry point
└── ...                # build files specific to the language

The agent talks to both in the same JSON-RPC 2.0 shape:

• initialize — handshake; returns {server_version, tools, hooks}
• tools/<name> — tool invocation; returns the tool's result
• hooks/<name> — hook invocation (when any hook is declared)

Line-delimited JSON over stdin/stdout. stderr is forwarded to the agent's tracing output — that's your debug log.

Rust template (extensions/template-rust/)

Standalone Cargo project outside the agent workspace — its own Cargo.toml, own Cargo.lock, own target/. Keeps your extension's deps independent of the agent's.

template-rust/
├── Cargo.toml
├── Cargo.lock
├── plugin.toml
├── README.md
├── src/
│   └── main.rs        # JSON-RPC loop
└── target/            # (gitignore)

src/main.rs implements:

#![allow(unused)]
fn main() {
// pseudocode
loop {
    let line = read_line_from_stdin();
    let req: JsonRpcRequest = parse(line);
    let result = match req.method.as_str() {
        "initialize" => handshake_info(),
        "tools/ping" => ping(req.params),
        "tools/add"  => add(req.params),
        "hooks/before_message" => pass(),
        _ => method_not_found(),
    };
    write_line_to_stdout(json!({ "jsonrpc": "2.0", "id": req.id, "result": result }));
}
}

Build with cargo build --release; the release binary at ./target/release/template-rust is what plugin.toml::transport.command points at.

Python template (extensions/template-python/)

template-python/
├── plugin.toml
├── main.py       # #!/usr/bin/env python3
└── README.md

stdlib only (no pip install). Same JSON-RPC loop over stdin/stdout. Logs to stderr via print(..., file=sys.stderr).

Good for quick extensions where starting a Python interpreter per tool call is acceptable (batch workloads, cron-ish tasks, one-off scripting).

Promoting a template to your own extension

flowchart LR
    COPY[copy template-rust<br/>to my-extension] --> EDIT[edit plugin.toml<br/>id, version, tools]
    EDIT --> CODE[implement tools/...]
    CODE --> BUILD[cargo build --release]
    BUILD --> VAL[agent ext validate<br/>./my-extension/plugin.toml]
    VAL --> INSTALL[agent ext install<br/>./my-extension --link --enable]
    INSTALL --> DOCTOR[agent ext doctor<br/>--runtime]

Conventions in the shipped templates

• plugin.toml declares the minimum required capabilities — no phantom hooks or tools
• requires.bins / requires.env left empty; add your own
• [context] passthrough = false — opt in explicitly when you need per-agent / per-session state
• License left blank — pick one and add it to [meta]

Gotchas

• Rust template builds in its own workspace. Don't cargo add from the repo root — that edits the agent workspace, not the extension.
• Python template spawns a new interpreter per extension, not per tool call. Stdin/stdout stay open for the life of the process. Don't exit after one tool call.
• JSON-RPC ids must echo back. If your handler drops the id field, the agent can't correlate the reply.

1Password extension

A bundled stdio extension that wraps the op CLI with a service-account token. Read-only: it never creates or edits secrets. Two main use cases:

• Look up a secret you don't already have in env (read_secret).
• Use a secret in a command without ever exposing it to the agent (inject_template).

Source: extensions/onepassword/. Skill prompt: skills/onepassword/SKILL.md.

Tools

read_secret

{ "action": "read_secret", "reference": "op://Prod/Stripe/api_key" }

Default response (reveal off):

{
  "ok": true,
  "reference": "op://Prod/Stripe/api_key",
  "vault": "Prod", "item": "Stripe", "field": "api_key",
  "length": 26,
  "fingerprint_sha256_prefix": "3f9a7c2e1b48d5a0",
  "reveal": false
}

With OP_ALLOW_REVEAL=true|1|yes set on the agent process, the response also contains { "value": "...", "reveal": true }.

inject_template

Resolves {{ op://Vault/Item/field }} placeholders via op inject. Two execution paths:

Template-only

{ "action": "inject_template",
  "template": "Authorization: Bearer {{ op://Prod/API/token }}\n" }

• Reveal off → { length, fingerprint_sha256_prefix, reveal: false }
• Reveal on → { rendered: "Authorization: Bearer abc…", reveal: true }

Exec (piped to a command)

{ "action": "inject_template",
  "template": "Bearer {{ op://Prod/API/token }}",
  "command": "curl",
  "args": ["-H", "@-", "https://api.example.com/me"] }

• command must be in OP_INJECT_COMMAND_ALLOWLIST (comma-separated). Default empty → exec mode disabled.
• Rendered template is never returned to the LLM. Only the downstream command's exit_code, stdout (capped at max_stdout_bytes, default 4096, max 16384), and stderr.
• Both stdout and stderr are redacted before being returned — Bearer JWT, sk-…, sk-ant-…, AKIA…, and 32+ char hex tokens are replaced with [REDACTED:<label>].

Dry run

{ "action": "inject_template",
  "template": "{{ op://A/B/c }} {{ op://X/Y/z }}",
  "dry_run": true }

Validates each op:// reference's shape without resolving values. Returns references_validated.

Configuration

Environment variables consumed by the extension: