Configure Hermes Agent with Local LiteLLM Proxy

Set up Hermes Agent (CLI and Desktop) to use a locally hosted LiteLLM proxy as its custom LLM provider. The proxy is backed by Vertex AI Claude models (no auth, http://127.0.0.1:4000).

Prerequisites

• Hermes Agent installed (~/.hermes/)
• LiteLLM proxy running locally (see skill litellm-vertex-ai-proxy)
• The proxy must be accessible at http://127.0.0.1:4000 (default port)
• Python3 with pyyaml installed (used by scripts/restore.sh)

Model Selection Architecture

Hermes stores the default model in three locations that must stay in sync. There is no separate fast_model concept (unlike Goose).

Field	Location	Purpose
model.default	Top-level in config.yaml	Active default model
providers.<name>.default_model	Provider entry	Provider-level default
providers.<name>.model	Provider entry	Currently selected model

The restore.sh script updates all three atomically.

Note: hermes model may not persist changes to config.yaml. Use scripts/restore.sh or direct file editing for reliable updates.

Reference Configuration

File: ~/.hermes/config.yaml (relevant sections only)

⚠️ Use this exact schema. Do NOT write from memory or improvise field names. Copy this template and substitute only the marked placeholders.

model:
  base_url: http://127.0.0.1:4000/v1
  default: claude-opus-4-6              # default model for conversation
  provider: litellm-vertex-ai
  key_env: HERMES_CUSTOM_LITELLM_VERTEX_AI_API_KEY
providers:
  litellm-vertex-ai:
    api: http://127.0.0.1:4000/v1
    name: LiteLLM Vertex AI
    discover_models: true
    default_model: claude-opus-4-6      # must match model.default
    base_url: http://127.0.0.1:4000/v1
    model: claude-opus-4-6              # must match model.default
    models:
      claude-opus-4-6: {}
      claude-sonnet-4-6: {}
      claude-haiku-4-5: {}
    key_env: HERMES_CUSTOM_LITELLM_VERTEX_AI_API_KEY

Key Fields Explained

Field	Value	Why
model.provider	litellm-vertex-ai	Named provider identifier (not custom)
model.base_url	http://127.0.0.1:4000/v1	Local LiteLLM proxy with /v1 path
model.default	claude-opus-4-6	Default conversation model
key_env	HERMES_CUSTOM_...	Auto-generated by Hermes; no actual key needed (proxy has no auth)
discover_models	true	Hermes queries proxy for available models at runtime
models	hash with {} values	Model catalog for the provider

---

Workflow

Step 1 — Pre-flight Check

Run the verification script to see current state:

bash ~/.agents/skills/hermes-litellm-provider/scripts/verify.sh

If the proxy is not running, start it:

systemctl --user start litellm-proxy

Step 2 — Create or Restore the Provider

Option A — Script (recommended):

bash ~/.agents/skills/hermes-litellm-provider/scripts/restore.sh

Optionally override the default model:

bash ~/.agents/skills/hermes-litellm-provider/scripts/restore.sh \
  --default-model claude-sonnet-4-6

The script:

• Discovers models from the live proxy
• Backs up config.yaml before modifying
• Surgically updates only model: and providers.litellm-vertex-ai:
• Preserves all other settings (headroom-proxy, agent, toolsets, etc.)

Option B — Interactive wizard:

hermes model

1. Select “Custom endpoint (enter URL manually)”
2. API base URL → http://127.0.0.1:4000/v1
3. API key → press Enter to skip (no auth needed)
4. API mode → accept the default (auto-detect)
5. Pick a default model from the discovered list

Note: hermes model may not persist changes. Verify with scripts/verify.sh after using the wizard.

Step 3 — Verify Configuration

bash ~/.agents/skills/hermes-litellm-provider/scripts/verify.sh

All 11 checks should pass.

Step 4 — Test from CLI

hermes chat --oneshot "Say hello in one sentence"

Step 5 — Test from Desktop (Optional)

1. Launch Hermes Desktop
2. Type /model in a chat session
3. Confirm all proxy models are listed and selectable
4. Send a test message

---

Recovery Procedure

If the LiteLLM provider is lost (e.g., after a config reset):

bash ~/.agents/skills/hermes-litellm-provider/scripts/restore.sh
bash ~/.agents/skills/hermes-litellm-provider/scripts/verify.sh

---

Specification

ID	Capability	Verifiable By
S1	LiteLLM proxy is running with all endpoints healthy	scripts/verify.sh checks S1a, S1b
S2	Hermes config.yaml has correct model section (provider, base_url, default)	scripts/verify.sh checks S2a–S2d
S3	Provider entry exists with correct base_url, discover_models, default_model, and consistency with model.default	scripts/verify.sh checks S3a–S3d
S4	Config models match live LiteLLM models	scripts/verify.sh check S4

Tests

Test	Spec	Command	Expected Result
T1	S1	bash scripts/verify.sh 2>&1 \| grep S1	Both S1a and S1b show ✅
T2	S2	bash scripts/verify.sh 2>&1 \| grep S2	All S2a–S2d show ✅
T3	S3	bash scripts/verify.sh 2>&1 \| grep S3	All S3a–S3d show ✅
T4	S4	bash scripts/verify.sh 2>&1 \| grep S4	S4 shows ✅
T5	S1–S4	bash scripts/verify.sh	Exit code 0, all 11 checks pass
T6	S2–S4	Strip provider from config → bash scripts/restore.sh → bash scripts/verify.sh	Restore + verify both succeed; other config preserved

Troubleshooting

Symptom	Cause	Fix
“Connection refused”	LiteLLM proxy not running	systemctl --user start litellm-proxy
Only 1 model in picker	Missing discover_models: true	Run scripts/restore.sh
Model resets after restart	hermes model not persisting	Edit config.yaml directly or use scripts/restore.sh
Hermes ignores provider	model.provider wrong	Verify with scripts/verify.sh
Desktop shows old provider	Cached state	Restart Hermes Desktop

Changelog

See CHANGELOG.md for version history.