Self-Hosted AI Agent Guide for Canada

Why self-host an AI agent in Canada?

Canadian businesses face specific considerations when adopting AI automation. Data residency, PIPEDA obligations, and client confidentiality mean that sending every prompt and document to a US-based API may not be the right default. Self-hosting gives you control over where your data lives, who can access it, and how agent workflows are reviewed.

Data residency

Under Canadian privacy law, organizations are responsible for personal information regardless of where it is processed. For many regulated industries (legal, healthcare, financial services), keeping data on Canadian soil simplifies compliance, audit, and client assurance. Self-hosting in a Canadian data center means prompts, documents, and agent memory stay within Canadian jurisdiction.

PIPEDA accountability

PIPEDA's accountability principle (Principle 1) means your organization remains responsible for personal information even when using third-party services. When you self-host, you control the infrastructure, the access logs, and the retention policy — making it easier to demonstrate compliance. For more detail, see our PIPEDA and AI automation guide.

Privacy and control

Self-hosting means your agent workflows, tool configurations, memory store, and secrets management all run on infrastructure you own or rent directly. You decide how logs are stored, how long data is retained, and which model providers (if any) receive your prompts. This is especially important for businesses handling client documents, financial records, or proprietary process information.

Cost predictability

VPS hosting has a fixed monthly cost. If your team runs many agent workflows — daily reports, recurring research, customer follow-up drafts — managed API costs can grow unpredictably. A self-hosted setup with a fixed VPS bill plus optional model API usage gives you a predictable ceiling.

Educational content only, not legal advice. Privacy, PIPEDA, provincial, and sector-specific obligations should be reviewed with qualified counsel.

Hosting options for a self-hosted AI agent in Canada

On-premises

Running the agent on hardware you own — a Mac mini in the office, a Linux workstation, or a small server in a locked room. This gives you the most control but requires managing power, network, and hardware reliability yourself. Best for teams that already maintain local servers and want zero third-party infrastructure involvement.

• Zero data leaves your physical premises
• No monthly hosting bill beyond power and internet
• Requires reliable uptime — consider a UPS and backup internet
• Hardware maintenance is your responsibility

VPS in a Canadian data center

A virtual private server (VPS) in a Canadian region is the sweet spot for most teams. You get a dedicated Linux instance in a professionally managed data center, root access, and a fixed monthly price. Three major options with Canadian regions:

What specs do you need?

Hermes Agent itself is lightweight — it orchestrates tools and calls models but does not run the models locally by default. A 2 vCPU / 4 GB RAM VPS (around $24-48/mo) comfortably handles agent orchestration, persistent memory, cron jobs, and background workflows for a small team. If you plan to run local models with llama.cpp, budget for 8+ GB RAM and a GPU if available.

Setting up Hermes Agent on a VPS

This section walks through deploying Hermes Agent on a Linux VPS in a Canadian data center. You'll have a running agent with persistent memory, cron scheduling, and secure secret management.

Option A: Direct install (recommended for simplicity)

Hermes Agent installs via pip into a Python virtual environment. This is the fastest path to a working agent and the easiest to debug.

Step 1: Provision your VPS and secure it

After creating your VPS (choose Ubuntu 24.04 LTS in your Canadian region of choice), SSH in and run basic hardening:

# Update system
sudo apt update && sudo apt upgrade -y

# Create a non-root user for the agent
sudo adduser hermes
sudo usermod -aG sudo hermes

# Configure firewall — allow only SSH and the agent dashboard port
sudo ufw allow OpenSSH
sudo ufw allow 9119/tcp comment 'Hermes dashboard'
sudo ufw enable

# Install dependencies
sudo apt install -y python3 python3-pip python3-venv git curl

Step 2: Install Hermes Agent

# Clone and set up as the hermes user
su - hermes
git clone https://github.com/nousresearch/hermes-agent.git ~/.hermes/hermes-agent
cd ~/.hermes/hermes-agent
python3 -m venv venv
source venv/bin/activate
pip install -e .

# Run the guided setup
hermes setup

Step 3: Configure secrets securely

Never hardcode API keys in config files. Use environment variables or a secrets manager:

# Create a .env file with restricted permissions
touch ~/.hermes/.env
chmod 600 ~/.hermes/.env
cat > ~/.hermes/.env << 'EOF'
OPENROUTER_API_KEY=sk-or-v1-...
# Add other provider keys as needed
ANTHROPIC_API_KEY=sk-ant-...
OPENAI_API_KEY=sk-...
EOF

For teams using privacy-safe workflows, consider using the 1Password CLI to inject secrets at runtime instead of storing them in a .env file:

# Install 1Password CLI and sign in
# Then reference secrets in your agent config with op:// references
hermes config set openrouter.api_key "op://vault/item/field"

Step 4: Verify the install

# Run a quick health check
hermes status

# Start the gateway (background, with auto-restart)
hermes gateway start --port 9119

# Test with a simple prompt
echo "Say hello from Canada" | hermes run

Option B: Docker (portable and isolated)

If your team standardizes on Docker, this approach keeps Hermes Agent isolated from the host system and makes it easy to move between environments.

# docker-compose.yml
version: '3.8'
services:
  hermes-agent:
    image: ghcr.io/nousresearch/hermes-agent:latest
    container_name: hermes-agent
    restart: unless-stopped
    ports:
      - "127.0.0.1:9119:9119"  # dashboard (localhost only)
    volumes:
      - ./hermes-data:/home/hermes/.hermes  # persistent config and memory
      - ./skills:/home/hermes/.hermes/skills
    environment:
      - HERMES_CONFIG_PATH=/home/hermes/.hermes/config.yaml
      - OPENROUTER_API_KEY=${OPENROUTER_API_KEY}
      - ANTHROPIC_API_KEY=${ANTHROPIC_API_KEY}
    env_file:
      - .env  # secrets, gitignored
    command: ["hermes", "gateway", "start", "--port", "9119"]

Store your secrets in a .env file (gitignored) or use Docker secrets for production. Never commit .env to version control.

Running in production: systemd service

For a long-running agent on a VPS, a systemd service ensures the agent starts on boot and restarts if it crashes. This is the recommended production setup for Linux VPS deployments.

# /etc/systemd/system/hermes-agent.service
[Unit]
Description=Hermes Agent Gateway
After=network-online.target
Wants=network-online.target

[Service]
Type=simple
User=hermes
WorkingDirectory=/home/hermes
Environment="PATH=/home/hermes/.hermes/hermes-agent/venv/bin:/usr/local/bin:/usr/bin:/bin"
EnvironmentFile=/home/hermes/.hermes/.env
ExecStart=/home/hermes/.hermes/hermes-agent/venv/bin/hermes gateway start --port 9119
Restart=on-failure
RestartSec=10
StandardOutput=journal
StandardError=journal
SyslogIdentifier=hermes-agent

# Security hardening
NoNewPrivileges=yes
PrivateTmp=yes
ProtectSystem=strict
ProtectHome=read-only
ReadWritePaths=/home/hermes/.hermes
ReadOnlyPaths=/home/hermes/.hermes/hermes-agent

[Install]
WantedBy=multi-user.target

Enable and start the service:

sudo systemctl daemon-reload
sudo systemctl enable hermes-agent
sudo systemctl start hermes-agent
sudo systemctl status hermes-agent   # verify it's running

# View logs
sudo journalctl -u hermes-agent -f

TLS with a reverse proxy

If you need the dashboard accessible beyond localhost (e.g., for remote monitoring), put an Nginx reverse proxy with Let's Encrypt in front of it:

# Install certbot and nginx
sudo apt install -y nginx certbot python3-certbot-nginx

# Create nginx config for the dashboard subdomain
sudo tee /etc/nginx/sites-available/hermes-dashboard << 'EOF'
server {
    listen 443 ssl;
    server_name agent-dash.yourdomain.ca;

    ssl_certificate     /etc/letsencrypt/live/agent-dash.yourdomain.ca/fullchain.pem;
    ssl_certificate_key /etc/letsencrypt/live/agent-dash.yourdomain.ca/privkey.pem;

    location / {
        proxy_pass http://127.0.0.1:9119;
        proxy_http_version 1.1;
        proxy_set_header Upgrade $http_upgrade;
        proxy_set_header Connection "upgrade";
        proxy_set_header Host $host;
        proxy_set_header X-Real-IP $remote_addr;
        proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
        proxy_set_header X-Forwarded-Proto $scheme;
    }
}
EOF

sudo ln -s /etc/nginx/sites-available/hermes-dashboard /etc/nginx/sites-enabled/
sudo nginx -t && sudo systemctl reload nginx
sudo certbot --nginx -d agent-dash.yourdomain.ca

Security best practices for self-hosted AI agents

Secrets management

Your agent needs API keys for model providers (OpenRouter, Anthropic, OpenAI) and potentially OAuth tokens for integrations (Google Workspace, GitHub). These are high-value secrets — treat them like database passwords.

• Never hardcode secrets in config files or version control
• .env files: Keep them outside the repo, chmod 600, and add to .gitignore
• 1Password CLI: Use op:// references so secrets never touch disk in plaintext
• macOS Keychain: On Mac, store encryption passphrases in the Keychain and reference them from scripts
• Docker secrets: In Swarm or Compose with secrets support, use /run/secrets/ mounts
• Environment variables: Pass secrets at process start — they're in memory, not on disk

Firewall rules

Your VPS firewall should follow the principle of least privilege:

• SSH (port 22): allow from your office IP or a VPN range only
• Dashboard (port 9119): bind to 127.0.0.1 only unless behind a reverse proxy with TLS and authentication
• All other inbound ports: deny by default
• Outbound: allow to the internet for model API calls, but log for audit

Agent permissions

Hermes Agent uses a tools and toolsets system to control what the agent can do. When deploying in production:

• Run the agent as a dedicated hermes user, not root
• Restrict filesystem access to the agent's working directory
• Use enabled_toolsets in cron jobs to limit which tools each automated workflow can access
• Review agent memory periodically — it persists across sessions and may contain sensitive summaries
• Consider encrypting the persistent memory store at rest

For a deeper dive, see our guide on privacy-safe AI workflows for Canadian SMBs.

Self-hosted vs managed API: tradeoffs

Where does your data actually go?

Even with a self-hosted agent, when you use a model provider like OpenRouter or Anthropic, your prompts and the model's responses are processed on their servers. Self-hosting the agent keeps your workflow logic, tool configurations, memory, and orchestration in Canada. Whether prompts leave Canada depends on which model provider you choose:

• OpenRouter: Routes to various providers (Anthropic, OpenAI, Google, Meta, etc.). Data handling policies vary. Most providers process data in the US. OpenRouter's own infrastructure is US-based.
• Anthropic API (direct): US-based processing. Anthropic does not train on API data by default, but data transits through US servers.
• OpenAI API (direct): US-based processing. OpenAI offers a data processing agreement for API customers, but data is processed in the US.
• Self-hosted model (llama.cpp): Run models like Llama, Mistral, or Qwen entirely on your VPS. Zero data leaves your server. This requires more RAM (16+ GB for 7B models) and CPU/GPU resources but gives you complete data sovereignty.

For Canadian teams where prompt privacy is critical, running a local model with llama.cpp inside your Canadian VPS is the strongest option. See our Canadian data residency guide for a detailed breakdown.

Latency

VPS in Canadian regions (Toronto, Montréal) typically have 15-30ms latency to major model APIs on the US east coast. A self-hosted agent adds negligible overhead — the bottleneck is almost always the model inference time, not agent orchestration.

Maintenance

• Weekly: check systemd service health, review disk usage
• Monthly: apply OS security patches (sudo apt update && sudo apt upgrade)
• Quarterly: review agent memory, rotate API keys, review cron job outputs
• Per Hermes release: review changelog, test upgrade in staging before applying to production

PIPEDA compliance checklist for self-hosted AI agents

This checklist is a starting point for Canadian teams evaluating self-hosted agent deployments. It is not a substitute for legal review, but it covers the practical steps that align with PIPEDA's ten principles.

For more, read our full PIPEDA and AI automation overview and the privacy-safe workflow design guide.

Canadian-specific considerations

Provincial privacy laws

In addition to PIPEDA, several provinces have substantially similar privacy legislation that applies to private-sector organizations within the province:

• Québec: Law 25 (effective 2023-2024) adds mandatory breach reporting, privacy impact assessments, and stronger consent requirements. Québec-based organizations may need a PIA before deploying AI automation that processes personal information.
• Alberta: PIPA governs private-sector privacy. Similar to PIPEDA with some additional requirements for notification.
• British Columbia: PIPA applies to private-sector organizations in BC.
• Ontario: PHIPA for health information custodians applies regardless of PIPEDA for health data.

Cross-border data flows

If you self-host in Canada but use a US-based model API (OpenRouter, Anthropic, OpenAI), prompts containing personal information cross the border. Under PIPEDA, you remain accountable. Assess whether the model API provider offers adequate safeguards. For sensitive data, consider running a local model on your Canadian VPS so prompts never leave the country.

Industry-specific regulations

Certain sectors have additional requirements beyond PIPEDA:

• Healthcare: Provincial health privacy laws (e.g., Ontario's PHIPA, Alberta's HIA) may restrict where health information can be processed. Consult provincial requirements before using any AI tool with health data.
• Financial services: OSFI guidelines and provincial regulators may impose data residency and outsourcing requirements. Review your regulator's guidance on cloud and AI.
• Legal profession: Law society rules on client confidentiality and solicitor-client privilege may restrict using third-party AI tools without client consent and adequate safeguards.

This is an educational overview, not legal advice. Consult qualified counsel for your specific industry, province, and use case before deploying AI automation that handles regulated data.

Frequently asked questions

Does self-hosting mean my prompts never leave Canada?

Self-hosting the agent keeps your workflow configuration, tool orchestrations, memory, and logs in Canada. If you use a cloud model API (OpenRouter, Anthropic), prompts are still processed on that provider's servers — typically in the US. To keep prompts in Canada, you need to run a local model (via llama.cpp or similar) on your Canadian VPS. See our data residency guide for the full picture.

How much does a self-hosted setup cost per month?

A typical small-team setup: $25-50/mo for a 2 vCPU/4 GB VPS in a Canadian data center, plus model API costs (variable, typically $10-100/mo depending on usage). Running a local model adds ~$48-96/mo for a VPS with sufficient RAM. Compare this to managed services or hiring a VA — self-hosting is often the most cost-effective option for teams running daily agent workflows.

Can I self-host Hermes Agent on a Mac mini at home?

Yes. Hermes Agent runs on macOS, and a Mac mini with Apple Silicon is an excellent self-hosting platform — especially if you want to run local models via llama.cpp. The main tradeoffs: your home internet must be reliable, you need to handle power outages, and you may need a static IP or dynamic DNS. For business-critical workflows, a Canadian VPS with professional uptime is usually the safer choice.

What happens to my data if the VPS crashes?

Back up ~/.hermes/ regularly — it contains your config, memory store, skills, and logs. A simple daily cron job that tars and copies to an off-server location (another VPS, Google Drive, or an S3 bucket in a Canadian region) is sufficient. VPS providers also offer snapshot/backup add-ons for the full disk.

Is self-hosting harder than using the managed API?

It adds about 30-60 minutes of initial setup and 30 minutes per month for maintenance (patching, reviewing logs, checking backups). If your team values data control and privacy, this is a small investment. If you have no technical team member comfortable with Linux and SSH, a managed API or a pre-configured deployment service may be a better starting point.

Related Hermes Agent guides

• What is Hermes Agent? — Start here if you're new to the platform
• Install Hermes Agent on macOS, Linux, and WSL — Full setup walkthrough
• Canadian data residency and AI agents — Deeper dive on residency requirements
• PIPEDA and AI automation for Canadian businesses — PIPEDA overview
• Privacy-safe AI workflows for Canadian SMBs — Workflow design patterns
• Custom AI agents vs SaaS tools — Build vs buy comparison
• Canadian AI automation basics — Getting started with automation