Agent Guide: Mathbox Server

This server runs Docker, Certbot, and HAProxy. All services are exposed under *.mathbox.90.cz via HAProxy.

Scope of this document:

  • production host operations only
  • deployment and HAProxy/certificate procedures
  • no application-level feature behavior (see module README files)

Quick Rules (Must Follow)

  • Do not install system packages or new services on the host.
  • Deploy applications only inside Docker containers.
  • Each project must be controlled via a Docker Compose file in its project directory.
  • Read /etc/haproxy/haproxy.cfg before making changes to learn existing domains and ports.
  • Always acquire the HAProxy lock before changes and release it after.
  • Always configure a proper Let's Encrypt certificate with Certbot.

Key Paths

  • HAProxy config: /etc/haproxy/haproxy.cfg
  • Docker deployments: /home/agent/docker_deployments/$PROJECT
  • HAProxy helpers: /usr/local/sbin/haproxy-apply, /usr/local/sbin/haproxy-le

Core Rules

  • Only add or adjust projects; do not remove or break existing ones.
  • Project directories must be owned by the agent user.
  • Use domains in the form $PROJECT.mathbox.90.cz.

Server Lock Procedure

  • Before any HAProxy change, acquire the lock with haproxy-apply --lock.
  • If the lock is already held, do not change configuration. Inform the user.
  • After your changes are complete, release the lock with haproxy-apply --unlock.

Deploying a New Project (Docker)

  1. Create the project directory: /home/agent/docker_deployments/$PROJECT.
  2. Place the Docker config there (compose file, env files, etc.).
  3. Start the containers for the project.
  4. Choose a project port exposed on localhost.
  5. Update HAProxy to route a subdomain (e.g. $PROJECT.mathbox.90.cz) to that port.
  6. Issue a Let's Encrypt certificate for the new domain(s).

Standard Docker Compose Template

Use a docker-compose.yml in each project directory:

version: "3.8"
services:
  app:
    build: .
    ports:
      - "5000:5000"
    restart: unless-stopped

Deployment Checklist

  1. Confirm the project directory exists and is owned by agent.
  2. Ensure docker-compose.yml exists and describes the service.
  3. Start the service with docker compose up -d.
  4. Verify the local port responds on 127.0.0.1:<port>.
  5. Lock HAProxy, update config, and apply with haproxy-apply.
  6. Issue/renew certificates with haproxy-le.
  7. Validate HTTP and HTTPS from the public domain.
  8. Unlock HAProxy.

HAProxy Changes

Use the HAProxy apply helper to update config safely: - sudo /usr/local/sbin/haproxy-apply --lock - sudo /usr/local/sbin/haproxy-apply /var/lib/haproxy-agent/incoming/new.cfg - The script validates the config, backs up the current version, applies, and reloads. - On reload failure, it automatically rolls back. - sudo /usr/local/sbin/haproxy-apply --unlock

Let's Encrypt Certificates

HAProxy is configured to route /.well-known/acme-challenge/ to a local Certbot listener on 127.0.0.1:8899.

Use the helper to issue or renew certificates: - Issue: - sudo /usr/local/sbin/haproxy-le --email ops@mathbox.90.cz -d $PROJECT.mathbox.90.cz -d www.$PROJECT.mathbox.90.cz - Renew all: - sudo /usr/local/sbin/haproxy-le --renew

If a cert already exists for any requested domain, the helper auto-expands it to include all requested domains. The helper creates fullchain.pem.key symlinks to privkey.pem for HAProxy. Configure HAProxy to use fullchain.pem and rely on the .key sidecar file. HAProxy is reloaded automatically after successful issuance or renewal.

Allowed Sudo Commands

  • sudo /usr/local/sbin/haproxy-apply ...
  • sudo /usr/local/sbin/haproxy-le ...

Rollback

  • To revert HAProxy to the last working config: sudo /usr/local/sbin/haproxy-apply --revert

Troubleshooting

  • If a service is down, check the local port first: curl -sS http://127.0.0.1:<port>.
  • If HAProxy reload fails, check /var/log/haproxy.log or journalctl -u haproxy.
  • If cert issuance fails, re-run with --staging and check /var/log/letsencrypt/letsencrypt.log.

Documentation Requirement

For each project you deploy or modify, create a README in the project directory that includes: - Ports used - Domain(s) - How to start/stop the service - Special operational notes

Runtime User and File Ownership (mandatory)

  • Podrobny projektovy runtime kontrakt je v docs/container-runtime-contract.md.
  • Aplikacni proces v kontejneru MUSI bezet jako non-root uzivatel.
  • Pro tento projekt se runtime uzivatel predava pres HOST_UID a HOST_GID do Compose.
  • Pri remote deployi se HOST_UID/HOST_GID musi odvodit z uzivatele, pod kterym bezi deploy na serveru (id -u, id -g).
  • Bind mount adresare data/ a logs/ na hostu musi zustat zapisovatelne timto uzivatelem.
  • Pokud se objevi root-owned soubory v data/ nebo logs/, je potreba pred dalsim startem srovnat ownership na runtime UID:GID.