Hop3 0.8 Released¶

Hop3 0.8.0 is out. 0.7 made Hop3 a platform you can hand to someone else. 0.8 is for the server that has been handed over and filled: several apps, several people, one of whom has a production context open in another terminal. Most of this release is about that server. Apps keep what they wrote. Commands go where you meant. An account does what its role allows. A failure stops the operation that hit it.

Accounts have roles¶

There are four roles. An admin manages accounts and the server. A deployer deploys and operates apps and addons. A viewer reads apps, addons, backups and certificates, but no secret. A member signs in to the web interface and sees its profile.

hop3 user set-role alice viewer

The server checks the role on each command and dashboard page. A command above an account's role is refused, the message names the role it needs and the command that grants it, and hop3 exits with code 5. Upgrading keeps admins as admins and makes every other account a deployer, which is what each could already do. A new account starts as a member.

The same accounts can now guard an app. With [access] require = "auth", a visitor without a session is sent to the Hop3 sign-in and back to the page they asked for:

[access]
require = "auth"
role = "viewer"
public-paths = ["/hooks/.*", "/health"]

The app learns who is asking from X-Hop3-User and X-Hop3-Role, headers the gate sets and a client cannot forge. One sign-in covers all gated apps on the server, and signing out of Hop3 signs out of all of them. The gate works on nginx, Caddy and Traefik. The sign-in page itself lives on the admin domain, which the installer routes on nginx only for now.

Every command says where it goes¶

A checkout now names the server it deploys to:

hop3 context use staging

That binding lives in .hop3-local.toml, which is gitignored. Inside a project, a command that reaches a server uses the checkout's binding, -c, or HOP3_CONTEXT. Without one, it is refused. Such a command prints where it goes, and why, on stderr:

→ staging (https://staging.example.com), bound in this checkout

--why shows the whole resolution. Before 0.8, a project directory and the global current context could disagree on which server was meant, and the global one won without a word. It changed on every login.

A server can also mark itself as production. With PROTECTED = true in hop3-server.toml, a change is accepted only when -c or HOP3_CONTEXT named that server. The operator then types the app's name to confirm. The server enforces this from what the CLI reports, so a stale binding in someone's checkout cannot get past it.

Apps keep their state across a redeploy¶

Each deploy replaces an app's source tree. In 0.7, several paths lost what an app had written there:

• Declaring a volume deleted the data it was meant to keep. If an app kept uploads in its source tree and a later version declared [[volumes]] for them, the deploy replaced the tree before linking the new, empty volume. The old tree is now set aside until the volumes are linked. An empty volume is filled from it.
• A redeploy built on the previous build's output. An upgrade, a restore or a scale ran the build again over that output, so a recipe that unpacks the application failed (mv: cannot overwrite './vendor'), and so did the rollback meant to undo it. Each build now starts from the tree the app was delivered with.
• Apps shared one home directory. Apps ran with the server's HOME and /tmp, so Gitea and Forgejo could rewrite the authorized_keys that holds Hop3's git-push keys. Each app now has its own HOME and TMPDIR.

A new scratch volume type holds what an app writes and must not keep, such as compiled templates. It is reset at each deploy and never backed up.

The test harness now redeploys each app it tests and runs the validations a second time. It also fails an app that writes into the source tree while running, and lists the directories it wrote, ready to become [[volumes]] entries. Every recipe in the published catalog passes a deploy, an upgrade and a second check, installed from the published catalog.

App code can no longer reach root¶

Native apps all ran as hop3, the user that hop3-rootd, the privileged helper, admits. Any app's code could ask rootd for anything, and that amounted to root without an exploit. rootd now refuses a caller running inside the app runtime. Several other paths closed with it:

• An app's environment left the command line. Workers, php-fpm pools and apps with a start command received their [env] as export statements on a command line visible to ps. Any local account could read them. The environment now goes to a file only the app's owner can read.
• A PHP app's hidden files are no longer served. nginx served .env and .htaccess from a PHP app's document root. nginx and Caddy now answer them with 404.
• A server installs no dependency released in the last seven days. A compromised package is usually caught and yanked within that window. Hop3's own packages are exempt, so a release installs the day it ships.

One gap remains: apps still share one Unix user, so one app can read another's files, and the control plane's. Per-app users are the headline of 0.9. The design is in ADR 055.

After upgrading, redeploy each native app once. The environment and hidden-file fixes take effect at the next deploy; hop3 app restart reuses the old configuration.

The installer stops at a failure¶

The server installer had many steps that caught a failure, printed a warning cut to 100 characters, and went on to report success. A failed config write produced a server without database credentials, and a "Systemd setup issue" appeared on each container install. A failed pip install was printed under "installed successfully".

These steps now stop the install and shows the failing command's output. That covers systemd, TLS, nginx, the databases, the config file, the toolchains, Postfix and the S3 service. hop3-deploy-server now fails a deploy whose certificate request failed, where it used to install a self-signed certificate and report success. The test harness fails any server deploy that prints a ⚠, unless an allowlist entry says why that warning is safe. Those lines had sat unread in each run log for weeks. They hid two bugs and an install that failed on all Ubuntu 24.04 servers.

The installer also runs on Ubuntu 26.04 now.

Proxies, PHP and Nix¶

• Caddy and Traefik serve apps. Each deploy on a Caddy server used to fail validation. Traefik's routes pointed at a certificate resolver nothing defined. Both now serve apps, Caddy serves PHP apps through php-fpm, and a deploy a proxy cannot serve is refused before the build, with the reason.
• A Nix-built PHP app runs under php-fpm, as a native one has since 0.7.5. PHP's built-in server has one worker, and an app that requests its own URL deadlocks against it.
• A generated Nix app runs its setup once per deploy, before it starts. Migrations used to run at each start, inside the start timeout, which BookStack's outlasted.
• Generated Nix recipes build against nixos-26.05 and can fetch their source from GitHub by repository and revision.

A versioned catalog¶

The catalog moves to https://apps.hop3.cloud/catalog/v1/catalog.tar.gz. Each entry now says which server features it needs. The publisher works these out from the recipe and refuses one that under-declares them. A server that lacks a feature refuses the install by name, before it creates anything. Servers before 0.8 keep fetching the old address, which serves the same catalog, and they are refused the entries that need 0.8.

Other changes¶

• Backups restore what they took. A PostgreSQL restore loads into a fresh database in one transaction, and replaces the live one only once the load succeeded; it used to replay over the live tables and exit 0 on errors. An S3 backup now holds the bucket's objects: before 0.8 it held a manifest and restoring it put nothing back, so take a new backup of each app with an S3 addon.
• The catch email backend has a sink: Mailpit, installed with --with email, and hop3 server email caught lists what it caught.
• A command missing an argument exits 2 with the usage. It used to print the usage and exit 0, so hop3 addon postgres export > dump.sql without a name wrote the usage into the dump.
• --dry-run reaches the server. The CLI kept the flag for itself on all commands but deploy, so system cleanup --dry-run ran for real.
• hop3 addon expose and [[ports]] work on an upgraded server. Inserts into those tables failed on a database created by migration.

Installing and upgrading¶

On your server:

curl -LsSf https://hop3.cloud/install-server.py | sudo python3 -

On your laptop:

uv tool install hop3

The hop3 name on PyPI now installs the client. Releases 0.1 to 0.2 under that name were an unrelated 2024 server and are yanked. hop3-cli still works.

Existing installs upgrade in place, and schema migrations run automatically. When coming from 0.7:

1. Redeploy each native app once.
2. Run hop3 context use <name> in each project checkout.
3. On a production server, set PROTECTED = true in hop3-server.toml.
4. Take a new backup of each app with an S3 addon.

The changelog has the full list, including the commands whose arguments changed.

What's next¶

0.9 gives each app its own Unix user, so that app code cannot read the data of other apps or of the control plane (plan). After that, builds run as the app's user too. The installer's features also become independently installable.

Get involved¶

Hop3 is open source (Apache 2.0):

• Code: github.com/abilian/hop3
• Docs: hop3.cloud

New to Hop3? Start with Your First Deployment.