Hop3 0.7 Released¶

Hop3 0.7.0 is out. Where 0.5 was consolidation and 0.6 built platform capability, 0.7 is about a platform you can hand to someone else: install a real application from a browser, put a firewall in front of it, let it send email, and upgrade it without holding your breath.

It is also the final deliverable release of Hop3's NGI Zero grant.

Install an app from the dashboard¶

0.6 could browse a signed app catalog. 0.7 installs from it.

Open the catalog in the dashboard, pick an application, give it a name and a domain, and Hop3 provisions the databases it needs, installs the runtime, configures the reverse proxy, requests a certificate, runs the application's own first-run bootstrap, and creates an admin account for you. Twenty applications are in the catalog today: BookStack, Bugsink, Dolibarr, Easy!Appointments, Forgejo, Gitea, Invoice Ninja, Isso, Kanboard, Keycloak, LimeSurvey, Matomo, Mattermost, Miniflux, Nextcloud, Paheko, Radicale, Uptime Kuma, Vikunja and WordPress.

Then it signs in to check¶

Every catalog application ships a smoke test that authenticates through the app's own login, with the credential Hop3 generated, and confirms a wrong password is refused. It runs at the end of every deploy (dashboard deploys included) and on demand with hop3 app check --app <app>. Publishing to the catalog requires one.

An app can declare a [probe] account (non-privileged, owned and rotated by Hop3) so the check keeps working after you change the admin password.

A 200 from an app nobody can log into no longer counts as working.

What that found¶

Every application in the catalog deployed successfully and served a login page before this check existed. Bringing all twenty to a real login took twenty-three fixes, and almost all of them were in the platform:

• apps were served over plain HTTP as well as HTTPS, so the Secure cookies most of them set were never sent back, and every such login looped silently;
• PHP's built-in server ran with one worker, so any app that requests itself during installation deadlocked against its own only worker (ten of the twenty run on that server);
• an install whose deploy failed could not be retried, because the app record was committed before the deploy was attempted;
• [build].build was read and then ignored by seven of the twelve language toolchains;
• --clean reclaimed Hop3's records of addon databases without reclaiming the databases, so the next install adopted a populated one.

Then we pointed the same check at our own Nix recipes, which package each of those apps a second and third way. A deploy-oriented test matrix had scored one of those families 15 ok, 0 fail three days earlier. The sign-in check failed all sixteen; five turned out to be serving with no usable credential at all:

• Radicale's authentication type came from a variable that defaulted to none and was never set. Every calendar and address book was public.
• Isso's config had no administrator section, so its moderation dashboard was served disabled.
• Miniflux, Keycloak and LimeSurvey shipped literal passwords (changeme, password123) so the deployed instance had an administrator whose password is in our repository, while the operator held a generated one that did not work.

Every instrument we had reported those as working, because a server that authenticates nobody answers every request successfully. All are fixed, each now refuses to start unless explicitly configured, and they are listed under Security in the changelog. If you deployed the Nix variants of those apps, redeploy.

One more, found by reading recipes rather than by any check: both Nix packagings of Gitea and Forgejo deployed with open registration, because the setting that disables it lived in a shell script only the native recipe carried. An app with open registration signs in perfectly and refuses a wrong password; it passes the check. The bar is a floor.

A web application firewall¶

Hop3 already had a network firewall. 0.7 adds the Layer-7 half: LeWAF, a pure-Python, ModSecurity-compatible engine shipping the OWASP Core Rule Set, released standalone on PyPI.

Turn it on per app with [waf] enabled = true and traffic routes nginx → LeWAF → uWSGI, with the app itself on loopback so there is no path around it. Policy is declarative (named networks, per-path gates) and compiled to SecLang, validated before it is committed. Sources that keep tripping rules are scored and banned automatically, reconciled in-process, and visible through hop3 waf bans.

Email, as a backing service¶

Applications that send mail no longer arrive half-deployed. The operator picks a backend once at the server level and apps inherit it:

hop3 server email backend relay --provider resend --from-domain example.com

Three backends: relay (a provider or corporate smarthost, with EU-first provider profiles and a deliverability pre-flight that verifies SPF, DKIM and DMARC), catch (a development sink, validated end to end), and direct (the box as its own MTA, shipping as a preview).

An app opts in with [[addons]] type = "email" and is injected SMTP settings pointing at a queuing loopback relay, so the provider credential never enters an application's environment and the backend can change without touching a single app.

The platform uses the same channel to tell you things: turn on hop3 server email notifications and certificate-renewal failures and failed deploys reach you by mail.

Upgrades that roll back¶

hop3 app upgrade --app myapp

snapshots the app, redeploys it, runs its migrations, and verifies it comes back healthy. If the build, a migration, or the health check fails, the upgrade restores the pre-upgrade snapshot, so you are never stranded on a half-upgraded application. hop3 app rollback does it on demand.

Server upgrades got the same treatment from the other direction: after installing new code and migrating, the deployer confirms hop3-server actually answers before reporting the upgrade complete. If it does not, the deploy fails with the exact command to revert. Previously it could leave a silently dead server behind a "complete" message.

A cross-version end-to-end test covers this: install a baseline release on a fresh machine, then upgrade in place through a chain of versions, each installed by its own installer, so the test exercises the upgrade path people will actually take. It runs green on Docker and on a fresh cloud VPS.

Other changes¶

• hop3 scaffold writes a starter hop3.toml for the project in the current directory, with a #:schema line so your editor completes and validates it as you type. The JSON Schema is generated from the server's own validation models and published.
• Content-aware health checks: [healthcheck].contains = "..." means a deploy is only healthy when the app serves its own page. A bare 200 can be a placeholder or an error page.
• Failed deploys show the real cause: the actual error line, once, with a working pointer to the full log.
• hop3-deploy-server --provider hetzner --image <image> rebuilds the target from scratch before deploying, for a pristine box in one command.
• Twenty-two migration guides, covering Heroku, Dokku, Kubernetes, Vercel, Cloud Run and more, under Migrating to Hop3.

Other notable fixes¶

• Nix apps survive garbage collection: a running app no longer loses its files to nix-collect-garbage.
• A fresh install no longer serves a stale configuration. The installer started hop3-server before writing its config file, so first-time installs cached empty values. Redeploys were unaffected, which made it hard to spot.
• hop3 app credentials no longer shows a password for an account that was never created, and a bootstrap failure now includes the failing command's output.
• Signing keys stopped rotating on restart in the Nix recipes for Gitea and Forgejo, where they were minted inside a config file rewritten at every start. That invalidates every session and makes 2FA secrets undecryptable.

Installing and upgrading¶

On your server:

curl -LsSf https://hop3.cloud/install-server.py | sudo python3 -

On your laptop:

pip install hop3-cli

Existing installs upgrade in place and schema migrations run automatically. The WAF engine installs as the hop3-server[waf] extra; email needs Postfix, which hop3-install server --with email (included in --with all) provides.

What's next¶

0.7 closes the funded work. Deliberately out of scope for the grant: multi-tenancy and per-resource ownership, a monitoring and metrics dashboard, single sign-on, and multi-server deployment. Nearer term there is a reproducibility gate in CI for the Nix builds, an accessibility pass over the dashboard, and the email backends that ship as preview today.

Get involved¶

Hop3 is open source (Apache 2.0):

• Code: github.com/abilian/hop3
• Docs: hop3.cloud

Thanks to NLnet and the NGI Zero programme, whose funding paid for the Nix integration, the runtime work, the security and resilience features, and the twenty packaged applications that stress-tested all of it.

New to Hop3? Start with Your First Deployment. Coming from another platform? There is probably a migration guide for it.