ADR 030: Two-Level Build Architecture¶

Status: Final
Type: Feature
Created: 2025-11-28
Implemented-In: v0.5.0
Related-ADRs: 006, 008, 020, 022, 035

Context¶

A build system can conflate two distinct architectural levels into a single hierarchy:

Build orchestration (HOW to build): Should we build locally, in Docker, with Nix?
Language-specific tooling (WHAT to build): How do we build Python vs Node vs Java?

This conflation creates several problems:

Problem 1: A Dual Constructor Reveals Architectural Confusion¶

A single builder base class that serves both levels needs a dual constructor, and type narrowing fails on it:

def __init__(
    self,
    app_name_or_context: str | DeploymentContext,  # Dual constructor
    app_path: Path | None = None,
) -> None:
    if hasattr(app_name_or_context, "app_name"):  # ❌ Type narrowing fails
        context = app_name_or_context
        self.app_path = context.source_path.parent  # ❌ Type error

Root Cause: Such a class tries to serve two purposes:

Abstract base for language toolchains (Python, Node, Java)
Plugin interface for build orchestration (invoked by plugin system)

Problem 2: A Flat Hierarchy Cannot Support Multiple Build Methods¶

A flat hierarchy:

Builder (Protocol)
  ├── PythonBuilder
  ├── NodeBuilder
  ├── StaticBuilder
  └── DummyBuilder

Issues:

A DockerBuilder that builds ALL languages in containers has no clean place in the hierarchy.
A NixBuilder that builds ALL languages with Nix has no clean place either.
Python+Node in a single app (full-stack) cannot be expressed.

A flat hierarchy treats all *Builder classes equally in the plugin system, but:

PythonBuilder is language-specific (Level 2)
DockerBuilder is language-agnostic (Level 1)
They belong to different architectural levels

Problem 3: Composite and Alternative Builds¶

Multi-language builds: A Python backend + Node frontend needs:

One orchestrator: LocalBuilder
Two toolchains: PythonToolchain + NodeToolchain

A flat hierarchy cannot express this cleanly.

Alternative build methods: Users want to choose:

Local builds (fast, uses host tools)
Docker builds (reproducible, isolated)
Nix builds (reproducible, declarative)
Buildpack builds (Heroku/Cloud Foundry compatibility)

A flat hierarchy mixes these concerns with language-specific logic.

Decision¶

Hop3 adopts a two-level build architecture that separates orchestration from language-specific tooling:

Builder (Level 1) decides how to build: local, Docker, or Nix.
LanguageToolchain (Level 2) decides what to build: Python, Node, Go, Rust, Ruby, Java, PHP, or a generic fallback.

Only the LocalBuilder delegates to LanguageToolchains. DockerBuilder and NixBuilder bypass the LanguageToolchain layer entirely: their build logic lives inside the Dockerfile or the Nix expression (see ADR 006 §"Architectural Context" and ADR 008).

BuildContext vs DeploymentContext¶

The protocols rest on a separation of build-time and deployment-time concerns:

@dataclass
class BuildContext:
    """Context for build operations (before deployment).

    Contains information needed during the build phase, before deployment.
    Separate from DeploymentContext to avoid coupling build and deploy concerns.
    """
    app_name: str
    source_path: Path
    app_config: dict

    def __post_init__(self):
        assert self.source_path.is_dir()


@dataclass
class DeploymentContext:
    """Context for deployment operations (after build).

    Contains information needed during the deployment phase, after build.
    """
    app_name: str
    source_path: Path
    app_config: dict
    app: App | None = None  # The full App object from the database

    def __post_init__(self):
        assert self.source_path.is_dir()

Rationale: Build and deployment are two independent phases. Builders operate during the build phase and don't need deployment-specific information like the database App object.

Level 1: Builder (Orchestration)¶

Protocol: Builder (in hop3/core/protocols.py)
Responsibility: Orchestrate HOW to build (environment, isolation, reproducibility)
Examples: LocalBuilder, DockerBuilder, NixBuilder, BuildpackBuilder
Selection: Config-driven (global or per-app hop3.toml)
Hook: get_builders()
Location: hop3/plugins/build/*/ (e.g., hop3/plugins/build/local/)

class Builder(Protocol):
    """Top-level build orchestrator - defines HOW to build.

    Builders orchestrate the build process and may delegate to language
    toolchains for language-specific operations.

    Examples:
    - LocalBuilder: Builds on host using native language toolchains
    - DockerBuilder: Builds in container using Dockerfile
    - NixBuilder: Builds with Nix for reproducibility
    """
    name: str
    context: BuildContext

    def __init__(self, context: BuildContext) -> None:
        """Initialize the builder with a build context."""

    def accept(self) -> bool:
        """Check if this builder should be used for the given context."""

    def build(self) -> BuildArtifact:
        """Orchestrate the build process and return the artifact."""

Level 2: LanguageToolchain (Language-Specific Logic)¶

Protocol: LanguageToolchain (in hop3/core/protocols.py)
Responsibility: Execute WHAT to build (dependencies, compilation, bundling)
Examples: PythonToolchain, NodeToolchain, JavaToolchain, RubyToolchain
Selection: Auto-detection (presence of requirements.txt, package.json, etc.)
Hook: get_language_toolchains()
Location: hop3/plugins/build/language_toolchains/

class LanguageToolchain(Protocol):
    """Language-specific build toolchain - defines WHAT tools to use.

    Toolchains handle language-specific build operations like installing
    dependencies, compiling code, and bundling assets.

    Examples:
    - PythonToolchain: Uses pip/uv, creates virtualenv, compiles .pyc
    - NodeToolchain: Uses npm/yarn, runs webpack, transpiles JS
    - JavaToolchain: Uses maven/gradle, compiles .class files
    """
    name: str
    context: BuildContext

    def __init__(self, context: BuildContext) -> None:
        """Initialize the toolchain with a build context."""

    def accept(self) -> bool:
        """Check if this toolchain applies to the project.

        Examples:
        - PythonToolchain: checks for requirements.txt or pyproject.toml
        - NodeToolchain: checks for package.json
        """

    def build(self) -> BuildArtifact:
        """Execute language-specific build and return the artifact."""

Rationale¶

Why Two Levels?¶

Separation of Concerns:

Level 1 (Builder): Environment and isolation concerns
Where to build? (host, container, sandbox)
How to isolate? (none, Docker, Nix)
What resources? (CPU, memory, network access)
Level 2 (LanguageToolchain): Language-specific concerns
What package manager? (pip, npm, maven)
How to install dependencies?
How to compile/transpile code?

Orthogonal Variation:

LocalBuilder uses LanguageToolchains to build on the host
DockerBuilder encapsulates build logic in the Dockerfile (no toolchains)
NixBuilder uses Nix expressions (no toolchains)

LanguageToolchains are specific to LocalBuilder and enable:

Multi-language builds (Python + Node in a single app)
Auto-detection of applicable languages
Reusable language-specific build logic

Why "Builder" at Level 1?¶

Domain Language:

DevOps engineers ask: "What builder are you using?"
They mean: "Are you building locally, in Docker, or with Nix?"
The Builder protocol in hop3/core/protocols.py lives at this level

Consistency with Terminology:

Follows Heroku-inspired naming (Builder, Deployer, Addon)
Avoids generic suffixes like "Strategy" or "Method"
Natural and concrete term

Why "LanguageToolchain" at Level 2?¶

Established Terminology:

"Python toolchain", "Node toolchain" are industry-standard terms
Refers to the set of tools needed to build a language (pip, virtualenv, compiler, etc.)
More specific than generic "backend" or "strategy"

Avoids Confusion:

"Backend" could mean server-side code (vs frontend)
"Toolchain" is unambiguous: the build tools for a language

Consequences¶

Positive¶

✅ Clear Separation of Concerns: Build orchestration and language tooling are distinct

✅ Multi-Language Support: One LocalBuilder can use multiple toolchains

# Full-stack app: Python backend + Node frontend
builder = LocalBuilder(context)
artifact = builder.build()

# Inside LocalBuilder.build() implementation:
# - Discovers both PythonToolchain and NodeToolchain
# - Builds with each: python_artifact, node_artifact
# - Combines them into a single artifact

✅ Extensibility: Easy to add new builders (Nix, Buildpack) or toolchains (Rust, Go)

✅ Type Safety: Clear protocol boundaries enable proper type checking

✅ Solves the Type Error: The dual constructor issue goes away when we split the abstraction

Negative¶

⚠️ Renaming and restructuring: The original language-specific *Builder classes become *Toolchain, and the original Builder ABC becomes LanguageToolchain.

⚠️ Complexity: Two protocols

Requires clear documentation
Plugin authors need to understand the distinction

Neutral¶

🔄 Backwards Compatibility: A single get_builders() hook can return both Builders and LanguageToolchains, so the split can be introduced before the hooks are separated into get_builders() and get_language_toolchains().

Alternative Approaches Considered¶

Alternative 1: Keep Flat Hierarchy, Add Marker Attribute¶

class Builder(Protocol):
    name: str
    is_orchestrator: bool = False  # True for Docker/Nix, False for Python/Node

Rejected:

Doesn't solve the type narrowing issue
Still mixes two concerns in one abstraction
Unclear semantics (what does is_orchestrator mean?)

Alternative 2: Use Composition Instead of Protocols¶

class Builder:
    def __init__(self, toolchains: list[LanguageToolchain]):
        self.toolchains = toolchains

Rejected:

Doesn't work for DockerBuilder (doesn't use toolchains)
Forces all builders to use the same pattern
Less flexible than protocol-based approach

Alternative 3: Single Builder, Strategy Pattern for Toolchains¶

Keep Builder at Level 1, but have it use a "ToolchainStrategy" internally.

Rejected:

Adds unnecessary indirection
Still need to rename existing classes
More complex than two protocols

Future Considerations¶

Multi-Language Configuration¶

Users may need to configure which toolchains to use:

# hop3.toml
[build]
method = "local"  # or "docker", "nix"
toolchains = ["python", "node"]  # Explicit toolchain selection

[build.python]
package_manager = "uv"  # or "pip", "poetry"

[build.node]
package_manager = "pnpm"  # or "npm", "yarn"

Toolchain Dependencies¶

Some toolchains may depend on others:

TypeScript toolchain depends on Node toolchain
Sass toolchain depends on Node toolchain

May need dependency resolution in the future.

Performance Optimization¶

Building with multiple toolchains sequentially may be slow. Consider:

Parallel builds (if toolchains are independent)
Incremental builds (cache toolchain outputs)
Artifact reuse (don't rebuild unchanged components)

References¶

Plugin System: packages/hop3-server/src/hop3/core/plugins.py
Related ADRs:
ADR 020: Pluggable Architecture
ADR 022: Build/Deploy Plugin System