Skip to content

ADR 003: Config Parsing and Validation

  • Status: Accepted
  • Type: Feature
  • Created: 2024-07-17
  • Related-ADRs: 001, 002

Context

We decided early in the project to reuse an existing syntax for the hop3.toml files, the heart of the Hop3 platform.

We chose to favor TOML for several reasons, including:

  1. Simplicity and Readability: TOML was designed to be simple and easy to understand for humans, making it well suited to configuration files. It aims to be more readable and straightforward than YAML or JSON, which can become complex and verbose with large data structures.
  2. Explicit and Obvious: TOML is designed to map unambiguously to a hash table. It aims to be more explicit and less prone to errors or misinterpretation than YAML, which has more complex features like references and tags.
  3. Consistent Style: TOML has a more consistent style, whereas YAML can be written in different ways (flow style and block style) which might cause confusion.
  4. Strong Typing: TOML has a clear type system, including explicit types for dates and times, which JSON lacks. While YAML also supports data types, its type system can lead to surprising results due to its reliance on tags.

However, we also chose to support JSON and YAML as alternatives because the concrete syntax of the hop3.toml files is mostly irrelevant, as long as it produces a valid JSON object.

Decision

  • Parse the configuration once (and report errors as soon as possible), apply some transformations, and transform it into JSON which will then be the reference file (loaded by jsonlib when necessary, without further transformation).
  • Implement schema validation for the hop3.toml file (see Validation Requirements below).
  • Add specific code to validate the "env" section (because we don't know the keywords a priori), and possibly other sections.

Detailed Design

Configuration access is property-based: a Hop3Config class parses hop3.toml with tomllib, and an AppConfig class merges Procfile and hop3.toml, exposing fields through property accessors. ADR 001 and ADR 002 document which fields are active.

Validation at load time is limited to TOML parse errors. Semantic errors (missing required field, wrong type) surface when the accessor runs. We defer formal schema validation to keep the dependency list empty and to iterate quickly on the config surface.

Validation Requirements

The validation system must provide:

  1. Type Checking
  2. Verify field types match specification (string, integer, list, dict, etc.)

  3. Handle optional vs required fields appropriately

  4. Required Field Validation

  5. Enforce mandatory fields (e.g., [metadata].id when metadata section is present)

  6. Provide clear errors when required fields are missing

  7. Format Validation

  8. URL format for website, src-url, git-url fields
  9. Version string format for version fields
  10. Cron pattern format for scheduled tasks

  11. App name format (alphanumeric + hyphens, length limits)

  12. Semantic Validation

  13. Worker type conflicts (e.g., can't have both web and wsgi)
  14. Provider reference validation (env vars referencing non-existent providers)

  15. Port number ranges

  16. Error Message Quality

  17. Clear, actionable error messages
  18. Include line/column numbers when possible
  19. Suggest fixes for common mistakes
  20. Support machine-readable error format (for tooling)

Implementation Options

Several approaches can meet the validation requirements:

Option Pros Cons
Enhanced dataclasses No new deps, simple Manual validation code
Pydantic v2 Rich validation, JSON Schema export Additional dependency
attrs + cattrs Lightweight, Pythonic Less automatic validation
msgspec Fast, good validation Less mature ecosystem
JSON Schema Language-agnostic, IDE support Separate from Python code

The implementation choice is left open. Any approach that meets the validation requirements is acceptable.

Alternatives

  • An ad-hoc class with @property accessors and no schema layer validates little: it defers semantic checks to access time, though it stays simple and dependency-free.

Consequences

Benefits

  • Better Developer Experience (DX): Early feedback about invalid configuration syntax or basic semantics reaches developers and package-builders sooner.
  • Fewer Runtime Dependencies: The build-time/runtime reliance on TOML or YAML parsers is avoided.
  • Easier Evolution: The configuration format can evolve through a consistent schema.

Related ADRs: ADR 001: Config Files for Hop3, ADR 002: Detailed hop3.toml Format