Materia v0.8 has been released! This was a very busy summer so not as many user-facing features were added with this release, but the internal core is always improving.

Still, that doesn’t mean you won’t see anything new! Read on for the highlights, or skip directly to the changelog

More useful facts generation

Materia now gathers the facts it needs to generate plans in a much more standardized manner: Facts are gathered by collectors and organized into namespaces. The number of built in facts have also been expanded to cover information like “what components are installed” and “what was the last source revision applied and was it successful”?

Existing facts, such as the current hostname and network interface(s) available have been moved into the host namespace.

The following facts are now available: ~~~default host.hostname: The hosts reported (not configured in Materia) hostname host.interfaces: All network interfaces on the host and their IPv4/IPv6 addresses

materia.hostname: The configured hostname, if any materia.roles: What roles the host has materia.assigned_components: What components are assigned to the host materia.installed_components: What components are already installed

materia.last_revision: The last source revision used in an update operation materia.last_run: When the last update opperation occured materia.last_run_successful: Whether the last update operation was successful. ~~~

As always, these facts are accessible via the materia facts commands or the facts Varlink call for the server. The next step for fact generation is to allow custom external fact collectors.

As an internal note, anything gathered as a fact is now gathered only at the start of a Materia plan. For most users this shouldn’t affect anything, but it does make plan generation much more deterministic, so if you had issues with, say, network interfaces coming and going during plan generation that should happen no more!

Consistent Output Data

Those new source revision facts didn’t come from nowhere: Materia now saves output data on each plan and run to $MATERIA_DIR/output for your future perusal and consumption by external tools.

Each materia plan and materia update operation, whether via command line or server mode, will now generate a lastplan.toml and lastrun.toml file, respectively. Up to five instances (customizable in the future) of each will be saved; older files will be rotated to filename-x.toml. For example, if lastrun.toml already exists in the output directory, the next run will be saved as lastrun.toml and the existing run will become lastrun-1.toml.

These files are parsed by materia to generate the last_revision/last_run//etc facts, so be aware that if you delete them you won’t have those facts until the next run.

Server mode improvements

Server mode continues to get many improvements to bring it in-line with the existing CLI materia plan-materia upate loop.

Server mode now uses the exact same internal calls as the CLI; no more strange mismatches between the two!
Server mode webhook management has been expanded:
- You can now pass the security token as a Bearer: token instead of in the /update payload
- A new /facts endpoint that you can GET. This is useful for validating current deployment status
- There’s now a /health endpoint that gets automatically activated when any other endpoint is activated
Rollback now works the same for server mode as it does for the CLI. Note: this is still dependent on individual source support.

The Varlink API has also been updated to match the new facts command line arguments.

File management improvement

Materia can now manage the permissions of files it manages, such as its Quadlet and data files. This is included along with many other changes¹ that should help prevent any path escapes from happening during Materia execution.

If you haven’t been running latest, this will be visible as a big wave of Update actions being taken. This will only happen once as Materia fixes all the file permissions that lazily got installed incorrectly.

A new release pipeline

Materia releases have been re-organized into binary tarballs instead of raw binaries and are now signed via Cosign upon CI release. Release images are also signed via Cosign; please note that as :stable just points to the latest tagged release, it does not have its own signed image digest.

The public key for verifying the signed releases can be found in the repository root and on this website’s root.

The new binary tarballs also include the README and generated man pages for your convenience.

Other Changes You Should Know:

Public Packages

For a while now most of the Go packages used within Materia were public; this was to make it easier to make tools like athanor that re-use the Materia component manifest format.

At this point, all of the internal packages used in Materia are now available for use by other Go projects. This includes how it defines resources and components, plan generation and execution, source definitions, etc. The plan is for these packages API to stabilize along with CLI version 1.0².

Git source improvements

The git source now supports using Bearer authentication tokens for login via HTTP clones via git.auth_token. It can also detect if your local source cache has gotten corrupted and will redownload it.

`materia` image changes

The materia image is now built from podman/stable instead of tumbleweed:bci. There should be no user-facing differences from this change (except for a small image size) but please let us know if there’s any problems.

Changelog

- feat: most materia packages are now public. This will be stabilized and documented better in 0.9.
    - This may also fix some bugs in server mode that (hopefully) no-one has noticed
- feat: git repos can reset on corrupted HEAD
- feat: git source supports `git.auth_token` for `Bearer: <token>` HTTP auth
- refactor: rewrote file management backend
    - Files are now installed with consistent permissions and using `os.Root` to prevent symlink/path escaping
    - You may notice an initial wave of `Update` actions as materia fixes installed file permissions. Note that changing permissions does count as a normal update action, so all triggered effects (container restarts,etc) will still occur
- feat/refactor: The `materia` container image is now built from `podman/stable` instead of `tumbleweed/bci:latest`
- ci: release tarballs and images are now signed via cosign. Public key for validation is in the repo and the main primamateria.systems website
- feat/rework: Redid `facts` generation. Facts are now denoted in dotted-path format.
    - Existing facts and their new versions:
        - `hostname` -> `materia.hostname`
        - `interface` -> `host.interfaces`
    - New facts
        - `host.hostname`: hostname as the OS sees its
        - `materia.roles`: Assigned roles
        - `materia.assigned_components`: Assigned components
        - `materia.installed_components`: Installed components
- feat: generate a $OUTPUTDIR/lastrun.toml file on `materia update` or `materia server update`. Up to five last-plan and last-run files are stored.
    - If you have existing `lastrun.toml` files, you need to remove them before running `materia facts`.
- feat: facts `materia.output` now shows last applied revision and timestamp and whether it was succesful:
    - New facts:
        - `materia.last_revision`
        - `materia.last_run`
        - `materia.last_run_succesful`
- feat: server mode supports new `server.token` for `Bearer: token` style authentication:
    - this replaces the now deprecated `server.update_secret`.
- feat: other server config changes:
    - `server.url`: replaces `server.update_url`
    - `server.update_webhook` requires `server.update_secret` or `server.token`
    - `update_secret` or `token` checks are disabled by setting `server.url` to `localhost*`
- feat: new `/facts` endpoint for server mode. Enable with `server.facts_endpoint`  Requires `server.token` to be enabled.
- feat: new `/health` endpoint, automatically enabled when any other webhook is enabled or if `server.health` is `true`.
- bugfix: `lastplan`/`lastrun` files can no longer include un-redacted podman secrets. Thanks shirome on Matrix!
    - The planner no longer contains Podman secrets contents at all, even in memory, so this should not happen again
- feat: server and cli use same internal commands. Server now supports rollback
- chore: upgrade to Go 1.27.3
- feat/chore: rework `materia validate` to use new facts system

Deprecations

The file source has been renamed to local to prevent settings collisions with the file attributes engine. Setting the SOURCE_KIND to file will work until after 0.8 is released. This is a straight rename; no other settings to be adjusted besides replacing file with local.
The following server settings are deprecated:
- server.update_url: replaced with server.url
- server.update_secret: replaced with Bearer token auth with server.token
The following facts are deprecated:
- hostname: replaced with host.hostname
- interface: replaced with host.interfaces

Footnotes

Fun fact for the day: the file management code, along with the aforementioned facts generation code, was one of the oldest bits of the project! It’s basically sat almost untouched (outside of shuffling the file its in around) since the project started two years ago.↩︎
This isn’t a hard commitment; I’m not willing to let the API hold back the much-more-likely-to-be-used CLI if it would be a big deal.↩︎