Skip to content

apkg — new pipeline tooling for Obarun packages

Dear Obarun Community,

The build and deployment pipeline behind every Obarun package — [obcore], [obextra], [observice] and [obcommunity] — has been rebuilt from the ground up. The new tool is called apkg. It replaces the legacy apkg-*.sh bash scripts that have been quietly orchestrating Obarun package releases for more than 3 years. This post explains what changed, what it means for maintainers, and — most importantly — why contributing a package to [obcommunity] is now drastically easier than it used to be.

Why this matters

Until now, each package project carried a copy of a 12-script bash pipeline. Edge cases in version detection, signing, and tier promotion were handled inconsistently. Onboarding a new contributor involved a steep learning curve before they could understand how a package goes from a ‘PKGBUILD’ to the public repository.

apkg consolidates all of that into a single, tested Python application, packaged and shipped as obextra/apkg. The pipeline logic now lives in one place, behaves the same in every project, and is covered by an actual test suite.

Improvements

Several long-standing issues from the bash era are now properly handled:

  • Signing key isolation and Security — the Obarun signing key never leaves CI variables. Maintainers and operators do not hold it, even when publishing locally-built packages: the signature is added by a dedicated CI project after upload.
  • Epoch and split-package version comparison — a class of subtle bugs in the legacy pipeline that occasionally produced wrong or missed updates.
  • Concurrent repository operations — MinIO-backed locking with retry, replacing the old best-effort approach.
  • Cascading rebuilds — when a shared dependency bumps, downstream packages are now discovered, topologically sorted from .SRCINFO, and rebuilt in the correct order through a dedicated orchestrator, with parallel execution within independent topological waves.
  • Update detection — a single orchestrator job per repository detects upstream Arch bumps and dispatches the work, removing the need for per-package cron jobs.
  • Tier alignment — packages are promoted between staging, testing and stable automatically to match the tier where Arch currently has them, instead of relying on manual promotions in some cases.
  • Consistent exit codes and structured logging — every command follows the same convention (0 success, 111 failure) with a unified log format across the entire pipeline.

What changed under the hood

The legacy 12 bash scripts have been replaced by two binaries sharing a single library:

  • apkg — local administration: list, archive, delete, move, diff, audit packages stored in the Obarun repositories, plus optional helpers for the daily packaging workflow.
  • apkg-ci — CI engine: detects upstream Arch updates, builds, signs, deploys, promotes testing → stable, and runs dependency-aware cascading rebuilds.

All business logic lives in apkg.lib; the two binaries are thin frontends. The same code path runs whether a command is invoked locally on a maintainer's machine or inside a GitLab CI job.

What this means for package maintainers

The .gitlab-ci.yml of every package project is now a thin file that calls apkg-ci commands. There is nothing to copy, nothing to keep in sync between projects, and no bash internals to read in order to understand what the pipeline does. When the pipeline needs an improvement, the change happens once in apkg and propagates to every project on the next image rebuild.

For day-to-day work, apkg provides commands grouped by intent — build & publish, repo management, inspection, advanced — discoverable through apkg --help with full reference in docs/USAGE.md.

What this means for old/new contributors on Obcommunity

This is the part we care about most.

A contributor does not need to install or learn apkg to publish a package on Obcommunity. The CI pipeline runs on the GitLab side. From the contributor's machine, the entire packaging workflow comes down to standard git operations:

git add ...
git commit -m "..."
git push

That's it. Pushing the commit triggers the pipeline; the pipeline builds, signs, and deploys the package to the right tier of [obcommunity]. The contributor never touches apkg-ci, never holds the signing key, never logs into the repository server.

What needs to be respected is a minimal project skeleton, the same one used by every Obarun package. It looks like this:

mypackage/
├── mypackage/              # git submodule → upstream PKGBUILD source
│   └── .SRCINFO            # generated with `makepkg --printsrcinfo > .SRCINFO`
└── .gitlab-ci.yml          # template — copy as-is, two stages: build, deploy

The .gitlab-ci.yml is a 30-line template provided in the apkg repository under templates/skeleton/. Copy it once, never touch it again. Every other file in the layout is either standard PKGBUILD packaging or strictly optional.

If a contributor wants extra control, the control/ directory provides simple text files to adjust per-package behaviour without changing the pipeline:

  • control/tier — force staging, testing or stable.
  • control/cascade — declare which packages must be rebuilt when this one bumps.(require apkg use)
  • control/frozen — temporarily exclude a package from the orchestrator.(require apkg use)
  • control/makeflags, control/makeopts — adjust the build for memory-hungry packages.

Each is a single-line text file. Add one, commit, push — done.

If you have ever considered packaging software for Obarun but found the existing tooling intimidating, now is the time to give it another look. The path from "I want to package X" to "X is in [obcommunity]" is now short, predictable, and built on top of standard tools you already use.

Documentation

Getting involved

If you would like to contribute a package, improve an existing one, or report an issue with the new pipeline, the Obarun Git repository is the place to start. Feedback from early users of apkg is especially welcome — every report helps us refine the tooling further.

Thank you for your continued support of Obarun.

Eric Vidal: [email protected]