Skip to content

util/acct: implement a new hierarchical accounting system #398

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Draft
wants to merge 10 commits into
base: main
Choose a base branch
from

Conversation

dvdhrm
Copy link
Member

@dvdhrm dvdhrm commented May 9, 2025

This is a replacement for util/user.[ch]. It is written in Rust, but
exposes a C API that can be used by dbus-broker.

The new accounting system uses the acct_* namespace to ease the
transition and avoid naming conflicts. Furthermore, the system is
written in Rust and relies on the new build-system support for Rust
code (see #392).

Note that this is a single commit on top of #392. Once the Rust support is merged, this rebase is not necessary, anymore. Also note that this only introduces the accounting system as an alternative, it does not disable the old one, yet.

The accounting system works with the same algorithm as the previous
flat-accounting model, but extends it to a 2-layered hierarchy. The
algorithm is adopted to support an arbitrarily nested hierarchy, but the
code will only expose 2 layers:

  1. As before, all users are assigned a flat resource limit that they get
    full control over. But when user boundaries are crossed, a quota is
    applied to guarantee fair resource sharing. The algorithm used is
    still the fairdist, but the allocators were improved on.

  2. Beneath the user quotas, a new layer is introduced. Rather than users
    claiming resources, now all operations originate from an Actor.
    These always operate on behalf of a user, but allow us to distinguish
    different actors of the same user. A new resource tracing system is
    now added analog to the user quotas. It operates above the quotas and
    ensures we can semi-protect individual actors of the same user (which
    was not possible before). We do not provide a fully fair policy on
    this level, however, since no privilege boundaries are crossed.
    Instead, this provides a semi-fair policy which is a lot more
    lenient, but still allows us to protect against failing clients.

dvdhrm added 10 commits May 9, 2025 16:16
Switch the base image again from Fedora to Arch Linux. This requires us
to disable SELINUX in the unit-tests, but those will be run by the
packit integration, so we should be fine.

The advantages are:

 * Image builds take *significantly* less time: 10min for Fedora, 3min
   for Arch Linux (on the same infrastructure).

 * Images are significantly smaller in size: 6GiB for Fedora, 3GiB for
   ArchLinux (with the same packages).

We also get rolling releases, allowing us to control at which point we
can update. We do not expose the CI system to untrusted inputs, anyway
(nor do we pass sensitive data to it).

Signed-off-by: David Rheinsberg <[email protected]>
Extend the build system to compile Rust sources in our tree into an
internal, static library. This is then linked into `libbus.a`, which
itself is our static library that contains everything but the
entrypoints.

This does not add any Rust code, but merely extends the build-system to
support adding Rust code.

Additionally, we add infrastructure to call `rust-bindgen` on our own C
headers, so we get clean access to it from Rust.

Note that this significantly bumps the build-time requirements of the
project:

- Meson 1.7.0 (January 2025): Needed for support of Rust Edition 2024
- Rust 1.85 (February 2025): Needed for support of Rust Edition 2024
- rust-bindgen 0.71 (December 2024): Needed for `--rust-edition` and
  `--rust-target` configuration.

Signed-off-by: David Rheinsberg <[email protected]>
Instead of one static library with all Rust code, split it into two:

1) An rlib Rust library with all Rust code.
2) A staticlib Rust library that links the rlib and provides C stubs.

This split allows us to link the Rust library to other Rust code, like
doctests or unittests. It also keeps the C-API separate from the Rust
code and provides a clean path forward.

Signed-off-by: David Rheinsberg <[email protected]>
Export the generated symbols from our rlib, to ensure the C API can use
them as well. While at it, ensure we suppress common warnings with the C
naming scheme we use.

Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we do not use assignments in the bindgen options. Meson
parses these options to avoid adding duplicates, so lets provide them
cleanly.

Additionally, add `--use-core` now that we have rlib as `no_std`.

Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we generate doctest targets with the new Meson version. This
will test-run all Rust code in the documentation.

Signed-off-by: David Rheinsberg <[email protected]>
With everything settled, we can now reduce the build requirements and
turn some features into optional additions:

- Use Rust-2021 over Rust-2024, since we do not benefit much from the
  new edition.

- Reduce meson requirements to 1.3 and guard `mod_rust` additions by a
  version check.

- Reduce Rust MSV to 1.74.

- Reduce Bindgen MSV to 0.60 to at least get --allowlist-file.

Signed-off-by: David Rheinsberg <[email protected]>
Use plain version numbers in MSVs and require all call-sites to add
comparators, if necessary.

Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we do not run doctests when cross-compiling. This can easily
fail and there is no suitable way for us to check whether `rustdoc` is
available:

    mesonbuild/meson#14583

Signed-off-by: David Rheinsberg <[email protected]>
This is a replacement for `util/user.[ch]`. It is written in Rust, but
exposes a C API that can be used by dbus-broker.

The new accounting system uses the `acct_*` namespace to ease the
transition and avoid naming conflicts. Furthermore, the system is
written in Rust and relies on the new build-system support for Rust
code.

The accounting system works with the same algorithm as the previous
flat-accounting model, but extends it to a 2-layered hierarchy. The
algorithm is adopted to support an arbitrarily nested hierarchy, but the
code will only expose 2 layers:

1) As before, all users are assigned a flat resource limit that they get
   full control over. But when user boundaries are crossed, a quota is
   applied to guarantee fair resource sharing. The algorithm used is
   still the `fairdist`, but the allocators were improved on.

2) Beneath the user quotas, a new layer is introduced. Rather than users
   claiming resources, now all operations originate from an `Actor`.
   These always operate on behalf of a user, but allow us to distinguish
   different actors of the same user. A new resource tracing system is
   now added analog to the user quotas. It operates above the quotas and
   ensures we can semi-protect individual actors of the same user (which
   was not possible before). We do not provide a fully fair policy on
   this level, however, since no privilege boundaries are crossed.
   Instead, this provides a semi-fair policy which is a lot more
   lenient, but still allows us to protect against failing clients.

Signed-off-by: David Rheinsberg <[email protected]>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant