-
Notifications
You must be signed in to change notification settings - Fork 89
util/acct: implement a new hierarchical accounting system #398
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Draft
dvdhrm
wants to merge
10
commits into
bus1:main
Choose a base branch
from
dvdhrm:pr/acct
base: main
Could not load branches
Branch not found: {{ refName }}
Loading
Could not load tags
Nothing to show
Loading
Are you sure you want to change the base?
Some commits from the old base branch may be removed from the timeline,
and old review comments may become outdated.
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Switch the base image again from Fedora to Arch Linux. This requires us to disable SELINUX in the unit-tests, but those will be run by the packit integration, so we should be fine. The advantages are: * Image builds take *significantly* less time: 10min for Fedora, 3min for Arch Linux (on the same infrastructure). * Images are significantly smaller in size: 6GiB for Fedora, 3GiB for ArchLinux (with the same packages). We also get rolling releases, allowing us to control at which point we can update. We do not expose the CI system to untrusted inputs, anyway (nor do we pass sensitive data to it). Signed-off-by: David Rheinsberg <[email protected]>
Extend the build system to compile Rust sources in our tree into an internal, static library. This is then linked into `libbus.a`, which itself is our static library that contains everything but the entrypoints. This does not add any Rust code, but merely extends the build-system to support adding Rust code. Additionally, we add infrastructure to call `rust-bindgen` on our own C headers, so we get clean access to it from Rust. Note that this significantly bumps the build-time requirements of the project: - Meson 1.7.0 (January 2025): Needed for support of Rust Edition 2024 - Rust 1.85 (February 2025): Needed for support of Rust Edition 2024 - rust-bindgen 0.71 (December 2024): Needed for `--rust-edition` and `--rust-target` configuration. Signed-off-by: David Rheinsberg <[email protected]>
Instead of one static library with all Rust code, split it into two: 1) An rlib Rust library with all Rust code. 2) A staticlib Rust library that links the rlib and provides C stubs. This split allows us to link the Rust library to other Rust code, like doctests or unittests. It also keeps the C-API separate from the Rust code and provides a clean path forward. Signed-off-by: David Rheinsberg <[email protected]>
Export the generated symbols from our rlib, to ensure the C API can use them as well. While at it, ensure we suppress common warnings with the C naming scheme we use. Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we do not use assignments in the bindgen options. Meson parses these options to avoid adding duplicates, so lets provide them cleanly. Additionally, add `--use-core` now that we have rlib as `no_std`. Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we generate doctest targets with the new Meson version. This will test-run all Rust code in the documentation. Signed-off-by: David Rheinsberg <[email protected]>
With everything settled, we can now reduce the build requirements and turn some features into optional additions: - Use Rust-2021 over Rust-2024, since we do not benefit much from the new edition. - Reduce meson requirements to 1.3 and guard `mod_rust` additions by a version check. - Reduce Rust MSV to 1.74. - Reduce Bindgen MSV to 0.60 to at least get --allowlist-file. Signed-off-by: David Rheinsberg <[email protected]>
Use plain version numbers in MSVs and require all call-sites to add comparators, if necessary. Signed-off-by: David Rheinsberg <[email protected]>
Ensure that we do not run doctests when cross-compiling. This can easily fail and there is no suitable way for us to check whether `rustdoc` is available: mesonbuild/meson#14583 Signed-off-by: David Rheinsberg <[email protected]>
This is a replacement for `util/user.[ch]`. It is written in Rust, but exposes a C API that can be used by dbus-broker. The new accounting system uses the `acct_*` namespace to ease the transition and avoid naming conflicts. Furthermore, the system is written in Rust and relies on the new build-system support for Rust code. The accounting system works with the same algorithm as the previous flat-accounting model, but extends it to a 2-layered hierarchy. The algorithm is adopted to support an arbitrarily nested hierarchy, but the code will only expose 2 layers: 1) As before, all users are assigned a flat resource limit that they get full control over. But when user boundaries are crossed, a quota is applied to guarantee fair resource sharing. The algorithm used is still the `fairdist`, but the allocators were improved on. 2) Beneath the user quotas, a new layer is introduced. Rather than users claiming resources, now all operations originate from an `Actor`. These always operate on behalf of a user, but allow us to distinguish different actors of the same user. A new resource tracing system is now added analog to the user quotas. It operates above the quotas and ensures we can semi-protect individual actors of the same user (which was not possible before). We do not provide a fully fair policy on this level, however, since no privilege boundaries are crossed. Instead, this provides a semi-fair policy which is a lot more lenient, but still allows us to protect against failing clients. Signed-off-by: David Rheinsberg <[email protected]>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This is a replacement for
util/user.[ch]
. It is written in Rust, butexposes a C API that can be used by dbus-broker.
The new accounting system uses the
acct_*
namespace to ease thetransition and avoid naming conflicts. Furthermore, the system is
written in Rust and relies on the new build-system support for Rust
code (see #392).
Note that this is a single commit on top of #392. Once the Rust support is merged, this rebase is not necessary, anymore. Also note that this only introduces the accounting system as an alternative, it does not disable the old one, yet.
The accounting system works with the same algorithm as the previous
flat-accounting model, but extends it to a 2-layered hierarchy. The
algorithm is adopted to support an arbitrarily nested hierarchy, but the
code will only expose 2 layers:
As before, all users are assigned a flat resource limit that they get
full control over. But when user boundaries are crossed, a quota is
applied to guarantee fair resource sharing. The algorithm used is
still the
fairdist
, but the allocators were improved on.Beneath the user quotas, a new layer is introduced. Rather than users
claiming resources, now all operations originate from an
Actor
.These always operate on behalf of a user, but allow us to distinguish
different actors of the same user. A new resource tracing system is
now added analog to the user quotas. It operates above the quotas and
ensures we can semi-protect individual actors of the same user (which
was not possible before). We do not provide a fully fair policy on
this level, however, since no privilege boundaries are crossed.
Instead, this provides a semi-fair policy which is a lot more
lenient, but still allows us to protect against failing clients.