Skip to content

Add support for CEL based rules and profiles#14597

Open
yuumasato wants to merge 9 commits intoComplianceAsCode:masterfrom
yuumasato:add-cel-rules
Open

Add support for CEL based rules and profiles#14597
yuumasato wants to merge 9 commits intoComplianceAsCode:masterfrom
yuumasato:add-cel-rules

Conversation

@yuumasato
Copy link
Copy Markdown
Member

Description:

  • Adds support for CEL based rules along with documentation and some tests
  • The CEL rules are supported by Compliance Operator, so they are limited to ocp4 product.

Rationale:

  • As we integrate and expand CEL content support for OpenShift with Compliance Operator, we would like to have these rules maintained and developed side by side with other similar security contents.

Review Hints:

  • Build OCP4 content and check the 'ocp4-cel-content.yaml' in the bulid directory.

@xiaojiey
Copy link
Copy Markdown
Collaborator

I verified PR #14597 and PR ComplianceAsCode/compliance-operator#1103 together. Generally it is good. The only problem is there is no COPY --from=builder /content/build/ocp4-cel-content.yaml . in the BuildConfig, I have to create a BuildConfig manually.
I failed to set up a kubevirt cluster today. Will continue verify the rules tomorrow.

1. ### Step 1: Build the Content Locally
# Build OCP4 product
./build_product ocp4
# Verify CEL content exists
ls -lh build/ocp4-cel-content.yaml
2. ###Step 2: build content image:
##2.1. Updated the BuildConfig (this is the code change):
$ oc apply -f - <<'EOF'
  apiVersion: build.openshift.io/v1
  kind: BuildConfig
  metadata:
    name: openscap-ocp4-ds
    namespace: openshift-compliance
  spec:
    output:
      to:
        kind: ImageStreamTag
        name: openscap-ocp4-ds:latest
    runPolicy: Serial
    source:
      dockerfile: |
        FROM registry.fedoraproject.org/fedora-minimal:38 as builder

        WORKDIR /content

        COPY . .

        RUN microdnf -y install cmake make git /usr/bin/python3 python3-pyyaml python3-jinja2 openscap-utils

        RUN ./build_product --debug ocp4 rhcos4

        FROM registry.access.redhat.com/ubi8/ubi-minimal
        WORKDIR /
        COPY --from=builder /content/build/ssg-ocp4-ds.xml .
        COPY --from=builder /content/build/ssg-rhcos4-ds.xml .
        COPY --from=builder /content/build/ocp4-cel-content.yaml .
      type: Dockerfile
    strategy:
      dockerStrategy:
        noCache: true
      type: Docker
    triggers:
    - type: ImageChange
  EOF

 ##2.2 Triggered a new build:

$ oc start-build openscap-ocp4-ds -n openshift-compliance --from-dir=.

3. ###Verified the image contains CEL content:

$ oc run -it --rm verify-cel \
    --image=image-registry.openshift-image-registry.svc:5000/openshift-compliance/openscap-ocp4-ds:latest \
    --restart=Never \
    -n openshift-compliance \
    -- ls -la /

4. ###Created ProfileBundle with CEL content:
$ oc apply -f - <<'EOF'
  apiVersion: compliance.openshift.io/v1alpha1
  kind: ProfileBundle
  metadata:
    name: ocp4-with-cel
    namespace: openshift-compliance
  spec:
    contentImage: image-registry.openshift-image-registry.svc:5000/openshift-compliance/openscap-ocp4-ds:latest
    contentFile: ssg-ocp4-ds.xml
    celContentFile: ocp4-cel-content.yaml
  EOF
5 ### Check the result:
$ oc get pb
NAME              CONTENTIMAGE                                                                                    CONTENTFILE         CELCONTENTFILE          STATUS
ocp4              ghcr.io/complianceascode/k8scontent:latest                                                      ssg-ocp4-ds.xml                             VALID
ocp4-with-cel     image-registry.openshift-image-registry.svc:5000/openshift-compliance/openscap-ocp4-ds:latest   ssg-ocp4-ds.xml     ocp4-cel-content.yaml   VALID
rhcos4            ghcr.io/complianceascode/k8scontent:latest                                                      ssg-rhcos4-ds.xml                           VALID
upstream-ocp4     openscap-ocp4-ds:latest                                                                         ssg-ocp4-ds.xml                             VALID
upstream-rhcos4   openscap-ocp4-ds:latest                                                                         ssg-rhcos4-ds.xml                           VALID
$ oc get rules -n openshift-compliance -o json | jq -r '.items[] | select(.scannerType == "CEL") | .metadata.name'e'
ocp4-with-cel-kubevirt-enforce-trusted-tls-registries
ocp4-with-cel-kubevirt-no-permitted-host-devices
ocp4-with-cel-kubevirt-no-vms-overcommitting-guest-memory
ocp4-with-cel-kubevirt-nonroot-feature-gate-is-enabled
ocp4-with-cel-kubevirt-persistent-reservation-disabled
$ oc get rules | grep kubevirt
ocp4-with-cel-kubevirt-enforce-trusted-tls-registries                                        7m7s
ocp4-with-cel-kubevirt-no-permitted-host-devices                                             7m7s
ocp4-with-cel-kubevirt-no-vms-overcommitting-guest-memory                                    7m7s
ocp4-with-cel-kubevirt-nonroot-feature-gate-is-enabled                                       7m7s
ocp4-with-cel-kubevirt-persistent-reservation-disabled                                       7m7s

@jan-cerny jan-cerny added the OpenShift OpenShift product related. label Mar 25, 2026
We expect this profile to exclusively leverage the CEL rules.
Add a new build-script along with a new output type that builds the CEL
rules into the yaml that can be loaded by Compliance Operator.
@yuumasato
Copy link
Copy Markdown
Member Author

yuumasato commented Mar 25, 2026

Thanks for the review @xiaojiey.

Hopefully I have addessed the BuildConfig issue in the last commit.
Regarding the kubevirt rules, there is no need to thoroughly test them now. They won't be shipped downstream yet.

@yuumasato yuumasato added this to the 0.1.81 milestone Mar 25, 2026
@xiaojiey
Copy link
Copy Markdown
Collaborator

@yuumasato Sorry, I forgot to highlight, there is one more need to be updated. The --datastream-only in ocp-resources/ds-build-remote.yaml need to be removed. Otherwise, the cel content file won't be included.
The good news is once the cel file included, the pb will be created correctly with the right CELCONTENTFILE, no need to patch any more.
################# with --datastream-only parameter

$ ./build_product ocp4
$ ./utils/build_ds_container.py -c -p
2026-03-26 10:29:01,458:INFO: Building content for ocp4, rhcos4
...
2026-03-26 10:32:23,395:INFO: Build status: Running
2026-03-26 10:32:27,162:INFO: Build status: Failed
$ oc get pod
NAME                                              READY   STATUS    RESTARTS        AGE
compliance-operator-54dd495bbb-wpn2b              1/1     Running   2 (8m50s ago)   8m54s
ocp4-openshift-compliance-pp-57d9c88555-5ccpv     1/1     Running   0               8m37s
openscap-ocp4-ds-1-build                          0/1     Error     0               7m54s
rhcos4-openshift-compliance-pp-5d6fb55f67-lchn6   1/1     Running   0               8m37s
$ oc logs pod/openscap-ocp4-ds-1-build --all-containers 
[2/2] STEP 5/7: COPY --from=builder /content/build/ocp4-cel-content.yaml .
error: build error: building at STEP "COPY --from=builder /content/build/ocp4-cel-content.yaml .": checking on sources under "/var/lib/containers/storage/overlay/c6192ee843a64e2ea122abce26db828840cf621dec559a2eecd0a71bd40d0f13/merged": copier: stat: "/content/build/ocp4-cel-content.yaml": no such file or directory

################# without --datastream-only parameter

$ git diff
diff --git a/ocp-resources/ds-build-remote.yaml b/ocp-resources/ds-build-remote.yaml
index 68764deb8e..5f6ee69916 100644
--- a/ocp-resources/ds-build-remote.yaml
+++ b/ocp-resources/ds-build-remote.yaml
@@ -25,7 +25,7 @@ spec:
 
       RUN microdnf -y install cmake make git /usr/bin/python3 python3-pyyaml python3-jinja2 openscap-utils
 
-      RUN ./build_product --datastream-only --debug ocp4 rhcos4
+      RUN ./build_product --debug ocp4 rhcos4
 
       FROM registry.access.redhat.com/ubi8/ubi-minimal
       WORKDIR /
$ ./utils/build_ds_container.py -c -p
2026-03-26 11:39:54,642:INFO: Building content for ocp4, rhcos4
...
2026-03-26 11:43:10,806:INFO: Build status: Running
2026-03-26 11:43:15,440:INFO: Your image is available at image-registry.openshift-image-registry.svc:5000/openshift-compliance/openscap-ocp4-ds
2026-03-26 11:43:19,447:INFO: Created profile bundles for ocp4, rhcos4
$ oc get pod
NAME                                                      READY   STATUS      RESTARTS      AGE
compliance-operator-54dd495bbb-wpn2b                      1/1     Running     2 (76m ago)   76m
ocp4-openshift-compliance-pp-57d9c88555-5ccpv             1/1     Running     0             76m
openscap-ocp4-ds-3-build                                  0/1     Completed   0             4m56s
rhcos4-openshift-compliance-pp-5d6fb55f67-lchn6           1/1     Running     0             76m
upstream-ocp4-openshift-compliance-pp-7f856fd4-2m8q2      1/1     Running     0             100s
upstream-rhcos4-openshift-compliance-pp-f7d968bc6-prwvg   1/1     Running     0             98s
$ oc get pb 
NAME              CONTENTIMAGE                                 CONTENTFILE         CELCONTENTFILE          STATUS
ocp4              ghcr.io/complianceascode/k8scontent:latest   ssg-ocp4-ds.xml                             VALID
rhcos4            ghcr.io/complianceascode/k8scontent:latest   ssg-rhcos4-ds.xml                           VALID
upstream-ocp4     openscap-ocp4-ds:latest                      ssg-ocp4-ds.xml     ocp4-cel-content.yaml   VALID
upstream-rhcos4   openscap-ocp4-ds:latest                      ssg-rhcos4-ds.xml                           PENDING
$ oc get profiles -n openshift-compliance -o json | \
  jq -r '.items[] | select(.metadata.annotations["compliance.openshift.io/scanner-type"] == "CEL") | .metadata.name'
upstream-ocp4-cis-vm-extension
$ oc get rules -n openshift-compliance -o json | jq -r '.items[] | select(.scannerType == "CEL") | .metadata.name'
upstream-ocp4-kubevirt-enforce-trusted-tls-registries
upstream-ocp4-kubevirt-no-permitted-host-devices
upstream-ocp4-kubevirt-no-vms-overcommitting-guest-memory
upstream-ocp4-kubevirt-nonroot-feature-gate-is-enabled
upstream-ocp4-kubevirt-persistent-reservation-disabled
$ oc get profile upstream-ocp4-cis-vm-extension -o=jsonpath={.rules} | jq -r
[
  "upstream-ocp4-kubevirt-enforce-trusted-tls-registries",
  "upstream-ocp4-kubevirt-no-permitted-host-devices",
  "upstream-ocp4-kubevirt-no-vms-overcommitting-guest-memory",
  "upstream-ocp4-kubevirt-nonroot-feature-gate-is-enabled",
  "upstream-ocp4-kubevirt-persistent-reservation-disabled"
]

<pre>$ oc get hyperconverged kubevirt-hyperconverged -n openshift-cnv -o jsonpath='{.spec.featureGates.nonRoot}'</pre>
The output should be <tt>true</tt>.

checkType: Platform
Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I wonder if these should be part of new cel-scanner template similar to other rule in content repo

Copy link
Copy Markdown
Member Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't understand how you envision the template.
For now, the rules are just added to have some content to build.
We can certainly improve and optimize how the CEL rules work in future PRs.

Copy link
Copy Markdown
Member Author

@yuumasato yuumasato Mar 27, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Same thing with the profile file: products/ocp4/profiles/cis-vm-extension.profile

We are very likely migrating that to use a Control file, instead of listing individual rules.

Copy link
Copy Markdown
Contributor

@Vincent056 Vincent056 left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I think the PR looks good, just some questions on formatting and templating.

Adds --cel-content parameter that takes a comma separated list of
products to build cel-content for.

Add the new parameter with OCP4 product where it makes sense.
With addition of '--cel-content' as an option to build CEL content.
And with it being additional to data stream builds, having
'--datastream-only' parameter feels weird.

This add '--datastream' so that we can move away from
'--datastream-only' and be more consistent.
@yuumasato
Copy link
Copy Markdown
Member Author

@xiaojiey Thanks, instead of removing '--datastream-only' I have added a new parameter '--cel-content=ocp4'.
This way we don't unnecessarily build targets we don't need in our images.

@openshift-ci
Copy link
Copy Markdown

openshift-ci bot commented Mar 27, 2026

@yuumasato: The following test failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
ci/prow/4.18-images 73d1ff7 link true /test 4.18-images

Full PR test history. Your PR dashboard.

Details

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes-sigs/prow repository. I understand the commands that are listed here.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

OpenShift OpenShift product related.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants