The growing importance of software supply chain security has driven widespread adoption of Software Bill of Materials (SBOM) as a foundational artifact for transparency and risk management.
Software Composition Analysis (SCA) is the primary technology used to identify third-party components within software and to automate SBOM generation. While a wide variety of SCA tools are available for SBOM generation, detection accuracy remains a recognized challenge across the field, and further improvement is needed.
A first step toward such improvement is a publicly available dataset for objective evaluation of SBOM content.
SCA-Benchmark-Dataset addresses this need by providing a controlled, reproducible benchmark artifact for binary-level SCA evaluation — techniques that analyze compiled artifacts rather than source code or package manifests. A curated set of open-source C/C++ libraries is compiled from source and statically linked into a single ELF binary with a precisely defined ground truth composition, enabling objective measurement of detection accuracy across SCA tools.
- Ubuntu 24.04 LTS (x86_64)
sudo apt-get install -y \
build-essential \
cmake \
python3 \
perl \
flex| Package | Required by |
|---|---|
| build-essential | gcc, g++, make |
| cmake | Most libraries |
| python3 | builder.py / assemble.py |
| perl | OpenSSL's Configure script |
| flex | libpcap (grammar lexer generation) |
sudo apt-get install -y \
gcc-aarch64-linux-gnu \
g++-aarch64-linux-gnusudo apt-get install -y \
gcc-arm-linux-gnueabihf \
g++-arm-linux-gnueabihfEnsure source archives are present under source/<library>/.
The expected path for each archive is listed in the archive field of libraries.json.
# x86_64 (native)
python3 scripts/builder.py
# arm64 (cross-compile)
python3 scripts/builder.py --target arm64
# arm32 (cross-compile)
python3 scripts/builder.py --target arm32| Option | Default | Description |
|---|---|---|
--target {x86_64,arm64,arm32} |
x86_64 |
Target architecture |
--opt-level {O0,O1,O2,O3,Os,Oz} |
O0 |
Compiler optimization level |
--no-debug |
off | Omit -g (debug info) from compiler flags |
# x86_64
python3 scripts/assemble.py
# arm64
python3 scripts/assemble.py --target arm64
# arm32
python3 scripts/assemble.py --target arm32The binary is written to output/benchmark-<target>.elf.
| Option | Default | Description |
|---|---|---|
--target {x86_64,arm64,arm32} |
x86_64 |
Target architecture |
--opt-level {O0,O1,O2,O3,Os,Oz} |
O0 |
Compiler optimization level for stub |
--no-debug |
off | Omit -g (debug info) from stub compilation flags |
--output <filename> |
benchmark-<target>.elf |
Output ELF filename |
# O2 with debug info
python3 scripts/builder.py --opt-level O2
python3 scripts/assemble.py --opt-level O2 --output benchmark-x86_64-O2-debug.elf
# O2 without debug info
python3 scripts/builder.py --opt-level O2 --no-debug
python3 scripts/assemble.py --opt-level O2 --no-debug --output benchmark-x86_64-O2-nodebug.elf