Skip to content
Draft
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 4 additions & 0 deletions CHANGELOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -8,6 +8,9 @@ All notable changes to this project will be documented in this file.

- airflow, superset, druid, nifi: Add SBOMs for the frontend (npm) dependencies ([#1600]).
- nifi: Backport NIFI-15958 to log periodic progress while waiting for the content archive scan and provenance re-index, for `2.6.0`, `2.7.2`, and `2.9.0` ([#1611]).
- hbase: Add an SBOM for the web UI (npm) dependencies, which are unpacked from webjars and therefore not covered by the CycloneDX Maven plugin ([#1620]).
- trino: Add SBOMs for the web UI, both for the two npm projects behind it and for the pre-built JavaScript vendored into the source tree ([#1620]).
- hadoop, spark: Add SBOMs for the pre-built JavaScript that is vendored into the source tree for the HDFS and Spark web UIs ([#1620]).

### Changed

Expand All @@ -29,6 +32,7 @@ All notable changes to this project will be documented in this file.
[#1600]: https://github.com/stackabletech/docker-images/pull/1600
[#1611]: https://github.com/stackabletech/docker-images/pull/1611
[#1616]: https://github.com/stackabletech/docker-images/pull/1616
[#1620]: https://github.com/stackabletech/docker-images/pull/1620

## [26.7.0] - 2026-07-21

Expand Down
7 changes: 7 additions & 0 deletions boil.toml
Original file line number Diff line number Diff line change
Expand Up @@ -6,6 +6,13 @@ DELETE_CACHES = "true"
# CycloneDX specification version used for the SBOMs generated by cdxgen.
# 1.6 is the lowest version cdxgen 13 accepts as a generation target.
CDXGEN_SPEC_VERSION = "1.6"
# Node version used to run cdxgen in the builders that need it. It is unrelated to any product
# and to the Node version that a product uses to build its frontend, so it is configured once
# here instead of per product version. Products that pin `cdxgen-nodejs-version` in their own
# boil-config.toml override this value.
# Find the latest release here: https://github.com/nodejs/node/releases
# renovate: datasource=node-version packageName=node
CDXGEN_NODEJS_VERSION = "24.19.0"

[metadata]
documentation = "https://docs.stackable.tech/home/stable/"
Expand Down
20 changes: 18 additions & 2 deletions hadoop/hadoop/Dockerfile
Original file line number Diff line number Diff line change
Expand Up @@ -12,6 +12,7 @@ ARG AZURE_STORAGE_VERSION
ARG AZURE_KEYVAULT_CORE_VERSION
ARG ANALYTICSACCELERATOR_S3_VERSION
ARG STACKABLE_USER_UID
ARG CDXGEN_SPEC_VERSION

WORKDIR /stackable

Expand All @@ -21,8 +22,9 @@ COPY --chown=${STACKABLE_USER_UID}:0 shared/protobuf/stackable/patches/${PROTOBU
RUN <<EOF
microdnf update
# libstdc++ is a runtime dependency for boost,
# automake and libtool are needed to build protobuf
microdnf install libstdc++ automake libtool
# automake and libtool are needed to build protobuf,
# python3 runs shared/sbom/vendored_js.py, see the invocation further down
microdnf install libstdc++ automake libtool python3
microdnf clean all
rm -rf /var/cache/yum
mkdir /opt/protobuf
Expand Down Expand Up @@ -67,6 +69,8 @@ WORKDIR /build
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/hadoop/stackable/patches/patchable.toml /build/src/hadoop/hadoop/stackable/patches/patchable.toml
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/hadoop/stackable/patches/${PRODUCT_VERSION} /build/src/hadoop/hadoop/stackable/patches/${PRODUCT_VERSION}
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/hadoop/stackable/fuse_dfs_wrapper /build
COPY --chown=${STACKABLE_USER_UID}:0 shared/sbom/vendored_js.py /build/vendored_js.py
COPY --chown=${STACKABLE_USER_UID}:0 hadoop/hadoop/stackable/vendored-js/${PRODUCT_VERSION}.json /build/vendored-js.json
USER ${STACKABLE_USER_UID}
# Hadoop Pipes requires libtirpc to build, whose headers are not packaged in RedHat UBI, so skip building this module
# Build from source to enable FUSE module, and to apply custom patches.
Expand Down Expand Up @@ -115,6 +119,18 @@ cp -r hadoop-dist/target/hadoop-${NEW_VERSION} /stackable/hadoop-${NEW_VERSION}
sed -i "s/${NEW_VERSION}/${ORIGINAL_VERSION}/g" hadoop-dist/target/bom.json
mv hadoop-dist/target/bom.json /stackable/hadoop-${NEW_VERSION}/hadoop-${NEW_VERSION}.cdx.json

# Create the SBOM of the third-party JavaScript that is checked into the source tree, mostly
# for the HDFS web UI. These are pre-built files without any package manifest, so no SBOM
# generator can discover them and they are instead recorded in vendored-js.json. That manifest
# pins the SHA-256 of every file it describes, so this fails the build if a Hadoop upgrade
# changed any of them and the recorded versions have to be revisited.
python3 /build/vendored_js.py bom \
/build/vendored-js.json \
. \
"/stackable/hadoop-${NEW_VERSION}/hadoop-webapps-${NEW_VERSION}.cdx.json" \
"${ORIGINAL_VERSION}" \
"${CDXGEN_SPEC_VERSION}"

# HDFS fuse-dfs is not part of the regular dist output, so we need to copy it in ourselves
cp hadoop-hdfs-project/hadoop-hdfs-native-client/target/main/native/fuse-dfs/fuse_dfs /stackable/hadoop-${NEW_VERSION}/bin

Expand Down
121 changes: 121 additions & 0 deletions hadoop/hadoop/stackable/vendored-js/3.3.6.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,121 @@
{
"name": "hadoop-webapps",
"scan-dirs": [
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps",
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps",
"hadoop-tools/hadoop-sls/src/main/html"
],
"libraries": [
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"sha256": "dbd2a35e72edc7d6bde483481a912f1c38aa57fab2747d9b071d317339ee03a2"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.min.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"sha256": "9ee2fcff6709e4d0d24b09ca0fc56aade12b4961ed9c43fd13b03248bfb57afe"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/npm.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"note": "Part of the Bootstrap distribution, it only requires the other Bootstrap files.",
"sha256": "c7aa82a1aa7d45224a38d926d2adaff7fe4aef5bcdafa2a47bdac057f4422c2d"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap-editable.min.js",
"purl": "pkg:npm/x-editable@1.5.0",
"license": "MIT",
"note": "Version taken from the file header. 1.5.0 was never published to npm, but advisories are matched against version ranges, so the npm purl is still the useful identity.",
"sha256": "8e4041866b100f3afe72c70c1dd5d6405729ba0e327f07e14c4023d9d657753c"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/d3-v4.1.1.min.js",
"purl": "pkg:npm/d3@4.1.0",
"license": "BSD-3-Clause",
"note": "The file name claims 4.1.1, but the file is byte-identical to build/d3.min.js of npm d3 4.1.0 and its header says 4.1.0.",
"sha256": "3181906226d79dd39f94bfb7267fce2d077809cc3a23f23e87ea3f02ffae799d"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-full-2.0.0.min.js",
"purl": "pkg:npm/dustjs-linkedin@2.0.1",
"license": "MIT",
"note": "The file is byte-identical in dustjs-linkedin 2.0.1 through 2.2.10, so the release it was taken from cannot be determined by hashing and the lowest is recorded. The file name claims 2.0.0, which is the version of the dust build it contains rather than a release that ships this file.",
"sha256": "f11ba668337b8b61319b430164f631648b41949887bb8a7b9cc515f87bba3e3b"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-helpers-1.1.1.min.js",
"purl": "pkg:npm/dustjs-helpers@1.1.1",
"license": "MIT",
"note": "Version taken from the file name. It matches no published dustjs-helpers tarball, so it is either modified or predates the npm releases.",
"sha256": "ff65ffc9e919f9ab7922d82db9ea9d7840a7543001ccba2a8c4f11195a08a7f6"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-3.6.0.min.js",
"purl": "pkg:npm/jquery@3.6.0",
"license": "MIT",
"sha256": "80f04717f32ea0320c5e8618fbacedd1fee3a8775ad8292140a6113551d4b5b0"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery.dataTables.min.js",
"purl": "pkg:npm/datatables.net@1.10.7",
"license": "MIT",
"sha256": "7a101ba1668e04321dd15acb478546de82bea05c8887749c8532427577e5df7a"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dataTables.bootstrap.js",
"purl": null,
"name": "datatables.net-bs",
"note": "The DataTables Bootstrap 3 integration. It carries no version and matches no published tarball, so it is recorded without a version.",
"sha256": "205a054048f80cde368d980a57a66112d2f548c89b8f7a0202b23d06ba624627"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/moment.min.js",
"purl": "pkg:npm/moment@2.22.1",
"license": "MIT",
"sha256": "853e11d64268a12da71524bc7e1bb1f960243f3eee045f7839f796f1d23670e0"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/json-bignum.js",
"purl": null,
"name": "json-bignum",
"note": "Vendored from https://github.com/datalanche/json-bignum without a version and modified, so it matches no published tarball.",
"sha256": "243931ed9a223e3eabb3d7634d3ae9c735f5aa2ec6c5ba6ff54ce18c5dc47236"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js",
"purl": "pkg:npm/jquery@3.5.1",
"license": "MIT",
"sha256": "416a3b2c3bf16d64f6b5b6d0f7b079df2267614dd6847fc2f3271b4409233c37"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js",
"purl": "pkg:npm/d3@3.2.7",
"license": "BSD-3-Clause",
"note": "Version taken from the file header. It matches no published d3 tarball, so it is probably a custom build.",
"sha256": "2085bd03d15690b448e136c590d6982ed7397e730407fed1d414304c81761315"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js",
"purl": null,
"name": "bootstrap",
"license": "Apache-2.0",
"note": "Bootstrap 2.x, which was Apache-2.0 licensed. The file carries no version and matches no published tarball, so no version is recorded. Its header says \"Copyright 2012 Twitter, Inc.\".",
"sha256": "eabb9d96942adad6cbfbf964a4fe53c5bc585dd330cb829665ee15bbf2ca4f1d"
}
],
"own": [
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/explorer.js",
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/dn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/explorer.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/jn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/snn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dfs-dust.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/rest-csrf.js"
]
}
122 changes: 122 additions & 0 deletions hadoop/hadoop/stackable/vendored-js/3.4.2.json
Original file line number Diff line number Diff line change
@@ -0,0 +1,122 @@
{
"name": "hadoop-webapps",
"scan-dirs": [
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps",
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps",
"hadoop-tools/hadoop-sls/src/main/html"
],
"libraries": [
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"sha256": "dbd2a35e72edc7d6bde483481a912f1c38aa57fab2747d9b071d317339ee03a2"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap.min.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"sha256": "9ee2fcff6709e4d0d24b09ca0fc56aade12b4961ed9c43fd13b03248bfb57afe"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/npm.js",
"purl": "pkg:npm/bootstrap@3.4.1",
"license": "MIT",
"note": "Part of the Bootstrap distribution, it only requires the other Bootstrap files.",
"sha256": "c7aa82a1aa7d45224a38d926d2adaff7fe4aef5bcdafa2a47bdac057f4422c2d"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/bootstrap-3.4.1/js/bootstrap-editable.min.js",
"purl": "pkg:npm/x-editable@1.5.0",
"license": "MIT",
"note": "Version taken from the file header. 1.5.0 was never published to npm, but advisories are matched against version ranges, so the npm purl is still the useful identity.",
"sha256": "8e4041866b100f3afe72c70c1dd5d6405729ba0e327f07e14c4023d9d657753c"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/d3-v4.1.1.min.js",
"purl": "pkg:npm/d3@4.1.0",
"license": "BSD-3-Clause",
"note": "The file name claims 4.1.1, but the file is byte-identical to build/d3.min.js of npm d3 4.1.0 and its header says 4.1.0.",
"sha256": "3181906226d79dd39f94bfb7267fce2d077809cc3a23f23e87ea3f02ffae799d"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-full-2.0.0.min.js",
"purl": "pkg:npm/dustjs-linkedin@2.0.1",
"license": "MIT",
"note": "The file is byte-identical in dustjs-linkedin 2.0.1 through 2.2.10, so the release it was taken from cannot be determined by hashing and the lowest is recorded. The file name claims 2.0.0, which is the version of the dust build it contains rather than a release that ships this file.",
"sha256": "f11ba668337b8b61319b430164f631648b41949887bb8a7b9cc515f87bba3e3b"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dust-helpers-1.1.1.min.js",
"purl": "pkg:npm/dustjs-helpers@1.1.1",
"license": "MIT",
"note": "Version taken from the file name. It matches no published dustjs-helpers tarball, so it is either modified or predates the npm releases.",
"sha256": "ff65ffc9e919f9ab7922d82db9ea9d7840a7543001ccba2a8c4f11195a08a7f6"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery-3.6.0.min.js",
"purl": "pkg:npm/jquery@3.6.0",
"license": "MIT",
"sha256": "80f04717f32ea0320c5e8618fbacedd1fee3a8775ad8292140a6113551d4b5b0"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/jquery.dataTables.min.js",
"purl": "pkg:npm/datatables.net@1.11.5",
"license": "MIT",
"sha256": "2e288f534e4f2a5b1f4d17cc62149068ab6ef1a8dac45832b387e000719f28a3"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dataTables.bootstrap.js",
"purl": null,
"name": "datatables.net-bs",
"note": "The DataTables Bootstrap 3 integration. It carries no version and matches no published tarball, so it is recorded without a version.",
"sha256": "205a054048f80cde368d980a57a66112d2f548c89b8f7a0202b23d06ba624627"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/moment.min.js",
"purl": "pkg:npm/moment@2.29.4",
"license": "MIT",
"sha256": "9aec203698e15111ceda22d99911f578505e16d9dd92f17be8d31ca4e29f6990"
},
{
"file": "hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/json-bignum.js",
"purl": null,
"name": "json-bignum",
"note": "Vendored from https://github.com/datalanche/json-bignum without a version and modified, so it matches no published tarball.",
"sha256": "243931ed9a223e3eabb3d7634d3ae9c735f5aa2ec6c5ba6ff54ce18c5dc47236"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/jquery.js",
"purl": "pkg:npm/jquery@3.5.1",
"license": "MIT",
"sha256": "416a3b2c3bf16d64f6b5b6d0f7b079df2267614dd6847fc2f3271b4409233c37"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/d3.v3.js",
"purl": "pkg:npm/d3@3.2.7",
"license": "BSD-3-Clause",
"note": "Version taken from the file header. It matches no published d3 tarball, so it is probably a custom build.",
"sha256": "2085bd03d15690b448e136c590d6982ed7397e730407fed1d414304c81761315"
},
{
"file": "hadoop-tools/hadoop-sls/src/main/html/js/thirdparty/bootstrap.min.js",
"purl": null,
"name": "bootstrap",
"license": "Apache-2.0",
"note": "Bootstrap 2.x, which was Apache-2.0 licensed. The file carries no version and matches no published tarball, so no version is recorded. Its header says \"Copyright 2012 Twitter, Inc.\".",
"sha256": "eabb9d96942adad6cbfbf964a4fe53c5bc585dd330cb829665ee15bbf2ca4f1d"
}
],
"own": [
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/explorer.js",
"hadoop-hdfs-project/hadoop-hdfs-rbf/src/main/webapps/router/federationhealth.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/datanode/dn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/dfshealth.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/hdfs/explorer.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/journal/jn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/secondary/snn.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/dfs-dust.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/histogram-hostip.js",
"hadoop-hdfs-project/hadoop-hdfs/src/main/webapps/static/rest-csrf.js"
]
}
Loading