diff --git a/.github/workflows/build.yml b/.github/workflows/build.yml index d3b5adb9b8..fa02ea6d51 100644 --- a/.github/workflows/build.yml +++ b/.github/workflows/build.yml @@ -17,7 +17,7 @@ jobs: build-MOAB: runs-on: ubuntu-24.04 steps: - - run: sudo apt-get install mpich libmpich-dev liblapack-dev + - run: sudo apt-get update && sudo apt-get install mpich libmpich-dev liblapack-dev - name: cache-MOAB id: cache-MOAB uses: actions/cache@v4 @@ -97,7 +97,7 @@ jobs: needs: [build-MOAB, build-Catch2, build-TCLAP, build-ZSTD] runs-on: ubuntu-24.04 steps: - - run: sudo apt-get install mpich doxygen libmpich-dev liblapack-dev libzstd1 libzstd-dev libtclap-dev protobuf-compiler libprotobuf-dev + - run: sudo apt-get update && sudo apt-get install mpich doxygen libmpich-dev liblapack-dev libzstd1 libzstd-dev libtclap-dev protobuf-compiler libprotobuf-dev - name: checkout uses: actions/checkout@v4 - name: fetch-MOAB diff --git a/.github/workflows/format.yml b/.github/workflows/format.yml index 700aaf484b..e1ec4efd52 100644 --- a/.github/workflows/format.yml +++ b/.github/workflows/format.yml @@ -7,7 +7,7 @@ jobs: clang-format: runs-on: ubuntu-22.04 steps: - - run: sudo apt-get install clang-format + - run: sudo apt-get update && sudo apt-get install clang-format - uses: actions/checkout@v2.3.4 with: repository: Sarcasm/run-clang-format diff --git a/CMakeLists.txt b/CMakeLists.txt index f435d00718..7c0e42f61e 100644 --- a/CMakeLists.txt +++ b/CMakeLists.txt @@ -47,7 +47,7 @@ option(MGARD_ENABLE_UNSTRUCTURED "Enable experimental unstructured mesh support. option(MGARD_ENABLE_TIMING "Enable timing." OFF) -option(MGARD_ENABLE_DATA_COMPATIBILITY "Also build the separate MGARD 1.6.0-compatible decompression pipeline (src/mgard-x-160) so data compressed by MGARD 1.6.x can be decompressed. Roughly doubles MGARD-X build time." OFF) +option(MGARD_ENABLE_DATA_COMPATIBILITY "Also build the separate MGARD 1.6.0-compatible decompression pipeline (src/mgard-x-160) so data compressed by MGARD 1.6.x can be decompressed. Adds build time and library size; turn OFF if no 1.6.x data needs to be read." ON) include(CTest) diff --git a/doc/MGARD-X.md b/doc/MGARD-X.md index d062a76c62..c26df1ff0b 100644 --- a/doc/MGARD-X.md +++ b/doc/MGARD-X.md @@ -85,13 +85,13 @@ MGARD-X is a portable implementation of the MGARD lossy compressor supporting va |MGARD\_ENABLE\_SYCL|ON/OFF|OFF|Enable portable GPU compression/decompression with SYCL on Intel GPUs| |MGARD\_ENABLE\_AUTO\_TUNING|ON/OFF|OFF|Enable auto tuning feature| |MGARD\_ENABLE\_MDR|ON/OFF|OFF|Enable building MDR and MDR-X. MPI is optional: the parallel ```pmdr-x-qoi``` and ```pmdr-x-qoi-io``` executables are built only when MPI is found| -|MGARD\_ENABLE\_DATA\_COMPATIBILITY|ON/OFF|OFF|Also build the MGARD 1.6.0-compatible decompression pipeline so that data compressed by MGARD 1.6.x and earlier can be decompressed (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)). Roughly doubles the MGARD-X build time| +|MGARD\_ENABLE\_DATA\_COMPATIBILITY|ON/OFF|ON|Also build the MGARD 1.6.0-compatible decompression pipeline so that data compressed by MGARD 1.6.x and earlier can be decompressed (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)). Adds build time and library size; turn OFF if no 1.6.x data needs to be read| ## Decompressing data from earlier versions Data compressed by MGARD 1.7.0 uses a different format from MGARD 1.6.x (hierarchy geometry, Huffman layout, default number of levels, and domain decomposition header). Decompression reads the software version recorded in the header of the compressed data and handles each version as follows: * **Compressed by 1.7.0:** decompressed by the 1.7.0 pipeline. -* **Compressed by 1.6.x or earlier:** decompressed by a separate pipeline that reproduces the MGARD 1.6.0 decompressor, if MGARD-X is built with ```-DMGARD_ENABLE_DATA_COMPATIBILITY=ON```. The result is the same as decompressing with MGARD 1.6.0 (bit-identical on CPUs). Without this option, decompression returns ```mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure```. +* **Compressed by 1.6.x or earlier:** decompressed by a separate pipeline that reproduces the MGARD 1.6.0 decompressor, which is built by default (```MGARD_ENABLE_DATA_COMPATIBILITY=ON```). The result is the same as decompressing with MGARD 1.6.0 (bit-identical on CPUs). If MGARD-X is built with ```-DMGARD_ENABLE_DATA_COMPATIBILITY=OFF```, decompression returns ```mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure```. This is transparent to users: the same CLI and high-level APIs decompress both formats, and decompression needs no extra configuration. Compression always produces the 1.7.0 format. Data compressed by MGARD 1.6.x with the variable-size domain decomposition (```domain_decomposition_type::Variable```) is not supported, because its block sizes are not recorded in the header. @@ -230,7 +230,7 @@ An executable ```mgard-x``` will be built after building the MGARD-X library. To + mgard_x::compress_status_type::NotSupportHigherNumberOfDimensionsFailure + mgard_x::compress_status_type::NotSupportDataTypeFailure + mgard_x::compress_status_type::BackendNotAvailableFailure - + mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure: the data was compressed by MGARD 1.6.x or earlier and MGARD-X was built without ```MGARD_ENABLE_DATA_COMPATIBILITY``` (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)) + + mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure: the data was compressed by MGARD 1.6.x or earlier and MGARD-X was built with ```MGARD_ENABLE_DATA_COMPATIBILITY=OFF``` (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)) ## Using low-level APIs * **Step 1: Build Hierarchy** diff --git a/doc/release notes/1.7.0.md b/doc/release notes/1.7.0.md index 6cc1cd7f25..eecd9f954d 100644 --- a/doc/release notes/1.7.0.md +++ b/doc/release notes/1.7.0.md @@ -9,7 +9,7 @@ * A BlockDelta (FLE-based) lossless backend for mgard-x, as a Huffman alternative. * A portable, warp-cooperative LZ4 implementation for mgard-x, removing the nvcomp dependency. * Blackwell (sm_120) GPU build support. -* Decompression of data compressed by MGARD 1.6.x and earlier, through a separate MGARD 1.6.0-compatible decompression pipeline enabled with the new CMake option `MGARD_ENABLE_DATA_COMPATIBILITY` (default OFF). Decompression detects the version from the header and routes 1.6.x data automatically; output matches MGARD 1.6.0 (bit-identical on CPUs). Builds without the option report such data with the new status `DataCompatibilityNotEnabledFailure`. +* Decompression of data compressed by MGARD 1.6.x and earlier, through a separate MGARD 1.6.0-compatible decompression pipeline enabled with the new CMake option `MGARD_ENABLE_DATA_COMPATIBILITY` (default ON). Decompression detects the version from the header and routes 1.6.x data automatically; output matches MGARD 1.6.0 (bit-identical on CPUs). Builds with the option OFF report such data with the new status `DataCompatibilityNotEnabledFailure`. * Generic block domain decomposition with two modes (`Config::domain_decomposition_mode`): `Auto` (default) decomposes only when the data does not fit the memory limit and picks the block size, and `Manual` uses a user-given block shape (`Config::domain_decomposition_block_size`), which may differ per dimension. Available in the `mgard-x` and `mdr-x` CLIs as `-dd auto|manual` and `-dd-size`. * `MemoryManager::Copy3D` in RuntimeX (native 3D copies on CUDA and HIP), used for copying blocks between host and device in fewer transfers. * Opt-in ZSTD compression of MDR-X bitplane groups (`lossless = Huffman_Zstd`; `mdr-x -l huffman-zstd`).