Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
4 changes: 2 additions & 2 deletions .github/workflows/build.yml
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@ jobs:
build-MOAB:
runs-on: ubuntu-24.04
steps:
- run: sudo apt-get install mpich libmpich-dev liblapack-dev
- run: sudo apt-get update && sudo apt-get install mpich libmpich-dev liblapack-dev
- name: cache-MOAB
id: cache-MOAB
uses: actions/cache@v4
Expand Down Expand Up @@ -97,7 +97,7 @@ jobs:
needs: [build-MOAB, build-Catch2, build-TCLAP, build-ZSTD]
runs-on: ubuntu-24.04
steps:
- run: sudo apt-get install mpich doxygen libmpich-dev liblapack-dev libzstd1 libzstd-dev libtclap-dev protobuf-compiler libprotobuf-dev
- run: sudo apt-get update && sudo apt-get install mpich doxygen libmpich-dev liblapack-dev libzstd1 libzstd-dev libtclap-dev protobuf-compiler libprotobuf-dev
- name: checkout
uses: actions/checkout@v4
- name: fetch-MOAB
Expand Down
2 changes: 1 addition & 1 deletion .github/workflows/format.yml
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ jobs:
clang-format:
runs-on: ubuntu-22.04
steps:
- run: sudo apt-get install clang-format
- run: sudo apt-get update && sudo apt-get install clang-format
- uses: actions/checkout@v2.3.4
with:
repository: Sarcasm/run-clang-format
Expand Down
2 changes: 1 addition & 1 deletion CMakeLists.txt
Original file line number Diff line number Diff line change
Expand Up @@ -47,7 +47,7 @@ option(MGARD_ENABLE_UNSTRUCTURED "Enable experimental unstructured mesh support.

option(MGARD_ENABLE_TIMING "Enable timing." OFF)

option(MGARD_ENABLE_DATA_COMPATIBILITY "Also build the separate MGARD 1.6.0-compatible decompression pipeline (src/mgard-x-160) so data compressed by MGARD 1.6.x can be decompressed. Roughly doubles MGARD-X build time." OFF)
option(MGARD_ENABLE_DATA_COMPATIBILITY "Also build the separate MGARD 1.6.0-compatible decompression pipeline (src/mgard-x-160) so data compressed by MGARD 1.6.x can be decompressed. Adds build time and library size; turn OFF if no 1.6.x data needs to be read." ON)

include(CTest)

Expand Down
6 changes: 3 additions & 3 deletions doc/MGARD-X.md
Original file line number Diff line number Diff line change
Expand Up @@ -85,13 +85,13 @@ MGARD-X is a portable implementation of the MGARD lossy compressor supporting va
|MGARD\_ENABLE\_SYCL|ON/OFF|OFF|Enable portable GPU compression/decompression with SYCL on Intel GPUs|
|MGARD\_ENABLE\_AUTO\_TUNING|ON/OFF|OFF|Enable auto tuning feature|
|MGARD\_ENABLE\_MDR|ON/OFF|OFF|Enable building MDR and MDR-X. MPI is optional: the parallel ```pmdr-x-qoi``` and ```pmdr-x-qoi-io``` executables are built only when MPI is found|
|MGARD\_ENABLE\_DATA\_COMPATIBILITY|ON/OFF|OFF|Also build the MGARD 1.6.0-compatible decompression pipeline so that data compressed by MGARD 1.6.x and earlier can be decompressed (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)). Roughly doubles the MGARD-X build time|
|MGARD\_ENABLE\_DATA\_COMPATIBILITY|ON/OFF|ON|Also build the MGARD 1.6.0-compatible decompression pipeline so that data compressed by MGARD 1.6.x and earlier can be decompressed (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions)). Adds build time and library size; turn OFF if no 1.6.x data needs to be read|

## Decompressing data from earlier versions
Data compressed by MGARD 1.7.0 uses a different format from MGARD 1.6.x (hierarchy geometry, Huffman layout, default number of levels, and domain decomposition header). Decompression reads the software version recorded in the header of the compressed data and handles each version as follows:

* **Compressed by 1.7.0:** decompressed by the 1.7.0 pipeline.
* **Compressed by 1.6.x or earlier:** decompressed by a separate pipeline that reproduces the MGARD 1.6.0 decompressor, if MGARD-X is built with ```-DMGARD_ENABLE_DATA_COMPATIBILITY=ON```. The result is the same as decompressing with MGARD 1.6.0 (bit-identical on CPUs). Without this option, decompression returns ```mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure```.
* **Compressed by 1.6.x or earlier:** decompressed by a separate pipeline that reproduces the MGARD 1.6.0 decompressor, which is built by default (```MGARD_ENABLE_DATA_COMPATIBILITY=ON```). The result is the same as decompressing with MGARD 1.6.0 (bit-identical on CPUs). If MGARD-X is built with ```-DMGARD_ENABLE_DATA_COMPATIBILITY=OFF```, decompression returns ```mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure```.

This is transparent to users: the same CLI and high-level APIs decompress both formats, and decompression needs no extra configuration. Compression always produces the 1.7.0 format. Data compressed by MGARD 1.6.x with the variable-size domain decomposition (```domain_decomposition_type::Variable```) is not supported, because its block sizes are not recorded in the header.

Expand Down Expand Up @@ -230,7 +230,7 @@ An executable ```mgard-x``` will be built after building the MGARD-X library. To
+ mgard_x::compress_status_type::NotSupportHigherNumberOfDimensionsFailure
+ mgard_x::compress_status_type::NotSupportDataTypeFailure
+ mgard_x::compress_status_type::BackendNotAvailableFailure
+ mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure: the data was compressed by MGARD 1.6.x or earlier and MGARD-X was built without ```MGARD_ENABLE_DATA_COMPATIBILITY``` (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions))
+ mgard_x::compress_status_type::DataCompatibilityNotEnabledFailure: the data was compressed by MGARD 1.6.x or earlier and MGARD-X was built with ```MGARD_ENABLE_DATA_COMPATIBILITY=OFF``` (see [Decompressing data from earlier versions](#decompressing-data-from-earlier-versions))

## Using low-level APIs
* **Step 1: Build Hierarchy**
Expand Down
2 changes: 1 addition & 1 deletion doc/release notes/1.7.0.md
Original file line number Diff line number Diff line change
Expand Up @@ -9,7 +9,7 @@
* A BlockDelta (FLE-based) lossless backend for mgard-x, as a Huffman alternative.
* A portable, warp-cooperative LZ4 implementation for mgard-x, removing the nvcomp dependency.
* Blackwell (sm_120) GPU build support.
* Decompression of data compressed by MGARD 1.6.x and earlier, through a separate MGARD 1.6.0-compatible decompression pipeline enabled with the new CMake option `MGARD_ENABLE_DATA_COMPATIBILITY` (default OFF). Decompression detects the version from the header and routes 1.6.x data automatically; output matches MGARD 1.6.0 (bit-identical on CPUs). Builds without the option report such data with the new status `DataCompatibilityNotEnabledFailure`.
* Decompression of data compressed by MGARD 1.6.x and earlier, through a separate MGARD 1.6.0-compatible decompression pipeline enabled with the new CMake option `MGARD_ENABLE_DATA_COMPATIBILITY` (default ON). Decompression detects the version from the header and routes 1.6.x data automatically; output matches MGARD 1.6.0 (bit-identical on CPUs). Builds with the option OFF report such data with the new status `DataCompatibilityNotEnabledFailure`.
* Generic block domain decomposition with two modes (`Config::domain_decomposition_mode`): `Auto` (default) decomposes only when the data does not fit the memory limit and picks the block size, and `Manual` uses a user-given block shape (`Config::domain_decomposition_block_size`), which may differ per dimension. Available in the `mgard-x` and `mdr-x` CLIs as `-dd auto|manual` and `-dd-size`.
* `MemoryManager::Copy3D` in RuntimeX (native 3D copies on CUDA and HIP), used for copying blocks between host and device in fewer transfers.
* Opt-in ZSTD compression of MDR-X bitplane groups (`lossless = Huffman_Zstd`; `mdr-x -l huffman-zstd`).
Expand Down
Loading