Skip to content

Conversation

@t20100
Copy link
Member

@t20100 t20100 commented Sep 24, 2025

This PR aims at updating SZ3 to the current latest version (3.3.0).

As it is, it seg fault when reading data written with the previous version of the library:

python test/test.py terminate with:

testSZ3 (__main__.TestHDF5PluginRead.testSZ3)
libc++abi: terminating with uncaught exception of type std::invalid_argument: magic number mismatch, the input data is not compressed by SZ3

This is due to an incompatible change in the cd_values stored in the hdf5 file.

With v3.3.0, the cd_values are a dump of "SZ3::Config", saved by Config::save.
See those links for save to/load from hdf5 cd_values:

Related to #351

5d2fd2638 SZ3.3  (silx-kit#94)
dd09ba727 Rename the flag ENABLE_GSL into SZ3_ENABLE_GSL (silx-kit#100)
f600741d1 Fix NaN handling in the linear quantizer (silx-kit#91)
c91eea70f start (silx-kit#96)
8ac77c58e Merge remote-tracking branch 'origin/master'
054967f8c bugfix hdf5
cd4a2611b Merge pull request silx-kit#90 from szcompressor/JLiu-1-patch-1
21c08473e Update README.md
e8a6b1569 Merge pull request silx-kit#82 from szcompressor/JLiu-1-patch-1
ead8d62a6 Add QoZ description
8bcf24c52 update demo
854c4bf59 Merge pull request silx-kit#74 from rroohhh/msvc_build
81c2818ee fixup! add ALWAYS_INLINE
89df80e54 bump to 3.2.2
1c46e0765 Merge pull request silx-kit#73 from juntyr/patch-1
f67f91411 Guarantee that decompress doesn't modify the compressed data
88095a8bb clear build warning
45786ed54 fixbug for sz size bound
7ddd65dda add static_assert
db63591e3 add ALWAYS_INLINE
f1258442b improve build speed when SZ3_DEBUG_TIMINGS is on
2ddfac9b0 update logic for buffer size checking
07cee4b95 update logic for buffer size checking
95d906be9 update logic for buffer size checking
f7b9432ee update logic for buffer size checking
d1aaa33e9 update logic for buffer size checking
634add9d1 update logic for dispatcher
780870a04 Merge pull request silx-kit#70 from juntyr/patch-1
0f3dde778 Ensure lossless fallback always has enough space
cebae4d1f Fix UB for small array compression
10ebed0f8 update logic for exceptions
29234fd91 update logic for lossless dispatch
4dafde041 improved error handing: throw exception and print to stderr instead of exit(0)
49a2694cf bugfix
00c672882 checking if compression mode is valid
8564514f1 bugfix for regression predictor
76a2d0a88 update hdf5
e79c03a46 bugfix for HDF5
68716ebc0 bugfix for openmp
61816ba12 1. update the logic to check data version 2. dispatcher logic change. SZ3 will enter lossless only mode automatically when needed 3. bug fix, see comments in Lossless_zstd.hpp
9bd968ff4 1. update the logic to check data version 2. dispatcher logic change. SZ3 will enter lossless only mode automatically when needed 3. bug fix, see comments in Lossless_zstd.hpp
1d8179986 1. update annotations 2. fix make warnings
67be543fd 1. update annotations 2. fix make warnings
18b44fafb 1. update annotations 2. fix make warnings
7105dd71a 1. update annotations 2. fix make warnings
c7a23a95a 1. update annotations 2. fix make warnings
26e172083 1. update annotations 2. fix make warnings
876a3ca92 1. update annotations 2. fix make warnings
dbd73bb87 1. move version check to config.load() 2. config.save() now use less space 3. update h5z-sz3 with readme and compact cd_value array
160e3e4f1 1. move version check to config.load() 2. config.save() now use less space 3. update h5z-sz3 with readme and compact cd_value array
03062dc40 1. move version check to config.load() 2. config.save() now use less space 3. update h5z-sz3 with readme and compact cd_value array
8a6220da6 1. move version check to config.load() 2. config.save() now use less space 3. update h5z-sz3 with readme and compact cd_value array
a952c77dd add clang-format
b3dab4018 Merge pull request silx-kit#60 from szcompressor/newapi
d70c22264 update readme
35115beeb update readme
352c1a0f6 update compress api
fe6eef7e3 update compress api
75c906dda update compress api
ad47f20d2 update compress api
52b4f439c update compress api
7a5447c79 update compress api
cb87901dd Merge remote-tracking branch 'origin/newapi' into newapi
c1d6504e1 update hdf5 filter
533292f9f Merge branch 'refs/heads/master' into newapi
b3dac0c65 update readme
3f478178d Merge pull request silx-kit#56 from juntyr/32bit-hash
f8fc2cb33 fix int overflow in quantization
e52a2c30f Add support for 32bit systems by falling back to std::unordered_map
f6cae37f5 add link to fortran api
90c66bed1 modify MDZ CR calculation to include the first frame which is stored losslessly.
09494b13c fix bug in MDZ for 1D inputs
dde630245 add noprediction compressor
47593621f bugfix
fdd6dee6e API rewrite data version check
a887550ed API rewrite data version check
5bab19c5d API rewrite data version check
1ea5d5b07 API rewrite data version check
e78f4832f Add annotation
be68d6cbe bug fix for openmp
800c3248e bug fix for interp
d06710a55 bug fix for interp
7fc2cb519 bug fix for MDZ
3568914cd Merge remote-tracking branch 'origin/master'
5f835fb74 bug fix for MDZ
be68d645b Merge pull request silx-kit#47 from robertu94/master
e308ebf85 Merge branch 'master' of github.com:szcompressor/sz3
6198db715 default initalize config
162674f87 Merge pull request silx-kit#46 from mrmbernardi/patch-1
f834756f9 Fix SZFastFrontend::size_est() returning undefined value
13e79072e config bugfix
0ac567a76 Merge pull request silx-kit#45 from szcompressor/hdf5
d6beca327 add config support to hdf5 filter
05965ac2b update 1d lorenzo fast mode
d2a03eae4 add lorenzo 1D fast mode
fa9a7f179 Merge pull request silx-kit#44 from szcompressor/namespace
2437574dc change namespace from SZ to SZ3, to avoid conflict with SZ2
3236a6eb2 Merge branch 'huffman'
5995fa9f2 update HDF5 filter
1fc15171c update HDF5 filter
aadaaeb90 change namespace from SZ to SZ3, to avoid conflict with SZ2
e1af6bea2 change namespace from SZ to SZ3, to avoid conflict with SZ2
bdc4b503b new huffman
9aee288dd Merge pull request silx-kit#43 from mrmbernardi/decompress_output
ab7896ec4 update stateNum in preprocess_encode
44839697c Decompress function returns output matching original_dtype
f420554d3 update readme
31d7cbb6a add annotation for interfaces
d0606d3ae Merge pull request silx-kit#41 from szcompressor/mdz
3443ae347 update readme
1219a336d disable SZ3_DEBUG_TIMINGS by default
09d43dee5 clear runtime output for MDZ
f051a8361 support 3D input for MDZ
89f3b7c78 bugfix in smoke test
5a1cf1aad Merge remote-tracking branch 'origin/master'
40e9b3de5 bugfix in smoke test
140e09e9c remove the print information (OpenMP threads = ), which might be annoying and seems unneeded.
1e1dc102e Merge pull request silx-kit#38 from aumuell/uint8
9b2bf8ad5 fix missing uint8_t with GCC 13
f7d865e6e Merge pull request silx-kit#37 from robertu94/hdf5_fix
73674f3db link to MPI when hdf5 is parallel
13f835d54 openmp bugfix
318fe953f openmp bugfix
612b4993d update pysz
bce7f1202 Merge pull request silx-kit#32 from exmakhina/master
62d01de0b python: load library from a sensible default path, if not provided
ce7fb031e change sprintf to snprintf
1b1abcf39 interp+lorenzo sampling bugfix
9f396935d bugfix

git-subtree-dir: lib/SZ3
git-subtree-split: 5d2fd2638bc3512c0b4aed594d6e8a130d0d3ed2
@t20100 t20100 mentioned this pull request Sep 24, 2025
4 tasks
@t20100
Copy link
Member Author

t20100 commented Sep 25, 2025

The format of cd_values has changed, this implies either:

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant