Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

tst_specific_filters.sh fails from "Testing processing of filter blosc for map nc 10c10" for 4.9.2 #2868

Closed
bzfbmsch opened this issue Feb 22, 2024 · 3 comments

Comments

@bzfbmsch
Copy link

Dear all,
Building netcdf-c 4.92 from source. I downloaded the released tar.gz-file, not the from git.
The system is openSUSE 15.5, with gcc-11, prerequisites hdf5-14.3 + plugins. The plugins include blosc.

I am interested to learn more on the deflation tools besides libz and have installed several libraries including their development parts. May be I am a little bit lazy, but I am just using those from the linux distribution openSUSE 15.5. The critical tool is the libblosc, I have version 1.21.1-bp155.2.1.7. I have also build several hdf5-plugins that includes blosc. The plugin is also linked with the system version of libblosc. After configure and make, I end up in make check with the same error like that reported in ticket: #2572:

FAIL: tst_specific_filters.sh
...... and in the logfile:

findplugin.sh loaded
*** Testing processing of filter deflate for map nc
*** Testing processing of filter szip for map nc
*** Testing processing of filter bzip2 for map nc
*** Testing processing of filter blosc for map nc
10c10
< ivar:_Filter = "32001,0,0,4,256,5,1,1" ;

          ivar:_Filter = "32001,2,2,4,256,5,1,1" ;

14c14
< fvar:_Filter = "32001,0,0,4,256,5,1,1" ;

          fvar:_Filter = "32001,2,2,4,256,5,1,1" ;

Inspired from the discussion found in the ticket, I tried a workaround. I edited tst_specific_filters.sh and changed line 18:

if test "x$TESTNCZARR" = x1 ; then
. "$srcdir/test_nczarr.sh"
BLOSCARGS="32001,0,0,0,256,5,1,1"
BLOSCCODEC='[{"id": "blosc","clevel": 5,"blocksize": 256,"cname": "lz4","shuffle": 1}]'
else
here>>>>>>>>>>>>>>>>>>>>>
BLOSCARGS="32001,2,2,4,256,5,1,1"
BLOSCCODEC='[{"id": "blosc","clevel": 5,"blocksize": 256,"cname": "lz4","shuffle": 1}]'
fi

With this change, the error message was not displayed any more, but I am not sure that the error is really gone. May be, I have added the wrong result as referene.

Here some more details on the configuration after the configure step:
---->>>>
Shared Library: yes
Static Library: yes
Extra libraries: -lm -lz -ldl -lzip -lsz -lbz2 -lzstd -lblosc -lxml2 -lcurl
XML Parser: libxml2

Features


Benchmarks: no
NetCDF-2 API: yes
HDF4 Support: no
HDF5 Support: yes
NetCDF-4 API: yes
CDF5 Support: yes
NC-4 Parallel Support: no
PnetCDF Support: no

DAP2 Support: yes
DAP4 Support: yes
Byte-Range Support: yes

S3 Support: no

NCZarr Support: yes
NCZarr Zip Support: yes

Diskless Support: yes
MMap Support: yes
JNA Support: yes
ERANGE Fill Support: yes
Relaxed Boundary Check: yes

Multi-Filter Support: yes
Quantization: yes
Logging: no
SZIP Write Support: yes
Standard Filters: deflate bz2 blosc zstd
ZSTD Support: yes
Parallel Filters: yes
----->>>

I am not sure, if this is sufficient for an error report. If needed, I can deliver all details on the compiler and the configure flags. For the moment, any hints for a clean solution are very welcome.

Kind regards,

Martin Schmidt

@dirteat
Copy link

dirteat commented Sep 2, 2024

+1

@DennisHeimbigner
Copy link
Collaborator

I believe the problem is that the parameters for libblosc differ.
I am not entirely sure who is responsible for the differences: HDF5, OS (e.g. opensus vs ubuntu), or libblosc.
Perhaps the correct thing to do is to effectively ignore those two differing parameters since they are not
really important. I will investigate.

DennisHeimbigner added a commit to DennisHeimbigner/netcdf-c that referenced this issue Oct 29, 2024
re: Issue Unidata#2868

There are three parameters (zero, one, and two) that are currently
used in testing, but should not be. Parameters zero and one
are ignored by the code and parameter two is the typesize
parameter that either takes on the value 0 or 4 (== sizeof(int)).
See nc_test4/tst_specific_filters.sh.

Unfortunately, these three parameters differ depending on the blosc library.
The solution is to force them to a specific value -- namely 0,0,0 --
in the test script.
@DennisHeimbigner
Copy link
Collaborator

See PR #3046

@WardF WardF closed this as completed Oct 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

4 participants