Fix: ZarrAvgMerger ValueError with zarr_format 3 #8477

kolasaniv1996 · 2025-06-09T19:28:16Z

This PR fixes the issue with ZarrAvgMerger when using zarr v3, where the compressor argument is not supported.

Changes:

Added deprecated_arg decorator for compressor, value_compressor, and count_compressor (since 1.5.0, to be removed in 1.7.0)
Added codecs, value_codecs, and count_codecs support for zarr format 3
Updated tests to handle both zarr v2 and zarr v3 compatibility
Fixed issue where tests would fail with zarr v3 due to compressor usage

This addresses the issue reported in #8476 where using compressor with zarr v3 causes ValueError.

Signed-off-by: kolasaniv1996 [email protected]

kolasaniv1996 · 2025-06-09T19:43:37Z

@KumoLiu @ericspod @Nic-Ma Please review the changes when you get a chance

monai/inferers/merger.py

ericspod · 2025-06-10T13:20:40Z

My understanding was that this argument was left in place to be compatible with zarr 2 as well as zarr 3. There was discussion on this here. A related issue raised in #8476 in our tests occurs when using zarr 3 with our tests, it would be nice to integrate a fix in this test by adding logic to remove this argument here and replace it with a use of the codecs argument (I think).

KumoLiu · 2025-06-12T03:15:35Z

Hi @kolasaniv1996, I wanted to check in to see if you’re planning to complete this PR. It seems it might be a blocker for our v1.5 release. Let me know if there's anything we can do to assist. Thanks.

kolasaniv1996 · 2025-06-12T03:49:43Z

@KumoLiu I'm working on it

Signed-off-by: kolasaniv1996 <[email protected]>

- Added deprecated_arg decorator for compressor, value_compressor, and count_compressor (since 1.5.0, to be removed in 1.7.0) - Added codecs, value_codecs, and count_codecs support for zarr format 3 - Updated tests to handle both zarr v2 and zarr v3 compatibility - Fixed issue where tests would fail with zarr v3 due to compressor usage Signed-off-by: kolasaniv1996 <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: kolasaniv1996 <[email protected]>

KumoLiu · 2025-06-12T09:03:01Z

CI error log:

======================================================================
ERROR: test_deprecated_compressor_warning (tests.inferers.test_zarr_avg_merger.ZarrAvgMergerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 562, in _init_array_metadata
    compressor_config = compressor.get_config()
AttributeError: 'str' object has no attribute 'get_config'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/MONAI/MONAI/tests/inferers/test_zarr_avg_merger.py", line 412, in test_deprecated_compressor_warning
    ZarrAvgMerger(merged_shape=TENSOR_4x4.shape, compressor="LZ4")
  File "/home/runner/work/MONAI/MONAI/monai/utils/module.py", line 477, in _wrapper
    return call_obj(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/inferers/merger.py", line 349, in __init__
    self.output = zarr.empty(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 317, in empty
    return create(shape=shape, fill_value=None, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 209, in create
    init_array(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 455, in init_array
    _init_array_metadata(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 564, in _init_array_metadata
    raise BadCompressorError(compressor) from e
zarr.errors.BadCompressorError: bad compressor; expected Codec object, found 'LZ4'

======================================================================
ERROR: test_deprecated_count_compressor_warning (tests.inferers.test_zarr_avg_merger.ZarrAvgMergerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 562, in _init_array_metadata
    compressor_config = compressor.get_config()
AttributeError: 'str' object has no attribute 'get_config'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/MONAI/MONAI/tests/inferers/test_zarr_avg_merger.py", line 430, in test_deprecated_count_compressor_warning
    ZarrAvgMerger(merged_shape=TENSOR_4x4.shape, count_compressor="LZ4")
  File "/home/runner/work/MONAI/MONAI/monai/utils/module.py", line 477, in _wrapper
    return call_obj(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/inferers/merger.py", line 365, in __init__
    self.counts = zarr.zeros(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 338, in zeros
    return create(shape=shape, fill_value=0, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 209, in create
    init_array(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 455, in init_array
    _init_array_metadata(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 564, in _init_array_metadata
    raise BadCompressorError(compressor) from e
zarr.errors.BadCompressorError: bad compressor; expected Codec object, found 'LZ4'

======================================================================
ERROR: test_deprecated_value_compressor_warning (tests.inferers.test_zarr_avg_merger.ZarrAvgMergerTests)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 562, in _init_array_metadata
    compressor_config = compressor.get_config()
AttributeError: 'str' object has no attribute 'get_config'

The above exception was the direct cause of the following exception:

Traceback (most recent call last):
  File "/home/runner/work/MONAI/MONAI/tests/inferers/test_zarr_avg_merger.py", line 421, in test_deprecated_value_compressor_warning
    ZarrAvgMerger(merged_shape=TENSOR_4x4.shape, value_compressor="LZ4")
  File "/home/runner/work/MONAI/MONAI/monai/utils/module.py", line 477, in _wrapper
    return call_obj(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/utils/deprecate_utils.py", line 223, in _wrapper
    return func(*args, **kwargs)
  File "/home/runner/work/MONAI/MONAI/monai/inferers/merger.py", line 357, in __init__
    self.values = zarr.zeros(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 338, in zeros
    return create(shape=shape, fill_value=0, **kwargs)
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/creation.py", line 209, in create
    init_array(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 455, in init_array
    _init_array_metadata(
  File "/opt/hostedtoolcache/Python/3.9.23/x64/lib/python3.9/site-packages/zarr/storage.py", line 564, in _init_array_metadata
    raise BadCompressorError(compressor) from e
zarr.errors.BadCompressorError: bad compressor; expected Codec object, found 'LZ4'

…ons and updating test assertions Signed-off-by: kolasaniv1996 <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: kolasaniv1996 <[email protected]>

…ects Signed-off-by: kolasaniv1996 <[email protected]>

for more information, see https://pre-commit.ci

Signed-off-by: vivek kolasani <[email protected]>

Signed-off-by: YunLiu <[email protected]>

KumoLiu · 2025-06-13T07:42:53Z

Hi @kolasaniv1996 and @ericspod, I try to fix the ci issue in this commit: cac8923
Let me know if you have any further concern.

BTW @kolasaniv1996, could you please follow this guide to fix the DCO issue: https://github.com/Project-MONAI/MONAI/pull/8477/checks?check_run_id=44029533563? Thanks!

Signed-off-by: YunLiu <[email protected]>

for more information, see https://pre-commit.ci

tests/inferers/test_zarr_avg_merger.py

Signed-off-by: YunLiu <[email protected]>

tests/inferers/test_zarr_avg_merger.py

Signed-off-by: YunLiu <[email protected]>

for more information, see https://pre-commit.ci

KumoLiu · 2025-06-13T15:39:27Z

/build

kolasaniv1996 requested review from KumoLiu, ericspod and Nic-Ma as code owners June 9, 2025 19:28

KumoLiu reviewed Jun 10, 2025

View reviewed changes

monai/inferers/merger.py Outdated Show resolved Hide resolved

kolasaniv1996 force-pushed the fix-zarr-compressor branch from c7177c3 to b09f93a Compare June 12, 2025 04:52

Fix: ZarrAvgMerger ValueError with zarr_format 3

708aec8

Signed-off-by: kolasaniv1996 <[email protected]>

kolasaniv1996 force-pushed the fix-zarr-compressor branch from 02bc380 to 708aec8 Compare June 12, 2025 05:12

kolasaniv1996 and others added 3 commits June 12, 2025 02:06

[pre-commit.ci] auto fixes from pre-commit.com hooks

7e6ae2d

for more information, see https://pre-commit.ci

Trigger CI to verify formatting fixes

eaf4ecc

Signed-off-by: kolasaniv1996 <[email protected]>

kolasaniv1996 and others added 4 commits June 12, 2025 11:15

Fix zarr v3 compatibility in tests by adding proper codec configurati…

c5cb6b9

…ons and updating test assertions Signed-off-by: kolasaniv1996 <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

11dd177

for more information, see https://pre-commit.ci

Trigger CI to verify all fixes

4dd4d12

Signed-off-by: kolasaniv1996 <[email protected]>

Fix: ZarrAvgMerger tests for zarr v2 compatibility by using Codec obj…

389b300

…ects Signed-off-by: kolasaniv1996 <[email protected]>

kolasaniv1996 force-pushed the fix-zarr-compressor branch 2 times, most recently from 93d2146 to 389b300 Compare June 12, 2025 17:37

pre-commit-ci bot and others added 9 commits June 12, 2025 17:37

[pre-commit.ci] auto fixes from pre-commit.com hooks

afd89e2

for more information, see https://pre-commit.ci

Merge branch 'dev' into fix-zarr-compressor

5e36aa2

Fix: DeprecationWarning message check in ZarrAvgMerger tests

528a48f

[pre-commit.ci] auto fixes from pre-commit.com hooks

40d26ed

for more information, see https://pre-commit.ci

Fix: E501 line too long errors in test_zarr_avg_merger.py

586c8d4

Fix: Apply isort autofix for import sorting and formatting

56fa3dd

Merge branch 'Project-MONAI:dev' into fix-zarr-compressor

24840e5

Fix value assignment

6117348

Signed-off-by: vivek kolasani <[email protected]>

Update _WITH_NAN

edd1cf3

Signed-off-by: vivek kolasani <[email protected]>

kolasaniv1996 and others added 2 commits June 13, 2025 00:41

cleanup type checks

43fc5e4

Signed-off-by: vivek kolasani <[email protected]>

fix ci

cac8923

Signed-off-by: YunLiu <[email protected]>

KumoLiu and others added 3 commits June 13, 2025 15:54

fix mypy

3db8a50

Signed-off-by: YunLiu <[email protected]>

fix ci

ec27f00

Signed-off-by: YunLiu <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

102bc90

for more information, see https://pre-commit.ci

KumoLiu reviewed Jun 13, 2025

View reviewed changes

tests/inferers/test_zarr_avg_merger.py Outdated Show resolved Hide resolved

KumoLiu added 2 commits June 13, 2025 23:11

Merge remote-tracking branch 'origin/dev' into fix-zarr-compressor

f6b65bc

remove warning test

01fa8eb

Signed-off-by: YunLiu <[email protected]>

KumoLiu reviewed Jun 13, 2025

View reviewed changes

tests/inferers/test_zarr_avg_merger.py Outdated Show resolved Hide resolved

KumoLiu and others added 2 commits June 13, 2025 23:37

fix unsupported codecs in zarr v3

bac24fb

Signed-off-by: YunLiu <[email protected]>

[pre-commit.ci] auto fixes from pre-commit.com hooks

5e2f67f

for more information, see https://pre-commit.ci

KumoLiu enabled auto-merge (squash) June 13, 2025 15:39

KumoLiu approved these changes Jun 13, 2025

View reviewed changes

KumoLiu merged commit 3d6021a into Project-MONAI:dev Jun 13, 2025
25 checks passed

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

Uh oh!

Fix: ZarrAvgMerger ValueError with zarr_format 3 #8477

Fix: ZarrAvgMerger ValueError with zarr_format 3 #8477

Uh oh!

kolasaniv1996 commented Jun 9, 2025 •

edited

Loading

Uh oh!

kolasaniv1996 commented Jun 9, 2025

Uh oh!

Uh oh!

ericspod commented Jun 10, 2025

Uh oh!

KumoLiu commented Jun 12, 2025

Uh oh!

kolasaniv1996 commented Jun 12, 2025

Uh oh!

KumoLiu commented Jun 12, 2025

Uh oh!

KumoLiu commented Jun 13, 2025 •

edited

Loading

Uh oh!

Uh oh!

Uh oh!

KumoLiu commented Jun 13, 2025

Uh oh!

Uh oh!

Uh oh!

Fix: ZarrAvgMerger ValueError with zarr_format 3 #8477

Fix: ZarrAvgMerger ValueError with zarr_format 3 #8477

Uh oh!

Conversation

kolasaniv1996 commented Jun 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

kolasaniv1996 commented Jun 9, 2025

Uh oh!

Uh oh!

ericspod commented Jun 10, 2025

Uh oh!

KumoLiu commented Jun 12, 2025

Uh oh!

kolasaniv1996 commented Jun 12, 2025

Uh oh!

KumoLiu commented Jun 12, 2025

Uh oh!

KumoLiu commented Jun 13, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

Uh oh!

KumoLiu commented Jun 13, 2025

Uh oh!

Uh oh!

Uh oh!

kolasaniv1996 commented Jun 9, 2025 •

edited

Loading

KumoLiu commented Jun 13, 2025 •

edited

Loading