Error Reading Sharded Zarrs on S3

Python zarr version: 3.2.1
zarr-java version: 0.1.3

Sharded zarrs which can be read without issue locally fail when the same data is read from S3 with a checksum of the sharding index is invalid error.

Example

I created a simple 2D zarr with the following python code:

  size = 1024
  zarrcube = zarr.create_array(
      store='/path/to/testsquare.zarr',
      shape=(size, size),
      shards=(size//4, size//4),
      chunks=(size//16, size//16),
      dtype='uint8')
  zarrcube[:, :] = np.random.randint(0, 256, (size, size))

I then uploaded the data to a publicly-readable S3 bucket in us-east-1 using the following commands:

cd testsquare.zarr
aws s3 sync . 's3://my-public-zarr-bucket/shardtest/testsquare.zarr'

I then attempted to read both the local and remote files using the following java code:

        String localPath = "/path/to/testsquare.zarr";

        URI endpoint = new URI("https://s3.us-east-1.amazonaws.com");

        S3ClientBuilder clientBuilder =  S3Client.builder()
                .httpClientBuilder(UrlConnectionHttpClient.builder()
                        .socketTimeout(Duration.ofMinutes(5)));
        clientBuilder.endpointOverride(endpoint);
        clientBuilder.region(Region.US_EAST_1);

        S3Configuration s3Config = S3Configuration.builder().pathStyleAccessEnabled(true)
            .build();
        clientBuilder.serviceConfiguration(s3Config);

        clientBuilder.credentialsProvider(AnonymousCredentialsProvider.create());

        S3Client client = clientBuilder.build();

        S3Store store = new S3Store(client, "gs-public-zarr-dev", "shardtest");
        Array s3Array = Array.open(store.resolve("testsquare.zarr"));

        Array localArray = Array.open(localPath);

        for (int i = 0; i < 10; i++) {
            long [] offset = new long[] {100l*i, 100l*i};
            long [] shape = new long[] {100l, 100l};
            localArray.read(offset, shape);
            s3Array.read(offset, shape);
        }

The following exception was thrown:

java.lang.RuntimeException: dev.zarr.zarrjava.ZarrException: The checksum of the sharding index is invalid. Stored: 1384733839 Computed: -246033701
	at dev.zarr.zarrjava.core.Array.lambda$read$2(Array.java:437)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.accept(ForEachOps.java:183)
	at java.base/java.util.Spliterators$ArraySpliterator.forEachRemaining(Spliterators.java:992)
	at java.base/java.util.stream.AbstractPipeline.copyInto(AbstractPipeline.java:509)
	at java.base/java.util.stream.ForEachOps$ForEachTask.compute(ForEachOps.java:290)
	at java.base/java.util.concurrent.CountedCompleter.exec(CountedCompleter.java:754)
	at java.base/java.util.concurrent.ForkJoinTask.doExec(ForkJoinTask.java:373)
	at java.base/java.util.concurrent.ForkJoinTask.invoke(ForkJoinTask.java:686)
	at java.base/java.util.stream.ForEachOps$ForEachOp.evaluateParallel(ForEachOps.java:159)
	at java.base/java.util.stream.ForEachOps$ForEachOp$OfRef.evaluateParallel(ForEachOps.java:173)
	at java.base/java.util.stream.AbstractPipeline.evaluate(AbstractPipeline.java:233)
	at java.base/java.util.stream.ReferencePipeline.forEach(ReferencePipeline.java:596)
	at java.base/java.util.stream.ReferencePipeline$Head.forEach(ReferencePipeline.java:765)
	at dev.zarr.zarrjava.core.Array.read(Array.java:407)
	at dev.zarr.zarrjava.core.Array.read(Array.java:344)
	at com.glencoesoftware.omero.zarr.ZarrPixelBufferTest.testSharding(ZarrPixelBufferTest.java:336)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke0(Native Method)
	at java.base/jdk.internal.reflect.NativeMethodAccessorImpl.invoke(NativeMethodAccessorImpl.java:77)
	at java.base/jdk.internal.reflect.DelegatingMethodAccessorImpl.invoke(DelegatingMethodAccessorImpl.java:43)
	at java.base/java.lang.reflect.Method.invoke(Method.java:569)
	at org.junit.runners.model.FrameworkMethod$1.runReflectiveCall(FrameworkMethod.java:59)
	at org.junit.internal.runners.model.ReflectiveCallable.run(ReflectiveCallable.java:12)
	at org.junit.runners.model.FrameworkMethod.invokeExplosively(FrameworkMethod.java:56)
	at org.junit.internal.runners.statements.InvokeMethod.evaluate(InvokeMethod.java:17)
	at org.junit.rules.ExternalResource$1.evaluate(ExternalResource.java:54)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.BlockJUnit4ClassRunner$1.evaluate(BlockJUnit4ClassRunner.java:100)
	at org.junit.runners.ParentRunner.runLeaf(ParentRunner.java:366)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:103)
	at org.junit.runners.BlockJUnit4ClassRunner.runChild(BlockJUnit4ClassRunner.java:63)
	at org.junit.runners.ParentRunner$4.run(ParentRunner.java:331)
	at org.junit.runners.ParentRunner$1.schedule(ParentRunner.java:79)
	at org.junit.runners.ParentRunner.runChildren(ParentRunner.java:329)
	at org.junit.runners.ParentRunner.access$100(ParentRunner.java:66)
	at org.junit.runners.ParentRunner$2.evaluate(ParentRunner.java:293)
	at org.junit.runners.ParentRunner$3.evaluate(ParentRunner.java:306)
	at org.junit.runners.ParentRunner.run(ParentRunner.java:413)
	at org.eclipse.jdt.internal.junit4.runner.JUnit4TestReference.run(JUnit4TestReference.java:93)
	at org.eclipse.jdt.internal.junit.runner.TestExecution.run(TestExecution.java:40)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:520)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.runTests(RemoteTestRunner.java:748)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.run(RemoteTestRunner.java:443)
	at org.eclipse.jdt.internal.junit.runner.RemoteTestRunner.main(RemoteTestRunner.java:211)
Caused by: dev.zarr.zarrjava.ZarrException: The checksum of the sharding index is invalid. Stored: 1384733839 Computed: -246033701
	at dev.zarr.zarrjava.v3.codec.core.Crc32cCodec.decode(Crc32cCodec.java:40)
	at dev.zarr.zarrjava.core.codec.CodecPipeline.decode(CodecPipeline.java:114)
	at dev.zarr.zarrjava.v3.codec.core.ShardingIndexedCodec.decodeInternal(ShardingIndexedCodec.java:205)
	at dev.zarr.zarrjava.v3.codec.core.ShardingIndexedCodec.decodePartial(ShardingIndexedCodec.java:254)
	at dev.zarr.zarrjava.core.codec.CodecPipeline.decodePartial(CodecPipeline.java:95)
	at dev.zarr.zarrjava.core.Array.lambda$read$2(Array.java:422)
	... 42 more

Please let me know if there's any other information you need from me.

I tested with the following versions and received the same error:

0.0.10
0.1.0
0.1.1
0.1.2
0.1.3

Made an interesting observation while testing. We encountered this error when attempting to retrieve image data from a couple different images on S3. One was s3://gs-public-zarr-dev/zarrv3/CMU-1-Sharded.zarr. The interesting thing about this example was that usually data was retrieved without issue, but sometimes it failed with the checksum error. What I've determined through testing is that you can successfully retrieve entire shards from S3, but any subdivision of a shard fails. With my toy example above, this means that you can do

long[] offset = new long[] {256, 512};
long[] shape = new long[] {512, 256};
s3Array.read(offset, shape);

and it will work just fine. But change any of those values by 1 and it will throw an exception.

Fixed by #80

Navigation Menu

Repository navigation

Error Reading Sharded Zarrs on S3 #79

Example

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Error Reading Sharded Zarrs on S3 #79

Description

Example

Activity

kkoz commented on Jun 30, 2026

kkoz commented on Jul 1, 2026

kkoz commented on Aug 31, 2026

Metadata

Metadata

Assignees

Labels

Type

Projects

Milestone

Relationships

Development

Issue actions