Repository navigation
Opening a zarr behind a S3Config as YAXArray fails #69
Description
When I try to open a zarr that needs credentials and a config as a YAXArray this fails because open_dataset tries to open the zarr with a store string and an empty path.
This is on current main of everything with zarr.jl at 0.11
This failed differently on previous versions but I think that worked at some point.
using Zarr
using YAXArrays
using AWSS3
cfg = AWSS3.AWS.AWSConfig(creds=nothing, region="us-west-2")
z = AWSS3.AWS.with_aws_config(cfg) do # This works
zopen("s3://mur-sst/zarr-v1")
end
yax = AWSS3.AWS.with_aws_config(cfg) do # This fails
open_dataset("s3://mur-sst/zarr-v1")
end
z = AWSS3.AWS.with_aws_config(cfg) do # This fails and is what open_dataset tries to open the zarr
zopen("s3://mur-sst/zarr-v1", "")
end
# This fails because of missing credentials
zwocfg = zopen("s3://mur-sst/zarr-v1")
@time yaxz = open_dataset(z) # This works but needs 13 seconds with 13 lock conflictsError Message
```julia julia> yax = AWSS3.AWS.with_aws_config(cfg) do open_dataset("s3://mur-sst/zarr-v1") end ERROR: ArgumentError: Specified store S3 Object Storage with: config: AWS.AWSConfig(nothing, "us-west-2", "json", 3) bucket: mur-sst in path is neither a ZArray nor a ZGroup in a recognized zarr format. Stacktrace: [1] ZarrCore.ZarrFormat(s::S3Store, path::String) @ ZarrCore ~/Documents/scratch/zarrs3/dev/Zarr/ZarrCore/src/ZGroup.jl:45 [2] #zopen#99 @ ~/Documents/scratch/zarrs3/dev/Zarr/ZarrCore/src/ZGroup.jl:109 [inlined] [3] kwcall(::@NamedTuple{path::String, fill_as_missing::Bool}, ::typeof(zopen), s::S3Store,mode::String) @ ZarrCore ~/Documents/scratch/zarrs3/dev/Zarr/ZarrCore/src/ZGroup.jl:102 [4] zopen(s::String, mode::String; kwargs::@kwargs{fill_as_missing::Bool, path::String}) @ ZarrCore ~/Documents/scratch/zarrs3/dev/Zarr/ZarrCore/src/ZGroup.jl:129 [5] zopen @ ~/Documents/scratch/zarrs3/dev/Zarr/ZarrCore/src/ZGroup.jl:127 [inlined] [6] ZarrExt.ZarrDataset(g::String; mode::String, path::String, kwargs::@kwargs{}) @ ZarrExt ~/Documents/scratch/zarrs3/dev/YAXArrayBase/ext/ZarrExt.jl:27 [7] ZarrDataset @ ~/Documents/scratch/zarrs3/dev/YAXArrayBase/ext/ZarrExt.jl:18 [inlined] [8] to_dataset(T::Type{ZarrExt.ZarrDataset}, g::String; kwargs::@kwargs{}) @ YAXArrayBase ~/Documents/scratch/zarrs3/dev/YAXArrayBase/src/datasets/datasetinterface.jl:101 [9] to_dataset @ ~/Documents/scratch/zarrs3/dev/YAXArrayBase/src/datasets/datasetinterface.jl:98 [inlined] [10] open_dataset(g::String; skip_keys::Tuple{}, driver::Symbol, force_datetime::Bool, path::Nothing) @ YAXArrays.Datasets ~/Documents/scratch/zarrs3/dev/YAXArrays/src/DatasetAPI/Datasets.jl:464 [11] open_dataset @ ~/Documents/scratch/zarrs3/dev/YAXArrays/src/DatasetAPI/Datasets.jl:462 [inlined] [12] #41 @ ./REPL[36]:2 [inlined] [13] with_aws_config(f::var"#41#42", config::AWS.AWSConfig) @ AWS ~/.julia/packages/AWS/Igcxn/src/AWSConfig.jl:166 [14] top-level scope @ REPL[36]:1
</p>
</details>
Activity
lazarusA commentedon Sep 30, 2026
maybe related to JuliaIO/Zarr.jl#309 (comment)