Skip to content

HDDS-15618. Fix ListObjects to include single whitespace-only S3 object keys#10589

Merged
ivandika3 merged 2 commits into
apache:masterfrom
Gargi-jais11:HDDS-15618
Jun 29, 2026
Merged

HDDS-15618. Fix ListObjects to include single whitespace-only S3 object keys#10589
ivandika3 merged 2 commits into
apache:masterfrom
Gargi-jais11:HDDS-15618

Conversation

@Gargi-jais11

Copy link
Copy Markdown
Contributor

What changes were proposed in this pull request?

s3-tests test_bucket_create_special_key_names fails because a key named **' '** is created but not returned by ListObjects. OmMetadataManagerImpl.getOzoneKey() uses StringUtils.isNotBlank(), which treats whitespace-only keys as absent, so the object is stored under /volume/bucket instead of /volume/bucket/.
List object uses prefix /volume/bucket/, so the key is omitted. Fix key encoding to allow whitespace-only names

Expected: [' ', '"', '$', '%', '&', "'", '<', '>', '_', '_ ', '_ _', '__']
Actual:   ['"', '$', '%', '&', "'", '<', '>', '_', '_ ', '_ _'] 

  # missing leading ' '

At index 0: ' ' != '"' 

What is the link to the Apache JIRA

https://issues.apache.org/jira/browse/HDDS-15618

How was this patch tested?

Added unit test.

@Gargi-jais11 Gargi-jais11 marked this pull request as ready for review June 23, 2026 08:28
@Gargi-jais11 Gargi-jais11 added the s3 S3 Gateway label Jun 23, 2026

@priyeshkaratha priyeshkaratha left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Gargi-jais11 for the patch. Changes LGTM

@ivandika3 ivandika3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Gargi-jais11 , left one comment.

@chungen0126

Copy link
Copy Markdown
Contributor

Thanks @Gargi-jais11 for working on this.

As far as I know, HDFS doesn't support filenames with spaces. I'm concerning about how this change impact that. Have you looked into it?

@Gargi-jais11

Copy link
Copy Markdown
Contributor Author

Thanks @Gargi-jais11 for working on this.

As far as I know, HDFS doesn't support filenames with spaces. I'm concerning about how this change impact that. Have you looked into it?

I have tested creating hdfs filenames with spaces and it does support. Evidence:

[datanode -1 ~]# echo "test" | hdfs dfs -put - "/tmp/ "

[datanode-1 ~]# hdfs dfs -ls "/tmp" | cat -A
Found 1 items$
-rw-r--r--   3 hdfs supergroup          5 2026-06-29 05:25 /tmp/ $

and even ozone supports creating key with space :

ozone sh key put /vol1/buck1/" " test
[datanode-1 ~]# ozone sh key list /vol1/buck1
[ {
  "volumeName" : "vol1",
  "bucketName" : "buck1",
  "name" : " ",
  "owner" : "om",
  "dataSize" : 11,
  "creationTime" : "2026-06-29T05:28:46.616Z",
  "modificationTime" : "2026-06-29T05:28:48.179Z",
  "replicationConfig" : {
    "replicationFactor" : "THREE",
    "requiredNodes" : 3,
    "minimumNodes" : 1,
    "replicationType" : "RATIS"
  },
  "metadata" : { },
  "tags" : { },
  "file" : true
} ]

The S3 issue was not “Ozone/HDFS rejects space keys.” It was a create vs list mismatch caused by a bug in how OM builds the internal metadata key. OM treated " " as “no key segment” and stored it at /vol1/buck1 instead of /vol1/buck1/.

@Gargi-jais11 Gargi-jais11 marked this pull request as draft June 29, 2026 05:49
@Gargi-jais11 Gargi-jais11 marked this pull request as ready for review June 29, 2026 07:42
@Gargi-jais11 Gargi-jais11 requested a review from ivandika3 June 29, 2026 07:42

@ivandika3 ivandika3 left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks @Gargi-jais11 for the update. LGTM +1.

@ivandika3 ivandika3 merged commit d1eee36 into apache:master Jun 29, 2026
90 of 91 checks passed
@ivandika3

Copy link
Copy Markdown
Contributor

Thanks @Gargi-jais11 for the patch and @chungen0126 for the review.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

s3 S3 Gateway

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants