Repository navigation
Reposition the temp file when seeking to offset zero - #363
Merged
Merged
Conversation
copyToLocalTempReader copies the existing file in with io.Copy, which leaves the temp file at EOF, then seeks back only when cursorPos > 0. So Seek(0, io.SeekStart) followed by Write appended instead of overwriting from the start. The mem backend already behaves correctly.
The minio/minio repository has been removed from Docker Hub, so the testcontainers conformance CI job failed to provision the s3 backend container: pull access denied for minio/minio, repository does not exist or may require 'docker login': denied: requested access to the resource is denied quay.io/minio/minio publishes the identical pinned tag (RELEASE.2025-09-07T16-13-09Z), so this is a drop-in replacement. Verified locally with: TESTCONTAINERS_RYUK_DISABLED=true go test -tags=vfsintegration -race -timeout 30m ./... The minio container now provisions correctly.
…ntent initWriters gated the download-existing-content step on cursorPos != 0, so Seek(0, io.SeekStart) followed by Write skipped downloading the existing object entirely. On Close, the object was replaced with only the newly written bytes, silently discarding everything else - matching os's bug fixed above, but via a different mechanism (the existing-object download is skipped outright here, rather than left at the wrong cursor position after a copy). The gate now checks whether Seek or Read was called before this first Write, not whether the resulting cursor happens to be zero - mirroring the os backend's fix and the vfs.File contract (Write-first always overwrites; Seek/Read-first preserves untouched content). Fixes C2FO#365. Added regression tests for both backends: - s3: TestSeekToStartThenWriteDownloadsExistingContent, mirroring the existing TestSeekThenWriteDownloadsExistingContent (offset 6) but at offset 0. - gs: TestSeekThenWrite, mirroring the os backend's table-driven test (offset 0 and offset 6) using the in-process fakestorage server. Both new tests were verified to fail against the pre-fix code (exact symptom: object ends up containing only the newly-written bytes) and pass with the fix. Also fixes a test-hygiene issue found in review of the os fix: the existing os TestSeekThenWrite reset a shared fixture file (test_files/test.txt) with a non-deferred call at the end of each subtest, so a failed assertion earlier in a subtest would leave that shared file mutated for later subtests/suite tests. Switched to a fixture-local file per subtest with a deferred cleanup instead. Verified locally end-to-end with the testcontainers integration suite (TESTCONTAINERS_RYUK_DISABLED=true go test -tags=vfsintegration ./...): the shared 'Seek to start, Write, Close, file exists' IO case now passes for os, mem, sftp, azure, gs, and s3 (both SSE variants). Note: the same shared IO conformance case also fails for ftp, but ftp's Write path (DataConn/OpenWrite) is a structurally different mechanism with no cursorPos/initWriters concept, so that is a separate bug outside the scope of C2FO#365 and this fix - flagging for a follow-up issue rather than addressing here.
The previous commit's per-subtest fixture rework built the fixture path via path.Join(s.tmploc.Path(), ...) - a vfs-style path (e.g. /C:/Temp/...) - and passed it straight to the stdlib os.WriteFile to seed the fixture, and to fileSystem.NewFile to open it. vfs.File operations translate that leading-slash-before-drive-letter form to a native path internally (toNativeOSPath), but raw os.* calls do not, so os.WriteFile failed on Windows CI: open /C:/Temp/.../test_files/seek_then_write_rewind.txt: The filename, directory name, or volume label syntax is incorrect. Rebuilt the fixture entirely through vfs.File (Location.NewFile + File.Write/Seek/Close), never touching the raw os package with a vfs path, so path translation stays consistent across platforms. Verified locally (macOS); Windows behavior inferred from reading toNativeOSPath directly since no Windows machine is available here - will confirm via the next CI run on this PR.
FTP writes streamed new bytes directly to the server via STOR with a REST offset, relying entirely on the server to preserve any existing content the write doesn't cover. Against a live vsftpd server (the testcontainers module), this held for writes that reach or extend past the original EOF, but Seek(0, io.SeekStart) followed by a shorter Write replaced the whole file with only the newly written bytes, losing everything after them - the ftp analog of C2FO#365 (fixed for s3/gs), surfaced by the same shared conformance case added in the first commit here. Fix: when Seek or Read happens before the first Write on an existing file, Write now downloads the full existing content into a local temp file first, patches it at the current offset, and defers the actual upload to Close, which uploads the merged result in a single plain STOR - rather than streaming straight through STOR+REST and trusting the server to keep the rest of the file intact. Fresh writes (no prior Seek/Read) keep the existing direct-streaming path unchanged. Read and Seek redirect to the buffered temp file once a merge is in progress, and Close always resets the seek/read/write tracking flags. Added TestSeekToStartThenWritePreservesExistingContent, verified to fail against the pre-fix code (mock expectations unmet - Write went straight to the live dataconn instead of buffering) and pass with the fix. Verified end-to-end with the real local testcontainers integration suite (TESTCONTAINERS_RYUK_DISABLED=true go test -tags=vfsintegration ./...): the shared 'Seek to start, Write, Close, file exists' IO case now passes for os, mem, sftp, azure, gs, s3 (both SSE variants), and ftp.
Contributor
There was a problem hiding this comment.
Copilot review overview
🟡 Changes recommended
The FTP regression test has an incorrect close expectation, and deleting a file with an active merge buffer can recreate it on a later Close.
Get a fresh assessment by requesting another Copilot review.
Review effort: Lite
Findings: 2
Open (4)
What changed in this PR
Fixes offset-zero writes after seeking, extending the behavior across local and remote backends and updating integration-test infrastructure.
Changes:
- Repositions OS temp files at offset zero.
- Preserves untouched content for S3, GS, and FTP writes after seek/read.
- Adds regression tests, conformance coverage, changelog entries, and updates the MinIO image registry.
| File | Description |
|---|---|
backend/os/file.go |
Fixes temp-file positioning. |
backend/os/file_test.go |
Tests offset-zero and middle seeks. |
backend/s3/file.go |
Preserves existing objects after seek/read. |
backend/s3/file_test.go |
Adds S3 regression coverage. |
backend/gs/file.go |
Preserves existing objects after seek/read. |
backend/gs/file_test.go |
Adds GCS regression coverage. |
backend/ftp/file.go |
Adds local merge buffering for FTP writes. |
backend/ftp/file_test.go |
Adds FTP regression coverage. |
backend/testsuite/io_conformance.go |
Adds shared offset-zero conformance coverage. |
testcontainers/containers_test.go |
Switches MinIO to Quay. |
CHANGELOG.md |
Documents the fixes and image update. |
💡 Add a code-review agent skill or configure MCP servers for context-aware, tailored reviews. Learn more in the docs.
…lexity - Delete now calls Close() first so a write-after-seek/read merge buffered in f.tempFile is flushed and cleaned up before the remote file is removed. Previously, deleting mid-write left the local temp file orphaned on disk, and a later Close call would still upload the buffered content, resurrecting the file Delete had just removed. Mirrors the existing s3/gs Delete behavior. - Extracted the tempFile-repositioning branch of Seek into a seekTempFile helper to bring Seek's cyclomatic complexity back under the golangci-lint threshold (16 -> under 15). only-new-issues in CI didn't catch this because the reported line is the unchanged function signature. - Added regression tests: TestDeleteFlushesBufferedWriteBeforeRemoving, TestReadThenWritePreservesExistingContent (Read, not just Seek, before Write triggers the same preserve-content merge), and TestSeekWithinBufferedWrite (covers seekTempFile's success and error paths, previously untested at 0% coverage). Co-authored-by: Cursor <cursoragent@cursor.com>
…ding close comment - testcontainers/README.md and doc.go still documented the s3 backend's conformance image as minio/minio after containers_test.go was pinned to quay.io/minio/minio (minio/minio no longer exists on Docker Hub). - The comment above Write's defensive dataconn close in ftp/file.go claimed it was closing the stale Seek-opened dataconn, but Exists' own SingleOp DataConn call already closes that one as a side effect of its mode mismatch handling; this close is actually a no-op on the fresh SingleOp connection Exists leaves behind. Corrected the comment to describe what actually happens; no behavior change. Co-authored-by: Cursor <cursoragent@cursor.com>
grant-higgins-0
approved these changes
Sep 21, 2026
dmcilvain44
approved these changes
Sep 22, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.


On the
osbackend,Seek(0, io.SeekStart)followed byWriteappends instead of overwriting fromthe start.
memgets it right.Offset 6 is the case the conformance suite already covers, which is why this has gone unnoticed.
Cause
copyToLocalTempReaderbuffers writes into a temp file. When the file exists andSeek/Readranfirst, it copies the original in with
io.Copy— leaving the temp file's cursor at EOF — and thenrepositions, but only when the cursor is past the start (
backend/os/file.go:512):For
cursorPos == 0the reposition is skipped, so the write lands at EOF.Change
Drop the
> 0guard — the seek is needed for offset zero precisely becauseio.Copymoved thecursor.
Tests
Two, both red with only
backend/os/file.goreverted:DefaultIOTestCases, somemdefines the expected value rather than measserting it. Only
osfails:mempasses it before and after.osFileTestsuite covering offset 0 and offset 6, since theconformance run is behind the
vfsintegrationtag and would not count toward the coverage gate.go test ./backend/os/ ./backend/mem/ ./backend/testsuite/is green, as isgo test -tags vfsintegration -run TestIOConformance ./backend/os/ ./backend/mem/.gofmt -llistsnone of the changed files. CHANGELOG entry added under Unreleased.
Scope
sftpandazuredelegateSeekstraight through and may share this shape, but I could notexercise them without credentials, so this PR is limited to
os.Separately and not included:
os,azureandsftpdo not validatewhence(Linux accepts3for
SEEK_DATA) whilemem,s3,gsandftpreturnvfs.ErrSeekInvalidWhence(
errors.go:25). Happy to send that separately if you want it.Disclosure: prepared with AI assistance; I verified the backend comparison and both red/green runs
myself.