fix: memory over allocation due to last partial block group - #832
Open
siddh34 wants to merge 1 commit into
Open
Conversation
3 tasks
dkovba
requested changes
Aug 7, 2026
Contributor
There was a problem hiding this comment.
Thank you for the contribution!
Below are the test results across the size/content matrix, cross-checked with e2fsck:
Requested Size: 32KiB
Content: none
Size: FAIL
audit.imageSize == testCase.requested = false; audit.imageSize = 134217728; testCase.requested = 32768
e2fsck: PASS
----------------------------------------
Requested Size: 64MiB
Content: none
Size: FAIL
audit.imageSize == testCase.requested = false; audit.imageSize = 134217728; testCase.requested = 67108864
e2fsck: PASS
----------------------------------------
Requested Size: 128MiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 128MiB+4KiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 130MiB+8KiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 160MiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 256MiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 1GiB
Content: none
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 4GiB
Content: none
Size: FAIL
allocatedMiB <= ceilingMiB = false; allocatedMiB = 65; ceilingMiB = 32
e2fsck: PASS
----------------------------------------
Requested Size: 63x128MiB
Content: none
Size: FAIL
allocatedMiB <= ceilingMiB = false; allocatedMiB = 127; ceilingMiB = 32
e2fsck: PASS
----------------------------------------
Requested Size: 63x128MiB+4KiB
Content: none
Size: FAIL
allocatedMiB <= ceilingMiB = false; allocatedMiB = 129; ceilingMiB = 32
e2fsck: FAIL rc=4
Block bitmap differences: +(32768--32930)
Fix? no
/imgs/63x128MiB+4KiB.img: ********** WARNING: Filesystem still has errors **********
/imgs/63x128MiB+4KiB.img: 11/524288 files (0.0% non-contiguous), 32737/2064385 blocks
----------------------------------------
Requested Size: 8GiB
Content: none
Size: FAIL
allocatedMiB <= ceilingMiB = false; allocatedMiB = 129; ceilingMiB = 32
e2fsck: FAIL rc=4
Block bitmap differences: +(32768--32930)
Fix? no
/imgs/8GiB.img: ********** WARNING: Filesystem still has errors **********
/imgs/8GiB.img: 11/524288 files (0.0% non-contiguous), 32737/2097152 blocks
----------------------------------------
Requested Size: 16GiB
Content: none
Size: FAIL
allocatedMiB <= ceilingMiB = false; allocatedMiB = 258; ceilingMiB = 32
e2fsck: FAIL rc=4
Block bitmap differences: +(32768--65826)
Fix? no
/imgs/16GiB.img: ********** WARNING: Filesystem still has errors **********
/imgs/16GiB.img: 11/1048576 files (0.0% non-contiguous), 32737/4194304 blocks
----------------------------------------
Requested Size: 160MiB
Content: 10MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 128MiB
Content: 50MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 160MiB
Content: 120MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 160MiB
Content: 124MiB
Size: PASS
e2fsck: FAIL rc=4
Block bitmap differences: +(32768--32806)
Fix? no
/imgs/160MiB_124MiB-content.img: ********** WARNING: Filesystem still has errors **********
/imgs/160MiB_124MiB-content.img: 135/16384 files (0.0% non-contiguous), 32737/40960 blocks
----------------------------------------
Requested Size: 160MiB
Content: 126MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 256MiB
Content: 130MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 1GiB
Content: 200MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 300MiB
Content: 260MiB
Size: FAIL
audit.imageSize == testCase.requested = false; audit.imageSize = 402653184; testCase.requested = 314572800
e2fsck: PASS
----------------------------------------
Requested Size: 4GiB
Content: 260MiB
Size: PASS
e2fsck: PASS
----------------------------------------
Requested Size: 63x128MiB
Content: 500MiB
Size: PASS
e2fsck: FAIL rc=4
Block bitmap differences: +(131072--160417)
Fix? no
/imgs/63x128MiB_500MiB-content.img: ********** WARNING: Filesystem still has errors **********
/imgs/63x128MiB_500MiB-content.img: 511/516096 files (0.0% non-contiguous), 131041/2064384 blocks
----------------------------------------
Requested Size: 8GiB
Content: 300MiB
Size: PASS
e2fsck: FAIL rc=4
Block bitmap differences: +(98304--109730)
Fix? no
/imgs/8GiB_300MiB-content.img: ********** WARNING: Filesystem still has errors **********
/imgs/8GiB_300MiB-content.img: 311/524288 files (0.0% non-contiguous), 98273/2097152 blocks
----------------------------------------
Requested Size: 16GiB
Content: 1000MiB
Size: PASS
e2fsck: FAIL rc=4
Block bitmap differences: +(262144--321828)
Fix? no
/imgs/16GiB_1000MiB-content.img: ********** WARNING: Filesystem still has errors **********
/imgs/16GiB_1000MiB-content.img: 1011/1048576 files (0.0% non-contiguous), 262113/4194304 blocks
----------------------------------------
Contributor
Author
|
@dkovba thanks for the tests I will update my pr in few days and will include these failed tests in it too! |
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 11, 2026 03:07
98555ce to
16a7a7b
Compare
Contributor
Author
|
Updates after issues discovered in new test cases as given by @dkovba
defining minimum inodes per group at line 41 This tries multiple candidate values for inodesPerGroup line 1093 it basically means smaller inode tables less metadata overhead
Now ext4 block bitmaps are per-group. Changes can be found at line 879 basically marking the packed metadata blocks that physically live inside a group as allocated in that group’s block bitmap also tested with e2fsck @dkovba please check according to your convience, thanks! Open to making any changes needed or I missed for this current pull request |
dkovba
reviewed
Aug 15, 2026
dkovba
left a comment
dkovba
left a comment
Contributor
There was a problem hiding this comment.
Thank you for updating the PR!
dkovba
requested changes
Aug 15, 2026
dkovba
left a comment
dkovba
left a comment
Contributor
There was a problem hiding this comment.
With the suggested implementation of the partial block support, we use significantly more physical space than on main and than mke2fs does.
With minimumInodesPerGroup: UInt32 = 896:
Requested Size: 16GiB
Content: none
Size: PASS
e2fsck: PASS
Logical Size vs mke2fs: PASS
Physical Size vs mke2fs: FAIL
physical <= mke2fs = false; physical = 30,457,856; mke2fs = 4,816,896
With the original block size derivation:
Requested Size: 16GiB
Content: none
Size: PASS
e2fsck: PASS
Logical Size vs mke2fs: PASS
Physical Size vs mke2fs: FAIL
physical <= mke2fs = false; physical = 269,647,872; mke2fs = 4,816,896
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 15, 2026 11:36
16a7a7b to
b01028b
Compare
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 15, 2026 11:42
b01028b to
0594735
Compare
Contributor
Author
Tested with let inc = Int(self.blockSize * self.inodeStrideKiB) / Int(EXT4.InodeSize) // minimizes the number of blockGroups needed to its lowest valuewhere self.inodeStrideKiB = 512 Results Here is my script for tests import ContainerizationArchive
import Foundation
import SystemPackage
import Testing
@testable import ContainerizationEXT4
struct E2FsckImageCase: Sendable {
let name: String
let imagePath: FilePath
let requestedSize: UInt64
let contentSize: UInt64
init(
name: String,
imagePath: FilePath,
requestedSize: UInt64,
contentSize: UInt64 = 0
) {
self.name = name
self.imagePath = imagePath
self.requestedSize = requestedSize
self.contentSize = contentSize
}
}
struct E2FsckResult: Sendable {
let exitCode: Int32
let stdout: String
let stderr: String
var passed: Bool {
exitCode == 0
}
}
struct EXT4E2FsckHarness {
let e2fsckPath: String
let mke2fsPath: String
init(
e2fsckPath: String = "/opt/homebrew/Cellar/e2fsprogs/1.47.4/sbin/e2fsck",
mke2fsPath: String = "/opt/homebrew/Cellar/e2fsprogs/1.47.4/sbin/mke2fs"
) {
self.e2fsckPath = e2fsckPath
self.mke2fsPath = mke2fsPath
}
func createReferenceImage(requestedSize: UInt64, at imagePath: FilePath) throws {
if FileManager.default.fileExists(atPath: imagePath.string) {
try FileManager.default.removeItem(at: imagePath.url)
}
FileManager.default.createFile(atPath: imagePath.string, contents: nil)
let handle = try FileHandle(forWritingTo: imagePath.url)
try handle.truncate(atOffset: requestedSize)
try handle.close()
let process = Process()
process.executableURL = URL(fileURLWithPath: mke2fsPath)
process.arguments = ["-F", "-q", "-t", "ext4", imagePath.string]
let stderrPipe = Pipe()
process.standardError = stderrPipe
process.standardOutput = Pipe()
try process.run()
process.waitUntilExit()
guard process.terminationStatus == 0 else {
let err = String(decoding: stderrPipe.fileHandleForReading.readDataToEndOfFile(), as: UTF8.self)
struct MkE2FsError: Swift.Error, CustomStringConvertible {
let description: String
}
throw MkE2FsError(description: "mke2fs failed: \(err)")
}
}
func createImage(for testCase: E2FsckImageCase) throws {
let formatter = try EXT4.Formatter(
testCase.imagePath,
minDiskSize: testCase.requestedSize
)
if testCase.contentSize > 0 {
let chunkSize = min(testCase.contentSize, 1.mib())
let chunk = Data(repeating: 0x41, count: Int(chunkSize))
let stream = RepeatingDataStream(chunk: chunk, totalBytes: testCase.contentSize)
try formatter.create(
path: FilePath("/content.bin"),
mode: EXT4.Inode.Mode(.S_IFREG, 0o644),
buf: stream
)
}
try formatter.close()
}
func runE2Fsck(on imagePath: FilePath) throws -> E2FsckResult {
let process = Process()
process.executableURL = URL(fileURLWithPath: e2fsckPath)
process.arguments = ["-fn", imagePath.string]
let stdoutPipe = Pipe()
let stderrPipe = Pipe()
process.standardOutput = stdoutPipe
process.standardError = stderrPipe
try process.run()
process.waitUntilExit()
let stdoutData = stdoutPipe.fileHandleForReading.readDataToEndOfFile()
let stderrData = stderrPipe.fileHandleForReading.readDataToEndOfFile()
return E2FsckResult(
exitCode: process.terminationStatus,
stdout: String(decoding: stdoutData, as: UTF8.self),
stderr: String(decoding: stderrData, as: UTF8.self)
)
}
func deleteImage(at imagePath: FilePath) throws {
if FileManager.default.fileExists(atPath: imagePath.string) {
try FileManager.default.removeItem(at: imagePath.url)
}
}
func imageSize(at imagePath: FilePath) throws -> UInt64 {
let handle = try FileHandle(forReadingFrom: imagePath.url)
return try handle.seekToEnd()
}
func allocatedMiB(at imagePath: FilePath) throws -> UInt64 {
let ext4 = try EXT4.EXT4Reader(blockDevice: imagePath)
let sb = ext4.superBlock
let blocksCount = UInt64(sb.blocksCountLow) | (UInt64(sb.blocksCountHigh) << 32)
let freeBlocks = UInt64(sb.freeBlocksCountLow) | (UInt64(sb.freeBlocksCountHigh) << 32)
let usedBlocks = blocksCount - freeBlocks
let usedBytes = usedBlocks * UInt64(sb.blockSize)
return usedBytes / 1024 / 1024
}
// Physical (used) and logical (file) size for any existing ext4 image, ours or a reference.
func footprint(at imagePath: FilePath) throws -> (logicalBytes: UInt64, physicalBytes: UInt64) {
let logicalBytes = try imageSize(at: imagePath)
let ext4 = try EXT4.EXT4Reader(blockDevice: imagePath)
let sb = ext4.superBlock
let blocksCount = UInt64(sb.blocksCountLow) | (UInt64(sb.blocksCountHigh) << 32)
let freeBlocks = UInt64(sb.freeBlocksCountLow) | (UInt64(sb.freeBlocksCountHigh) << 32)
let physicalBytes = (blocksCount - freeBlocks) * UInt64(sb.blockSize)
return (logicalBytes, physicalBytes)
}
}
final class RepeatingDataStream: ReadableStream {
private let chunk: Data
private let totalBytes: UInt64
private var writtenBytes: UInt64 = 0
private var chunkOffset: Int = 0
init(chunk: Data, totalBytes: UInt64) {
self.chunk = chunk
self.totalBytes = totalBytes
}
func read(_ buffer: UnsafeMutablePointer<UInt8>, maxLength len: Int) -> Int {
guard writtenBytes < totalBytes else {
return 0
}
let remaining = Int(min(UInt64(len), totalBytes - writtenBytes))
let bytes = chunk.withUnsafeBytes { chunkBytes in
guard let base = chunkBytes.baseAddress?.assumingMemoryBound(to: UInt8.self) else {
return 0
}
var copied = 0
while copied < remaining {
let available = chunk.count - chunkOffset
let toCopy = min(remaining - copied, available)
buffer.advanced(by: copied).update(from: base.advanced(by: chunkOffset), count: toCopy)
copied += toCopy
chunkOffset = (chunkOffset + toCopy) % chunk.count
}
return copied
}
writtenBytes += UInt64(bytes)
return bytes
}
}
private func formatBytes(_ bytes: UInt64) -> String {
let units: [(UInt64, String)] = [(1.gib(), "GiB"), (1.mib(), "MiB"), (1.kib(), "KiB")]
for (size, label) in units where bytes >= size && bytes % size == 0 {
return "\(bytes / size)\(label)"
}
return "\(bytes)B"
}
private func formatCount(_ n: UInt64) -> String {
let formatter = NumberFormatter()
formatter.numberStyle = .decimal
return formatter.string(from: NSNumber(value: n)) ?? "\(n)"
}
@Test func e2FsckMatrixComparesAgainstMke2fs() throws {
let harness = EXT4E2FsckHarness()
let cases: [(name: String, requested: UInt64, content: UInt64)] = [
// ("32KiB_none", 32.kib(), 0),
// ("64MiB_none", 64.mib(), 0),
("128MiB_none", 128.mib(), 0),
("1GiB_none", 1.gib(), 0),
("4GiB_none", 4.gib(), 0),
("8GiB_none", 8.gib(), 0),
("16GiB_none", 16.gib(), 0),
]
let root = FileManager.default.temporaryDirectory
.appendingPathComponent("ext4-mke2fs-compare-\(UUID().uuidString)", isDirectory: true)
try FileManager.default.createDirectory(at: root, withIntermediateDirectories: true)
defer { try? FileManager.default.removeItem(at: root) }
for testCase in cases {
let ourImage = FilePath(root.appendingPathComponent("\(testCase.name)-ours.img", isDirectory: false))
let refImage = FilePath(root.appendingPathComponent("\(testCase.name)-ref.img", isDirectory: false))
try harness.createImage(
for: E2FsckImageCase(name: testCase.name, imagePath: ourImage, requestedSize: testCase.requested, contentSize: testCase.content)
)
try harness.createReferenceImage(requestedSize: testCase.requested, at: refImage)
let fsckResult = try harness.runE2Fsck(on: ourImage)
let ours = try harness.footprint(at: ourImage)
let reference = try harness.footprint(at: refImage)
let sizePass = ours.logicalBytes == testCase.requested
let fsckPass = fsckResult.passed
let logicalPass = ours.logicalBytes == reference.logicalBytes
let physicalPass = ours.physicalBytes <= reference.physicalBytes
print("Requested Size: \(formatBytes(testCase.requested))")
print("Content: \(testCase.content > 0 ? formatBytes(testCase.content) : "none")")
print("Size: \(sizePass ? "PASS" : "FAIL")")
print("e2fsck: \(fsckPass ? "PASS" : "FAIL")")
print("Our logical size: \(formatBytes(ours.logicalBytes)), mke2fs logical size: \(formatBytes(reference.logicalBytes))")
print("Our physical size: \(formatBytes(ours.physicalBytes)), mke2fs physical size: \(formatBytes(reference.physicalBytes))")
print("Logical Size vs mke2fs: \(logicalPass ? "PASS" : "FAIL")")
print("Physical Size vs mke2fs: \(physicalPass ? "PASS" : "FAIL")")
if !physicalPass {
print("physical <= mke2fs = false; physical = \(formatCount(ours.physicalBytes)); mke2fs = \(formatCount(reference.physicalBytes))")
}
#expect(sizePass)
#expect(fsckPass)
#expect(physicalPass, "physical <= mke2fs = false; physical = \(ours.physicalBytes); mke2fs = \(reference.physicalBytes)")
}
}@dkovba please check when it is convenient for you also please reply to comment |
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
2 times, most recently
from
August 15, 2026 17:23
c167cb3 to
3d888ee
Compare
Contributor
|
It seems this code gives allocated size, ignoring sparseness: Consider something like this: Test results for |
Contributor
Author
|
@dkovba I will make some changes in few days. Thanks for pointing my tests in correct direction |
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 22, 2026 04:06
3d888ee to
e57324d
Compare
Contributor
Author
|
@dkovba I was able to reduce physical size across file allocation I want to ask you about the journal size you are use for testing containerization vs mke2fs (journalled)? let cases: [(name: String, requested: UInt64, content: UInt64, journalSize: UInt64?)] = [
("128MiB_nojournal", 128.mib(), 0, nil),
("1GiB_nojournal", 1.gib(), 0, nil),
("4GiB_nojournal", 4.gib(), 0, nil),
("8GiB_nojournal", 8.gib(), 0, nil),
("16GiB_nojournal", 16.gib(), 0, nil),
("128MiB_journal", 128.mib(), 0, 4.mib()),
("63x128MiB_journal", 63 * 128.mib(), 0, 32.mib()),
("63x128MiB+4KiB_journal", 63 * 128.mib() + 4.kib(), 0, 32.mib()),
("1GiB_journal", 1.gib(), 0, 32.mib()),
("4GiB_journal", 4.gib(), 0, 64.mib()),
("8GiB_journal", 8.gib(), 0, 128.mib()),
("16GiB_journal", 16.gib(), 0, 256.mib()),
]This are the test cases I am testing them with. This test cases are passing, result is pasted as below Also I faced a issue with highlighted line, so I am commenting this superblock.journalUUID = filesystemUUIDas e2fsck was searching for external journal when we aren't writing external journal the error is as follow |
dkovba
approved these changes
Aug 25, 2026
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
2 times, most recently
from
August 27, 2026 19:33
1412537 to
02283dd
Compare
Contributor
Author
|
@dkovba I will keep the branch in sync with main until it get merged. Also open to making more changes for this PR if something I missed |
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 27, 2026 19:52
02283dd to
a1b6496
Compare
dkovba
reviewed
Aug 27, 2026
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
2 times, most recently
from
August 29, 2026 01:36
499337d to
af47d80
Compare
dkovba
requested changes
Aug 29, 2026
Contributor
There was a problem hiding this comment.
Another test case that fails to pass the e2fsck check:
Requested Size: 128MiB+4KiB
Content: 124MiB
Logical size: PASS
Physical size: PASS
e2fsck: FAIL rc=8
ext2fs_check_desc: Corrupt group descriptor: bad block for inode table
/opt/homebrew/opt/e2fsprogs/sbin/e2fsck: Group descriptors look bad... trying backup blocks...
Note: if several inode or block bitmap blocks or part
of the inode table require relocation, you may wish to try
running e2fsck with the '-b 8193' option first. The problem
may lie only with the primary block group descriptors, and
the backup block group descriptors may be OK.
Inode table for group 1 is not in group. (block 32295)
WARNING: SEVERE DATA LOSS POSSIBLE.
Relocate? no
Pass 1: Checking inodes, blocks, and sizes
Illegal block number passed to ext2fs_test_block_bitmap #32769 for in-use block map
...
Illegal block number passed to ext2fs_mark_block_bitmap #32806 for metadata block map
Error while scanning inodes (8192): Corrupt group descriptor: bad block for inode table
e2fsck: aborted
Contributor
Author
|
@dkovba thanks for pointing it out I will update the pull request by the EOD testing a few more possibilities like it |
dkovba
requested changes
Aug 29, 2026
Contributor
There was a problem hiding this comment.
Another similar failure, which seems to be caused by a different issue:
Requested size: 128MiB+4KiB
Content: 125MiB+218*4KiB
Logical size: FAIL
audit.imageSize = 134225920; audit.blocksCount * 4.kib() = 134221824
audit.imageSize = 134225920; expectedSize = 134221824
Physical size: PASS
e2fsck: FAIL rc=8
ext2fs_check_desc: Corrupt group descriptor: bad block for inode bitmap
Note: if several inode or block bitmap blocks or part
of the inode table require relocation, you may wish to try
may lie only with the primary block group descriptors, and
the backup block group descriptors may be OK.
Inode bitmap for group 1 is not in group. (block 32769)
Relocate? no
Inode table for group 1 is not in group. (block 32770)
WARNING: SEVERE DATA LOSS POSSIBLE.
Relocate? no
Illegal block number passed to ext2fs_test_block_bitmap #32770 for in-use block map
...
Illegal block number passed to ext2fs_mark_block_bitmap #32779 for metadata block map
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
August 29, 2026 08:17
af47d80 to
14f1728
Compare
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
September 2, 2026 01:08
be3f21c to
2d1c231
Compare
dkovba
reviewed
Sep 2, 2026
dkovba
left a comment
dkovba
left a comment
Contributor
There was a problem hiding this comment.
Several new small things after taking another look.
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
September 2, 2026 01:12
2d1c231 to
488bd0b
Compare
Contributor
My previous response was confusing. Please consider restoring the |
Contributor
Author
I will throw cannotResizeFS error when final metadata does not fit, rather than expanding the image in case of content size being larger such that metadata is not fitting Currently these cases will still fail if we are doing == check for now I am commenting it out as it fails the tests Will modify them to test exact scenario accordingly in scenario of == check This ones where metadata is not fitting in requested size will be added to a new test case Edit: I have added this 
|
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
8 times, most recently
from
September 2, 2026 07:56
437280e to
2970e56
Compare
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
September 2, 2026 15:29
2970e56 to
909b62a
Compare
Contributor
|
@siddh34 What do you think about dropping the last partial group and rounding the size down when its own metadata doesn't fit in it, instead of throwing? Bigger partial groups would be kept as they are now, and |
dkovba
requested changes
Sep 2, 2026
dkovba
left a comment
dkovba
left a comment
Contributor
There was a problem hiding this comment.
Found another thing that has to be fixed.
| let packedInodeTableBlocks = extraGroupCount * UInt64(inodeTableSizePerGroup) | ||
| let packedMetadataEnd = UInt32(packedInodeTableStart + packedInodeTableBlocks) | ||
| // contentRequiredSize above permits an empty filesystem to grow to one block group. | ||
| // Beyond that minimum geometry, content cannot displace packed metadata. |
Contributor
Author
Sure we can do it. Sorry I got too fixated on edit: pushed the change with all tests passing 
|
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
7 times, most recently
from
September 4, 2026 01:12
b438798 to
9dff41d
Compare
dkovba
requested changes
Sep 4, 2026
| newSize = minBlocks * UInt64(self.blockSize) | ||
| } | ||
|
|
||
| // Preserve a byte tail in the backing file; only whole blocks belong to the filesystem. |
Contributor
There was a problem hiding this comment.
Consider rounding the image size up to a multiple of 512 so the created image can be attached with the Virtualization framework:
Requested size: 128MiB+100B
Logical size: 134217828
Unaligned tail: 100
VZ attachment: REJECTED
Error Domain=VZErrorDomain Code=5 "The disk image format is not recognized." UserInfo={NSLocalizedFailure=Invalid disk image., NSLocalizedFailureReason=The disk image format is not recognized.}
Contributor
Author
There was a problem hiding this comment.
Sure I will look into this! Will resolve this in few days
Contributor
Author
There was a problem hiding this comment.
This is how I am attaching images
#if os(macOS)
import Virtualization
#endif
#if os(macOS)
func vzAttachmentError(for imagePath: FilePath) -> String? {
do {
_ = try VZDiskImageStorageDeviceAttachment(url: imagePath.url, readOnly: false)
return nil
} catch {
return String(describing: error)
}
}◇ Test e2FsckMatrixReportsFailuresAtEnd() started.
CASE: 128MiB+100B_none
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/128MiB+100B_none.img
requested: 134217828 bytes
content: 0 bytes
allocatedMiB: 0
imageSize == blocksCount * blockSize: PASS
imageSize = 134221824
blocksCount * 4096 = 134221824
VZ attachment: PASS
e2fsck rc: 0
CASE: 1GiB_none
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/1GiB_none.img
requested: 1073741824 bytes
content: 0 bytes
allocatedMiB: 2
imageSize == blocksCount * blockSize: PASS
imageSize = 1073741824
blocksCount * 4096 = 1073741824
VZ attachment: PASS
e2fsck rc: 0
CASE: 63x128MiB_none
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/63x128MiB_none.img
requested: 8455716864 bytes
content: 0 bytes
allocatedMiB: 2
imageSize == blocksCount * blockSize: PASS
imageSize = 8455716864
blocksCount * 4096 = 8455716864
VZ attachment: PASS
e2fsck rc: 0
CASE: 128MiB+4KiB_126MiB
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/128MiB+4KiB_126MiB.img
requested: 134221824 bytes
content: 132120576 bytes
allocatedMiB: 130
imageSize == blocksCount * blockSize: PASS
imageSize = 136474624
blocksCount * 4096 = 136474624
VZ attachment: PASS
e2fsck rc: 0
CASE: 128MiB+4KiB_125MiB+218blk
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/128MiB+4KiB_125MiB+218blk.img
requested: 134221824 bytes
content: 131964928 bytes
allocatedMiB: 128
imageSize == blocksCount * blockSize: PASS
imageSize = 134217728
blocksCount * 4096 = 134217728
VZ attachment: PASS
e2fsck rc: 0
CASE: 256MiB_130MiB
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/256MiB_130MiB.img
requested: 268435456 bytes
content: 136314880 bytes
allocatedMiB: 134
imageSize == blocksCount * blockSize: PASS
imageSize = 268435456
blocksCount * 4096 = 268435456
VZ attachment: PASS
e2fsck rc: 0
CASE: 300MiB_260MiB
image: /var/folders/y4/rkj6lthx2vg9ls8cbyyy8nb00000gn/T/ext4-e2fsck-matrix-96840061-FDF7-447C-A8FC-33D2731D0AAD/300MiB_260MiB.img
requested: 314572800 bytes
content: 272629760 bytes
allocatedMiB: 266
imageSize == blocksCount * blockSize: PASS
imageSize = 314572800
blocksCount * 4096 = 314572800
VZ attachment: PASS
e2fsck rc: 0
========== E2FSCK SUMMARY ==========
All 7 cases passed.
Note:
For 128MiB + 100B Our image: rounds up to 128MiB + 4KiB, valid raw VZ disk. mke2fs reference: retains 128MiB + 100B, not a valid VZ raw disk
Tested a few examples from my side @dkovba please check on your side if current changes work
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
September 5, 2026 01:57
9dff41d to
d9f5761
Compare
siddh34
force-pushed
the
fix/memoryOverAllocation
branch
from
September 6, 2026 02:58
d9f5761 to
8825a91
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This resolves issue #647
This is fix for memory over allocation due to last partial block
I have explicitly laid out metadata for extra/last groups
Modifications:
Packed-region math is computation starts at line 718
function to calculate blocks in last group at line 58
while loop here didn't made any sense so replaced it with if check line 914