Describe the bug
PDBStorageTest.testCanAddLargeValues still runs the 512 MB of the test JVM out of heap after #1072. build-maven (ubuntu-latest, 17) of #1057 (run 35976815043) - a PR which touches only the replication Session and ServerWriter, built as a merge onto e333af0c8f, which contains 0e039c6473 (#1072):
[ERROR] Tests run: 14, Failures: 1, Errors: 0, Skipped: 0, Time elapsed: 14.77 s <<< FAILURE! -- in org.opends.server.backends.pdb.PDBStorageTest
[ERROR] org.opends.server.backends.pdb.PDBStorageTest.testCanAddLargeValues -- Time elapsed: 0.213 s <<< FAILURE!
java.lang.OutOfMemoryError: Java heap space
at org.forgerock.opendj.ldap.ByteString.toByteArray(ByteString.java:887)
at org.opends.server.backends.pdb.PDBStorage.bytesToValue(PDBStorage.java:1533)
at org.opends.server.backends.pdb.PDBStorage$WriteableStorageImpl.put(PDBStorage.java:442)
at org.opends.server.backends.pdb.PDBStorageTest$1.run(PDBStorageTest.java:165) <- the 63 MB put
at org.opends.server.backends.pdb.PDBStorage$WriteableStorageImpl.write(PDBStorage.java:679)
at org.opends.server.backends.pdb.PDBStorage.write(PDBStorage.java:1302)
at org.opends.server.backends.pdb.PDBStorageTest.testCanAddLargeValues(PDBStorageTest.java:154)
The other 33093 tests of the leg passed. #1071 failed on the 32 MB put, in Value.ensureFit(); this one gets one put further and fails in the copy which #1072 left in place on purpose.
Why
#1072 took the valueOfBytes() copy of each source out, which left about 165 MB live after the collection the 63 MB put triggers, against 311 MB before (its -Xlog:gc* table). What remains on the way into Persistit for the 63 MB value is still three humongous arrays: the source, toByteArray() in PDBStorage.bytesToValue(), and the 64 MB value buffer of the exchange - on top of the buffer pool of the test storage (76 MB, 20% of the quota) and the test server. That fits most of the time and not always: whether a free run of regions for the next 63 MB array is there is up to the collector's timing.
Both of the remaining options were named under "Left out" of #1072 and not taken there:
- the
toByteArray() copy in PDBStorage.bytesToValue() (main code; a ByteSequence gives no zero-copy access to its backing array);
- the
-Xmx512m of the test fork in the root pom.xml (argLine, lines 62 and 738), which would widen the margin for every class.
To Reproduce
Intermittent, as #1071 was. Run PDBStorageTest under JDK 17 with -Xmx512m -Xlog:gc*, the settings of the leg, and read the log of the fork: what stays live after the collection the 63 MB put triggers, plus the next 63 MB array, is close to the heap.
Expected behavior
The test keeps well inside the heap of the test JVM on every run, not on most of them - either by a larger margin for this class alone (for instance, the 63 MB value written in a storage or a fork of its own, or a heap setting for this class), or by taking another copy out of the path the test exercises.
Describe the bug
PDBStorageTest.testCanAddLargeValuesstill runs the 512 MB of the test JVM out of heap after #1072.build-maven (ubuntu-latest, 17)of #1057 (run 35976815043) - a PR which touches only the replicationSessionandServerWriter, built as a merge ontoe333af0c8f, which contains0e039c6473(#1072):The other 33093 tests of the leg passed. #1071 failed on the 32 MB put, in
Value.ensureFit(); this one gets one put further and fails in the copy which #1072 left in place on purpose.Why
#1072 took the
valueOfBytes()copy of each source out, which left about 165 MB live after the collection the 63 MB put triggers, against 311 MB before (its-Xlog:gc*table). What remains on the way into Persistit for the 63 MB value is still three humongous arrays: the source,toByteArray()inPDBStorage.bytesToValue(), and the 64 MB value buffer of the exchange - on top of the buffer pool of the test storage (76 MB, 20% of the quota) and the test server. That fits most of the time and not always: whether a free run of regions for the next 63 MB array is there is up to the collector's timing.Both of the remaining options were named under "Left out" of #1072 and not taken there:
toByteArray()copy inPDBStorage.bytesToValue()(main code; aByteSequencegives no zero-copy access to its backing array);-Xmx512mof the test fork in the rootpom.xml(argLine, lines 62 and 738), which would widen the margin for every class.To Reproduce
Intermittent, as #1071 was. Run
PDBStorageTestunder JDK 17 with-Xmx512m -Xlog:gc*, the settings of the leg, and read the log of the fork: what stays live after the collection the 63 MB put triggers, plus the next 63 MB array, is close to the heap.Expected behavior
The test keeps well inside the heap of the test JVM on every run, not on most of them - either by a larger margin for this class alone (for instance, the 63 MB value written in a storage or a fork of its own, or a heap setting for this class), or by taking another copy out of the path the test exercises.