Skip to content

Flaky ExchangeLockTest.deadlock: both lockers can abort on a two-way deadlock #313

Description

@vharseko

Summary

com.persistit.ExchangeLockTest.deadlock is flaky on CI. It fails at the second assertion, assertTrue(a._committed ^ b._committed). The timing assertion on the line before it passes.

Observed

[ERROR] com.persistit.ExchangeLockTest.deadlock -- Time elapsed: 0.122 s <<< FAILURE!
java.lang.AssertionError
	at org.junit.Assert.assertTrue(Assert.java:53)
	at com.persistit.ExchangeLockTest.deadlock(ExchangeLockTest.java:209)
Tests run: 592, Failures: 1, Errors: 0, Skipped: 5

Analysis

The test expects exactly one of the two lockers to commit. Both lockers can detect the deadlock at the same moment, and then both abort:

  1. In cycle 0, a locks key 1 and b locks key 2. In cycle 1 the coordinator calls go(0), which does not wait. So a (asking for key 2) and b (asking for key 1) enter TransactionIndex.wwDependency concurrently.
  2. wwDependency calls source.setDepends(target) and then isDeadlocked(source) (TransactionIndex.java ~L856). No victim is chosen.
  3. If both threads set their dependency before either one checks, both see the cycle A→B→A and both return UNCOMMITTED. Both transactions then roll back, a._committed ^ b._committed is false, and the assertion fails.

Data stays correct in this case: the deadlock is resolved, just with two victims instead of one. This is the same mechanism as TransactionIndexTest.testDeadlockedWwDependency, fixed test-side in #256 ("deadlock may be detected by any participant").

Note: the test does not report the lockers' _exception, so the log does not show directly that both aborted. The mechanism above is inferred from the code.

Possible fixes

  • Test-side (as in Stabilize flaky TransactionIndexTest.testDeadlockedWwDependency #256): assert that at most one locker committed and that at least one got a deadlock/rollback. For example, assertFalse(a._committed && b._committed), plus a check on _exception. Also include _exception in the failure message for better diagnostics.
  • Product-side: pick a deterministic deadlock victim in wwDependency/isDeadlocked, for example abort only the transaction with the larger start timestamp. Exactly one participant would then abort, and the original assertion would hold. This is more invasive than the test-side fix.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    testsTest code changes

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions