[#228] Create the shared HttpContext lazily and dump threads on a start-up hang

[vharseko]

Fixes #228

Problem

ui-smoke-tests (17, /myidm, samples/usecase/usecase1) in run 35873142464 (attempt 1) did not just log an extra SCR error: OpenIDM never became ready. Start OpenIDM waited 3 minutes for OpenIDM ready, the log stopped at the Could not obtain lock error, and the UI tests never ran. The log line after the stack trace names the reference that timed out: api-servlet(19) : Could not get service from ref [ServletRegistration]. That reference is ServletRegistrationSingleton, not the auth filter suggested in the issue.

The cause is a lock cycle between Felix SCR and pax-web 11:

• The pax-web configuration thread (a single-threaded executor) registers WebContainer (Activator.performConfiguration). SCR then activates ServletRegistrationSingleton, which registers ServletRegistration, and synchronously activates ServletComponent. ServletComponent waits for ServletRegistrationSingleton's SCR state lock.
• Another thread that needs ServletRegistration holds that state lock while activate() runs (SingleComponentManager.getServiceInternal). activate() called webContainer.createDefaultSharedHttpContext(), which calls ServerModel.run(task, false). From any thread other than the configuration thread, that call queues the task on the same executor and blocks in future.get() with no timeout.

SCR breaks the cycle only on the pax-web side, after ds.lock.timeout (5 s). This matches the log: Jetty started at 23:43:42 and the error was logged at 23:43:47. The activate() call came in with the move to pax-web 11 (#114).

Changes

• ServletRegistrationSingleton creates the shared HttpContext on first use (registerServlet, getContext, filter registration) instead of in activate(). It holds no lock while it does so: holding a monitor across the pax-web call would rebuild the same cycle, this time without a timeout. If two threads create the context at the same time, the second one is discarded (AtomicReference.compareAndSet).
• CI (Test on Unix, Start OpenIDM): if OpenIDM ready does not appear within the timeout, run jcmd <pid> Thread.print -l and print the output, and save it to openidm/logs/threaddump-<pid>.txt (included in the failure artifact).

Not covered

The log does not show which thread held the state lock, and it does not explain why start-up stayed stuck after the 5 s timeout: no further error was logged, so something waits without a timeout. The thread dump added to CI is meant to answer both questions the next time this happens.

Tests

• New ServletRegistrationSingletonTest:
  • activate() does not call WebContainer. This test fails on master with NoInteractionsWanted.
  • The shared context is created exactly once, on first use, and registerServlet uses it.
• The test checks the invariant only. It does not reproduce the SCR/pax-web deadlock.