Skip to content

DAOS-19616 placement: derive CURRENT layouts from PRE and POST - #19091

Open
liuxuezhao wants to merge 1 commit into
masterfrom
liang/cas_shard_move
Open

liuxuezhao wants to merge 1 commit into
masterfrom
liang/cas_shard_move

Conversation

@liuxuezhao

Copy link
Copy Markdown
Contributor

Generating CURRENT directly can mis-handle cascading shard relocation: a shard which an earlier rebuild put on a spare can be handed a different spare by a new, unrelated failure, so the CURRENT layout names a target that was never written to while the PRE_REBUILD layout, used by the migration fetch and every read-only client, keeps reading the old one.

CURRENT is now a transition view derived from the two canonical endpoints instead of an independent generation mode:

  • Build POST_REBUILD first; if its placement met no transitional target (DOWN/DRAIN/UP/NEW), POST is CURRENT (single-layout path).
  • Otherwise generate PRE_REBUILD and derive CURRENT from both while preserving every readable PRE target as the primary.
  • Add the POST target as a write-only peer for drain, reintegration, extension and collateral relocation of a healthy shard.
  • Use POST directly for failed PRE targets and unaffected objects.
  • Handle missing rebuild spares without invalid target lookups. The CURRENT branches of comp_need_remap() and the separate extend pass are removed; readable targets equal PRE, write destinations equal POST, so the placement, migration and rebuild-scan views can no longer disagree.

Placement UTs: PLACEMENT 18 now rejects readable duplicates only; a target can legitimately be the readable primary of one shard and the rebuild destination of another during extension plus reintegration. Add no_stale_read_source, extension_preserves_read_sources, no_hidden_peer_source and relocation_peer_alloc_failure, asserting that every healthy PRE read source stays in the CURRENT write set, that every readable CURRENT target was written before the failure and that pl_map_extend() failures are propagated. Test pool maps give targets fseq 1 as gen_pool_buf() does.

Also refine the migration comment on a fetch returning iod_size 0 (placement mismatch, older fetch epoch, unresolved DTX) and why -DER_DATA_LOSS is excluded from the rebuild status.

Build and run the reproducer from the repository root:

scons --config=force --jobs=16
install/bin/jump_pl_map install/lib64/libdaos.so
./utils/run_utest.py --suite_filter='^placement$' --sudo=no

Without the placement change the new tests fail on a PRE_REBUILD read source that CURRENT does not write to; with it all 43 cases pass.

Features: rebuild

Steps for the author:

  • Commit message follows the guidelines.
  • Appropriate Features or Test-tag pragmas were used.
  • Appropriate Functional Test Stages were run.
  • At least two positive code reviews including at least one code owner from each category referenced in the PR.
  • Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

  • Gatekeeper requested (daos-gatekeeper added as a reviewer).

Generating CURRENT directly can mis-handle cascading shard relocation:
a shard which an earlier rebuild put on a spare can be handed a
different spare by a new, unrelated failure, so the CURRENT layout
names a target that was never written to while the PRE_REBUILD
layout, used by the migration fetch and every read-only client, keeps
reading the old one.

CURRENT is now a transition view derived from the two canonical
endpoints instead of an independent generation mode:
- Build POST_REBUILD first; if its placement met no transitional
  target (DOWN/DRAIN/UP/NEW), POST is CURRENT (single-layout path).
- Otherwise generate PRE_REBUILD and derive CURRENT from both while
  preserving every readable PRE target as the primary.
- Add the POST target as a write-only peer for drain, reintegration,
  extension and collateral relocation of a healthy shard.
- Use POST directly for failed PRE targets and unaffected objects.
- Handle missing rebuild spares without invalid target lookups.
The CURRENT branches of comp_need_remap() and the separate extend
pass are removed; readable targets equal PRE, write destinations
equal POST, so the placement, migration and rebuild-scan views can no
longer disagree.

Placement UTs: PLACEMENT 18 now rejects readable duplicates only; a
target can legitimately be the readable primary of one shard and the
rebuild destination of another during extension plus reintegration.
Add no_stale_read_source, extension_preserves_read_sources,
no_hidden_peer_source and relocation_peer_alloc_failure, asserting
that every healthy PRE read source stays in the CURRENT write set,
that every readable CURRENT target was written before the failure and
that pl_map_extend() failures are propagated. Test pool maps give
targets fseq 1 as gen_pool_buf() does.

Also refine the migration comment on a fetch returning iod_size 0
(placement mismatch, older fetch epoch, unresolved DTX) and why
-DER_DATA_LOSS is excluded from the rebuild status.

Build and run the reproducer from the repository root:

  scons --config=force --jobs=16 \
    install/bin/jump_pl_map install/lib64/libdaos.so
  ./utils/run_utest.py --suite_filter='^placement$' --sudo=no

Without the placement change the new tests fail on a PRE_REBUILD read
source that CURRENT does not write to; with it all 43 cases pass.

Features: rebuild

Signed-off-by: Liang Zhen <zhen.liang@hpe.com>
Signed-off-by: Xuezhao Liu <xuezhao.liu@hpe.com>
@liuxuezhao
liuxuezhao requested review from a team as code owners September 16, 2026 14:53
@github-actions

Copy link
Copy Markdown

Ticket title is 'FIO crc mismatch detected on 4x node IB config while running HSN cable pulls test'
Status is 'Open'
Job should run at elevated priority (1)
https://daosio.atlassian.net/browse/DAOS-19616

@github-actions github-actions Bot added the priority Ticket has high priority (automatically managed) label Sep 16, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

priority Ticket has high priority (automatically managed)

Development

Successfully merging this pull request may close these issues.

2 participants