DAOS-19616 placement: derive CURRENT layouts from PRE and POST - #19091
Open
liuxuezhao wants to merge 1 commit into
Open
liuxuezhao wants to merge 1 commit into
liuxuezhao wants to merge 1 commit into
Conversation
Generating CURRENT directly can mis-handle cascading shard relocation:
a shard which an earlier rebuild put on a spare can be handed a
different spare by a new, unrelated failure, so the CURRENT layout
names a target that was never written to while the PRE_REBUILD
layout, used by the migration fetch and every read-only client, keeps
reading the old one.
CURRENT is now a transition view derived from the two canonical
endpoints instead of an independent generation mode:
- Build POST_REBUILD first; if its placement met no transitional
target (DOWN/DRAIN/UP/NEW), POST is CURRENT (single-layout path).
- Otherwise generate PRE_REBUILD and derive CURRENT from both while
preserving every readable PRE target as the primary.
- Add the POST target as a write-only peer for drain, reintegration,
extension and collateral relocation of a healthy shard.
- Use POST directly for failed PRE targets and unaffected objects.
- Handle missing rebuild spares without invalid target lookups.
The CURRENT branches of comp_need_remap() and the separate extend
pass are removed; readable targets equal PRE, write destinations
equal POST, so the placement, migration and rebuild-scan views can no
longer disagree.
Placement UTs: PLACEMENT 18 now rejects readable duplicates only; a
target can legitimately be the readable primary of one shard and the
rebuild destination of another during extension plus reintegration.
Add no_stale_read_source, extension_preserves_read_sources,
no_hidden_peer_source and relocation_peer_alloc_failure, asserting
that every healthy PRE read source stays in the CURRENT write set,
that every readable CURRENT target was written before the failure and
that pl_map_extend() failures are propagated. Test pool maps give
targets fseq 1 as gen_pool_buf() does.
Also refine the migration comment on a fetch returning iod_size 0
(placement mismatch, older fetch epoch, unresolved DTX) and why
-DER_DATA_LOSS is excluded from the rebuild status.
Build and run the reproducer from the repository root:
scons --config=force --jobs=16 \
install/bin/jump_pl_map install/lib64/libdaos.so
./utils/run_utest.py --suite_filter='^placement$' --sudo=no
Without the placement change the new tests fail on a PRE_REBUILD read
source that CURRENT does not write to; with it all 43 cases pass.
Features: rebuild
Signed-off-by: Liang Zhen <zhen.liang@hpe.com>
Signed-off-by: Xuezhao Liu <xuezhao.liu@hpe.com>
|
Ticket title is 'FIO crc mismatch detected on 4x node IB config while running HSN cable pulls test' |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Generating CURRENT directly can mis-handle cascading shard relocation: a shard which an earlier rebuild put on a spare can be handed a different spare by a new, unrelated failure, so the CURRENT layout names a target that was never written to while the PRE_REBUILD layout, used by the migration fetch and every read-only client, keeps reading the old one.
CURRENT is now a transition view derived from the two canonical endpoints instead of an independent generation mode:
Placement UTs: PLACEMENT 18 now rejects readable duplicates only; a target can legitimately be the readable primary of one shard and the rebuild destination of another during extension plus reintegration. Add no_stale_read_source, extension_preserves_read_sources, no_hidden_peer_source and relocation_peer_alloc_failure, asserting that every healthy PRE read source stays in the CURRENT write set, that every readable CURRENT target was written before the failure and that pl_map_extend() failures are propagated. Test pool maps give targets fseq 1 as gen_pool_buf() does.
Also refine the migration comment on a fetch returning iod_size 0 (placement mismatch, older fetch epoch, unresolved DTX) and why -DER_DATA_LOSS is excluded from the rebuild status.
Build and run the reproducer from the repository root:
scons --config=force --jobs=16
install/bin/jump_pl_map install/lib64/libdaos.so
./utils/run_utest.py --suite_filter='^placement$' --sudo=no
Without the placement change the new tests fail on a PRE_REBUILD read source that CURRENT does not write to; with it all 43 cases pass.
Features: rebuild
Steps for the author:
After all prior steps are complete: