Repository navigation
Conversation
|
Ticket title is 'FIO crc mismatch detected on 4x node IB config while running HSN cable pulls test' |
github-actions
Bot
added
the
priority
6121af8 to
8550258
Compare
|
Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net/job/daos-stack/job/daos/job/PR-19314/8/display/redirect |
8550258 to
9139b34
Compare
|
Test stage Functional on EL 9 completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-19314/9/execution/node/1357/log |
9139b34 to
ffd36c1
Compare
1. In theory, it is possible that when exclude some engine, related ULT maybe trying to abort a DTX because of some failure happened during local modification (but related modification has succeeded on remote targets). Related DTX_ABORT RPC has been sent out but not been received or processed by related remote targets. Under such case, the DTX leader is switched because of related engine is excluded. And then DTX resync is triggered by new DTX leader who may regard such DTX as committable since all alive targets claim 'PREPARED'. Then the new DTX leader will make different decision that conflict with old DTX leader did. That may cause related DTX to be aborted on some targets but committed on the others, as to data inconsistency among related redundancy group. The new logic forbids to abort DTX from old DTX leader to avoid such inconsistency. 2. Set dae->dae_aborting before vos_dtx_abort_internal() to prevent potential race with other DTX commit/abort/cleanup operations. 3. During DTX resync, the DTX CHECK RPC should not be sent to the target which UP status and co_in_ver is newer than related DTX version. Because related DTX is either non-exist or removed on the target being reintegrated. 4. Convert dtx_leader_begin() returned failure -DER_TX_RESTART as -DER_INPROGRESS to help client to properly handle related RPC. 5. New test cases for DTX abort with fense. Signed-off-by: Fan Yong <[email protected]> Signed-off-by: Liang Zhen <[email protected]>
ffd36c1 to
d9372a7
Compare
|
Test stage Functional Hardware Medium Verbs Provider MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-19314/11/execution/node/1674/log |
|
Test stage Functional Hardware Medium MD on SSD completed with status FAILURE. https://jenkins-3.daos.hpc.amslabs.hpecorp.net//job/daos-stack/job/daos/view/change-requests/job/PR-19314/11/execution/node/1688/log |
Nasf-Fan
changed the title
|
Replaced by #19364 |
In theory, it is possible that when exclude some engine, related ULT maybe trying to abort a DTX because of some failure happened during local modification (but related modification has succeeded on remote targets). Related DTX_ABORT RPC has been sent out but not been received or processed by related remote targets. Under such case, the DTX leader is switched because of related engine is excluded. And then DTX resync is triggered by new DTX leader who may regard such DTX as committable since all alive targets claim 'PREPARED'. Then the new DTX leader will make different decision that conflict with old DTX leader did. That may cause related DTX to be aborted on some targets but committed on the others, as to data inconsistency among related redundancy group. The new logic forbids to abort DTX from old DTX leader to avoid such inconsistency.
Set dae->dae_aborting before vos_dtx_abort_internal() to prevent potential race with other DTX commit/abort/cleanup operations.
During DTX resync, the DTX CHECK RPC should not be sent to the target which UP status and co_in_ver is newer than related DTX version. Because related DTX is either non-exist or removed on the target being reintegrated.
Convert dtx_leader_begin() returned failure -DER_TX_RESTART as -DER_INPROGRESS to help client to properly handle related RPC.
New test cases for DTX abort with fense.
Steps for the author:
After all prior steps are complete: