DAOS-18481 test: recovery/check_start_corner_case.yaml - Reset checker state

[shimizukko]

test_start_back_to_back interferes with test_two_pools_healthy. The repaired fault in test_start_back_to_back (orphan container) appears in the check query output during test_two_pools_healthy. The test expects the query output to be clean at the beginning of the test, so reset the checker state by stopping (dmg check stop) and restarting with --reset (dmg check start --reset) after confirming the repair result.

Skip-unit-tests: true
Skip-fault-injection-test: true
Skip-func-hw-test-medium: false
Test-tag: DMGCheckStartCornerCaseTest

Steps for the author:

• Commit message follows the guidelines.
• Appropriate Features or Test-tag pragmas were used.
• Appropriate Functional Test Stages were run.
• At least two positive code reviews including at least one code owner from each category referenced in the PR.
• Testing is complete. If necessary, forced-landing label added and a reason added in a comment.

After all prior steps are complete:

• Gatekeeper requested (daos-gatekeeper added as a reviewer).

[github-actions]

Ticket title is 'recovery/check_start_corner_case.py:DMGCheckStartCornerCaseTest.test_two_pools_healthy - Checker didn't detect inconsistent container label'
Status is 'In Review'
Labels: 'ci_master_weekly,weekly_test'
https://daosio.atlassian.net/browse/DAOS-18481

[daltonbohning]

Why do the tests only work if executed serially? Are tests leaving the system in a bad state?

[shimizukko]

Why do the tests only work if executed serially? Are tests leaving the system in a bad state?

I learned that the checker state doesn't get reset by simply restarting the server (dmg system stop; dmg system start), so the output will contain the old result and that interferes with subsequent test. We can reformat the system to clear, but more efficient way is to reset the checker by restarting with --reset after confirming the repair result.

[daltonbohning]

Why do the tests only work if executed serially? Are tests leaving the system in a bad state?

I learned that the checker state doesn't get reset by simply restarting the server (dmg system stop; dmg system start), so the output will contain the old result and that interferes with subsequent test. We can reformat the system to clear, but more efficient way is to reset the checker by restarting with --reset after confirming the repair result.

Thanks. I think this is a better approach than restarting the system