Skip to content

Conversation

@nccatoni
Copy link
Collaborator

Motivation

Changes

Workflow

  1. ⚠️ Create your PR as draft ⚠️
  2. Work on you PR until the CI passes
  3. Mark it as ready for review
    • Test logic is modified? -> Get a review from RFC owner.
    • Framework is modified, or non obvious usage of it -> get a review from R&P team

🚀 Once your PR is reviewed and the CI green, you can merge it!

🛟 #apm-shared-testing 🛟

Reviewer checklist

  • Anything but tests/ or manifests/ is modified ? I have the approval from R&P team
  • A docker base image is modified?
    • the relevant build-XXX-image label is present
  • A scenario is added, removed or renamed?

@github-actions
Copy link
Contributor

github-actions bot commented Jan 22, 2026

CODEOWNERS have been resolved as:

tests/test_the_test/test_slow_and_crash_decorators.py                   @DataDog/system-tests-core
conftest.py                                                             @DataDog/system-tests-core
pyproject.toml                                                          @DataDog/system-tests-core
utils/__init__.py                                                       @DataDog/system-tests-core
utils/_decorators.py                                                    @DataDog/system-tests-core

@nccatoni nccatoni force-pushed the nccatoni/force-skip-replacement branch from bbdc33c to a02db23 Compare January 22, 2026 11:35
@nccatoni nccatoni requested a review from cbeauchesne January 22, 2026 12:48
@datadog-official
Copy link

datadog-official bot commented Jan 22, 2026

⚠️ Tests

Fix all issues with Cursor

⚠️ Warnings

❄️ 2 New flaky tests detected

tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM.test_mix_snapshot[poc] from system_tests_suite (Datadog) (Fix with Cursor)
ValueError: Snapshot log6c01d-cba3-49d8-805c-792971e93e75 was not received.

self = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7fc2794c2c30>

    @missing_feature(context.library == "nodejs", reason="Not yet implemented", force_skip=True)
    @missing_feature(context.library == "golang", reason="Not yet implemented", force_skip=True)
    def test_mix_snapshot(self):
        self._assert()
>       self._validate_snapshots()

...
tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM.test_mix_snapshot[spring-boot-openliberty] from system_tests_suite (Datadog) (Fix with Cursor)
ValueError: Snapshot logec865-d3ad-455f-9c2b-590e96d64adf was not received.

self = <tests.debugger.test_debugger_probe_snapshot.Test_Debugger_Method_Probe_Snaphots_With_SCM object at 0x7f27684556a0>

    @missing_feature(context.library == "nodejs", reason="Not yet implemented", force_skip=True)
    @missing_feature(context.library == "golang", reason="Not yet implemented", force_skip=True)
    def test_mix_snapshot(self):
        self._assert()
>       self._validate_snapshots()

...

ℹ️ Info

🧪 All tests passed

This comment will be updated automatically if new data arrives.
🔗 Commit SHA: ca3e14e | Docs | Datadog PR Page | Was this helpful? Give us feedback!

Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Could you write some test?

By reading your code, I think it does not do what we expect (skip if it's annotated with flaky/bug/missing_feature), but inconditionally skip the test.

@nccatoni nccatoni force-pushed the nccatoni/force-skip-replacement branch from a02db23 to 2bf4a05 Compare January 23, 2026 10:43
@nccatoni nccatoni force-pushed the nccatoni/force-skip-replacement branch from 2bf4a05 to 5926ec2 Compare January 23, 2026 12:29
@nccatoni nccatoni requested a review from cbeauchesne January 23, 2026 12:39
@nccatoni nccatoni marked this pull request as ready for review January 23, 2026 14:56
@nccatoni nccatoni requested a review from a team as a code owner January 23, 2026 14:56
@nccatoni nccatoni force-pushed the nccatoni/force-skip-replacement branch from 2615cc5 to 4e08f0c Compare January 27, 2026 12:49
@nccatoni nccatoni requested a review from cbeauchesne January 27, 2026 12:53
Copy link
Collaborator

@cbeauchesne cbeauchesne left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I like this !

@nccatoni nccatoni enabled auto-merge (squash) January 27, 2026 13:45
@nccatoni nccatoni requested a review from Copilot January 28, 2026 16:47
Copy link

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces two new decorators (@slow and @scenario_crash) that replace the force_skip option in existing decorators like @bug and @missing_feature. These decorators leverage pytest's skip_if_xfail marker to conditionally skip tests only when combined with a declaration marker (bug, missing_feature, etc.).

Changes:

  • Added @slow and @scenario_crash decorators as aliases to pytest.mark.skip_if_xfail
  • Updated conftest.py to skip tests that have both skip_if_xfail and declaration markers
  • Added comprehensive test coverage for the new decorators

Reviewed changes

Copilot reviewed 5 out of 5 changed files in this pull request and generated 4 comments.

Show a summary per file
File Description
utils/_decorators.py Defines the new slow and scenario_crash decorators
utils/init.py Exports the new decorators in the public API
tests/test_the_test/test_slow_and_crash_decorators.py Comprehensive test suite validating decorator behavior
pyproject.toml Registers the new skip_if_xfail pytest marker
conftest.py Implements skip logic for tests with both markers

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

@nccatoni nccatoni merged commit 1b47d6a into main Jan 29, 2026
2579 of 2583 checks passed
@nccatoni nccatoni deleted the nccatoni/force-skip-replacement branch January 29, 2026 09:51
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants