Run stubtest to check stubs are complete
#1141
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Member
Are the |
Contributor
Author
No the |
Codecov Report❌ Patch coverage is
Additional details and impacted files@@ Coverage Diff @@
## master #1141 +/- ##
=======================================
Coverage 55.07% 55.07%
=======================================
Files 24 24
Lines 5420 5420
=======================================
Hits 2985 2985
Misses 2435 2435 ☔ View full report in Codecov by Sentry. 🚀 New features to boost your workflow:
|
Joao-Dionisio
approved these changes
Dec 30, 2025
Member
Joao-Dionisio
left a comment
Joao-Dionisio
left a comment
There was a problem hiding this comment.
Thank you @jonathanberthias !
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR adds a workflow that runs
stubteston the bundled type stubs. This means that whenever a class or method is added, the workflow will error until it is added to thescip.pyifile.Part of #1072
Type errors
A pre-requisite to running
stubtestis to havemypytype check the code without error. There were only a few errors reported bymypywhich I don't think are controversialAllow lists
For now, we rely on an allow list because the stubs are still wrong, they list all methods as taking
*args, **kwargsparameters which isn't true at runtime. The allow list silences all the known current errors until we manually fix them.There are 2 allow list files:
allowlistlists errors which are known and expected.todolists all the errors which we should aim to fix eventually.Entries
Each entry is a regex matching the fully qualified name of the object with an error. Each line can thus hide multiple errors. For instance, for the entry
pyscipopt.LP.addRow, these are the errors that get silenced:Conversely, some lines are actually duplicates. This is due to the re-exports. For instance, the entries:
point to the same underlying object, so both entries could be removed at the same time.
Maintenance
Finally, the workflow will also fail if an entry in the allow list does not generate any error, ensuring the allow list is always up to date.
How to fix?
The way forward is to fix all the errors in the
todoallow list. This means removing a couple of entries from the allow list, runningstubs/test.shand fixing any printed error.Unfortunately I don't know any way to automate this part.
stubtestdetects the argument names but it has no idea what annotation they should have, so some human judgment is required. Or a massive token budget for a LLM 😉