Skip to content

gh-143807: Limit executor growth #143814

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Open
Fidget-Spinner wants to merge 2 commits into
base: main
Choose a base branch
from
Open

gh-143807: Limit executor growth #143814

Fidget-Spinner wants to merge 2 commits into from

Conversation

@Fidget-Spinner
Copy link
Member

@Fidget-Spinner Fidget-Spinner commented Jan 13, 2026 •
edited
Loading

No unit test, because the repro provided doesn't repro on my system.

The requirement is there can only be 255 executors in a single code object, as the executor index is bounded by oparg, whch is a int8_t.

@Fidget-Spinner Fidget-Spinner changed the title Limit executor growth gh-143807: Limit executor growth Jan 13, 2026
@bedevere-app bedevere-app bot mentioned this pull request Jan 13, 2026
Open
@savannahostrowski
Copy link
Member

savannahostrowski commented Jan 30, 2026

Hmmm, can we confirm that this fixes the assertion failure on @devdanzin's system then at least, since this could at least be repro'd there?

@devdanzin
Copy link
Member

devdanzin commented Jan 30, 2026

I'll be able to test in about 6 hours.

@devdanzin
Copy link
Member

devdanzin commented Jan 31, 2026 •
edited
Loading

Sorry about the delay, had to bisect it not reproducing on HEAD.

e370c8d has made it stop aborting:

fca7fec aborts Tue Jan 13 06:02:27 2026 -0800
2873c31 aborts Wed Jan 14 03:11:53 2026 +0800
6db952e aborts Wed Jan 14 11:34:58 2026 +0000
^- Bad
v- Good 
e370c8d works Wed Jan 14 20:23:14 2026 +0800
a009e78 works Thu Jan 15 12:37:30 2026 +0100

Does that make sense?

Applying the patch (edit: and running the MRE) leads to a segfault (with or without ASan), can you reproduce this?

ASan output:

AddressSanitizer:DEADLYSIGNAL
=================================================================
==1542863==ERROR: AddressSanitizer: SEGV on unknown address (pc 0x5df3f5bc4544 bp 0x7fff285f0ce0 sp 0x7fff285f0cb0 T0)
==1542863==The signal is caused by a READ memory access.
==1542863==Hint: this fault was caused by a dereference of a high value address (see register values below).  Disassemble the provided pc to learn which register was used.
    #0 0x5df3f5bc4544 in _Py_ExecutorDetach /home/danzin/projects/repro_jit_cpython/Python/optimizer.c:1765:5
    #1 0x5df3f5bc5462 in executor_invalidate /home/danzin/projects/repro_jit_cpython/Python/optimizer.c:1789:5
    #2 0x5df3f5bbefed in uop_dealloc /home/danzin/projects/repro_jit_cpython/Python/optimizer.c:345:5
    #3 0x5df3f53dc246 in _Py_Dealloc /home/danzin/projects/repro_jit_cpython/Objects/object.c:3207:5
    #4 0x5df3f5bbc497 in _PyOptimizer_Optimize /home/danzin/projects/repro_jit_cpython/Python/optimizer.c:188:13
    #5 0x5df3f56980a3 in stop_tracing_and_jit /home/danzin/projects/repro_jit_cpython/Python/ceval.c:1461:15
    #6 0x5df3f5654939 in _PyEval_EvalFrameDefault /home/danzin/projects/repro_jit_cpython/Python/generated_cases.c.h:11868:27
    #7 0x5df3f5630e67 in _PyEval_EvalFrame /home/danzin/projects/repro_jit_cpython/./Include/internal/pycore_ceval.h:118:16
    #8 0x5df3f5630e67 in _PyEval_Vector /home/danzin/projects/repro_jit_cpython/Python/ceval.c:2541:12
    #9 0x5df3f5630884 in PyEval_EvalCode /home/danzin/projects/repro_jit_cpython/Python/ceval.c:1005:21
    #10 0x5df3f5c5b24e in run_eval_code_obj /home/danzin/projects/repro_jit_cpython/Python/pythonrun.c:1366:12
    #11 0x5df3f5c5a41b in run_mod /home/danzin/projects/repro_jit_cpython/Python/pythonrun.c:1469:19
    #12 0x5df3f5c54a1c in pyrun_file /home/danzin/projects/repro_jit_cpython/Python/pythonrun.c:1294:15
    #13 0x5df3f5c5257c in _PyRun_SimpleFileObject /home/danzin/projects/repro_jit_cpython/Python/pythonrun.c:518:13
    #14 0x5df3f5c518ed in _PyRun_AnyFileObject /home/danzin/projects/repro_jit_cpython/Python/pythonrun.c:81:15
    #15 0x5df3f5cce15a in pymain_run_file_obj /home/danzin/projects/repro_jit_cpython/Modules/main.c:410:15
    #16 0x5df3f5cce15a in pymain_run_file /home/danzin/projects/repro_jit_cpython/Modules/main.c:429:15
    #17 0x5df3f5ccc223 in pymain_run_python /home/danzin/projects/repro_jit_cpython/Modules/main.c:691:21
    #18 0x5df3f5ccc223 in Py_RunMain /home/danzin/projects/repro_jit_cpython/Modules/main.c:772:5
    #19 0x5df3f5ccd126 in pymain_main /home/danzin/projects/repro_jit_cpython/Modules/main.c:802:12
    #20 0x5df3f5ccd297 in Py_BytesMain /home/danzin/projects/repro_jit_cpython/Modules/main.c:826:12
    #21 0x795232c2a574 in __libc_start_call_main csu/../sysdeps/nptl/libc_start_call_main.h:58:16
    #22 0x795232c2a627 in __libc_start_main csu/../csu/libc-start.c:360:3
    #23 0x5df3f4ff34f4 in _start (/home/danzin/projects/repro_jit_cpython/python+0x2e14f4) (BuildId: 363d75eb6d935a25f4aa05007d344db781183568)

==1542863==Register values:
rax = 0xffffffffcdcdcdcd  rbx = 0x0000772231e3c8a0  rcx = 0xcdcdcdcdcdcdce9d  rdx = 0x0000772231e3c908
rdi = 0x0000772231e3c900  rsi = 0x00000ee4463c7920  rbp = 0x00007fff285f0ce0  rsp = 0x00007fff285f0cb0
 r8 = 0x0000000000000000   r9 = 0x00000f2a46601c75  r10 = 0x19b9b9b9ad2d2d46  r11 = 0x00000bbe7ec885b0
r12 = 0x0000000000000008  r13 = 0x0000765231e1dc60  r14 = 0xcdcdcdcd69696a37  r15 = 0xcdcdcdcdcdcdcdcd
AddressSanitizer can not provide additional info.
SUMMARY: AddressSanitizer: SEGV /home/danzin/projects/repro_jit_cpython/Python/optimizer.c:1765:5 in _Py_ExecutorDetach
==1542863==ABORTING

Backtrace:

Program received signal SIGSEGV, Segmentation fault.
0x0000555556406544 in _Py_ExecutorDetach (executor=executor@entry=0x7dcff6e3c8a0) at Python/optimizer.c:1765
1765        assert(instruction->op.code == ENTER_EXECUTOR);
(gdb) bt
#0  0x0000555556406544 in _Py_ExecutorDetach (executor=executor@entry=0x7dcff6e3c8a0) at Python/optimizer.c:1765
#1  0x0000555556407463 in executor_invalidate (op=0x7dcff6e3c8a0) at Python/optimizer.c:1789
#2  0x0000555556400fee in uop_dealloc (op=0x7dcff6e3c900) at Python/optimizer.c:345
#3  0x0000555555c1e247 in _Py_Dealloc (op=0x7dcff6e3c8a0) at Objects/object.c:3207
#4  0x00005555563fe498 in _PyOptimizer_Optimize (frame=frame@entry=0x7e8ff6de5378, tstate=tstate@entry=0x555556ed3030 <_PyRuntime+359408>) at Python/optimizer.c:188
#5  0x0000555555eda0a4 in stop_tracing_and_jit (tstate=0x555556ed3030 <_PyRuntime+359408>, frame=frame@entry=0x7e8ff6de5378) at Python/ceval.c:1461
#6  0x0000555555e9693a in _PyEval_EvalFrameDefault (tstate=<optimized out>, frame=<optimized out>, throwflag=<optimized out>) at Python/generated_cases.c.h:11868
#7  0x0000555555e72e68 in _PyEval_EvalFrame (tstate=0x555556ed3030 <_PyRuntime+359408>, frame=0x7e8ff6de5220, throwflag=0) at ./Include/internal/pycore_ceval.h:118
#8  _PyEval_Vector (tstate=<optimized out>, func=<optimized out>, locals=<optimized out>, args=<optimized out>, argcount=<optimized out>, kwnames=0x0) at Python/ceval.c:2541
#9  0x0000555555e72885 in PyEval_EvalCode (co=<optimized out>, globals=<optimized out>, locals=0x7c7ff6e863c0) at Python/ceval.c:1005
#10 0x000055555649d24f in run_eval_code_obj (tstate=tstate@entry=0x555556ed3030 <_PyRuntime+359408>, co=co@entry=0x7d3ff6e15e50, globals=globals@entry=0x7c7ff6e863c0,
    locals=locals@entry=0x7c7ff6e863c0) at Python/pythonrun.c:1366
#11 0x000055555649c41c in run_mod (mod=<optimized out>, filename=<optimized out>, globals=<optimized out>, locals=<optimized out>, flags=<optimized out>, arena=<optimized out>,
    interactive_src=<optimized out>, generate_new_source=<optimized out>) at Python/pythonrun.c:1469
#12 0x0000555556496a1d in pyrun_file (fp=fp@entry=0x7d4ff6defa80, filename=filename@entry=0x7ccff6dfaef0, start=start@entry=257, globals=globals@entry=0x7c7ff6e863c0,
    locals=locals@entry=0x7c7ff6e863c0, closeit=closeit@entry=1, flags=0x7bfff5c1e910) at Python/pythonrun.c:1294
#13 0x000055555649457d in _PyRun_SimpleFileObject (fp=<optimized out>, filename=<optimized out>, closeit=<optimized out>, flags=<optimized out>) at Python/pythonrun.c:518
#14 0x00005555564938ee in _PyRun_AnyFileObject (fp=fp@entry=0x7d4ff6defa80, filename=filename@entry=0x7ccff6dfaef0, closeit=closeit@entry=1, flags=flags@entry=0x7bfff5c1e910)
    at Python/pythonrun.c:81
#15 0x000055555651015b in pymain_run_file_obj (program_name=0x7caff6e230d0, filename=0x7ccff6dfaef0, skip_source_first_line=0) at Modules/main.c:410
#16 pymain_run_file (config=0x555556e9e0f8 <_PyRuntime+142520>) at Modules/main.c:429
#17 0x000055555650e224 in pymain_run_python (exitcode=0x7bfff5c1e500) at Modules/main.c:691
#18 Py_RunMain () at Modules/main.c:772
#19 0x000055555650f127 in pymain_main (args=<optimized out>) at Modules/main.c:802
#20 0x000055555650f298 in Py_BytesMain (argc=<optimized out>, argv=0x7fffffffdbb8) at Modules/main.c:826
#21 0x00007ffff7c2a575 in __libc_start_call_main (main=main@entry=0x55555591ec40 <main>, argc=argc@entry=2, argv=argv@entry=0x7fffffffdbb8) at ../sysdeps/nptl/libc_start_call_main.h:58
#22 0x00007ffff7c2a628 in __libc_start_main_impl (main=0x55555591ec40 <main>, argc=2, argv=0x7fffffffdbb8, init=<optimized out>, fini=<optimized out>, rtld_fini=<optimized out>,
    stack_end=0x7fffffffdba8) at ../csu/libc-start.c:360
#23 0x00005555558354f5 in _start ()

Output of running with PYTHON_LLTRACE=4 PYTHON_OPT_DEBUG=4:
143814_segfault_lltrace_opt_debug.txt

@Fidget-Spinner
Copy link
Member Author

Fidget-Spinner commented Jan 31, 2026

Thanks @devdanzin. The backtrace looks like the original repro for that other issue that you said fixed it thougj, so thats strange

In any case, this is an actual logic bug we should fix. So even if it doesnt manifest for us we should fix it.

markshannon
markshannon reviewed Feb 2, 2026
View reviewed changes
Python/optimizer.c
}
assert(size <= capacity);
if (size == capacity) {
if (capacity * 2 >= MAX_EXECUTORS_SIZE) {
Copy link
Member

@markshannon markshannon Feb 2, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This would be easier to follow if we did the test on new_capacity below:

if (new_capacity >= MAX_EXECUTORS_SIZE) {
     return -1;

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@markshannon markshannon markshannon left review comments

@savannahostrowski savannahostrowski Awaiting requested review from savannahostrowski

Assignees

No one assigned

Labels

awaiting core review

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

4 participants

@Fidget-Spinner @savannahostrowski @devdanzin @markshannon