Memory leak in test_sys with subinterpreters creation (AddressSanitizer detection)

[ashm-dev]

Bug report

Bug description:

Bug Description:

I've discovered a memory leak when running test_sys with AddressSanitizer enabled. The leak occurs during subinterpreter creation and the allocated memory is never freed.

Environment:

• CPython versions affected: 3.14 and 3.15 (main branch)
• Operating System: Linux (Ubuntu-based)
• Build configuration:

CC=clang CXX=clang++ ./configure \
    --disable-optimizations \
    --with-valgrind \
    --with-pydebug \
    --enable-pystats \
    --with-address-sanitizer

Steps to Reproduce:

1. Build CPython with the configuration above (tested on both 3.14 and main branches)
2. Run: ./python -m test test_sys
3. Observe AddressSanitizer output at the end

Expected Behavior:
No memory leaks should be detected by AddressSanitizer.

Actual Behavior:
AddressSanitizer reports approximately 4.5 MB leaked across 20 allocations. All allocations trace back to _PyInterpreterState_New → alloc_interpreter → _PyMem_DebugRawCalloc.

The test itself passes successfully, but the leak persists:

== Tests result: SUCCESS ==
1 test OK.
=================================================================
ERROR: LeakSanitizer: detected memory leaks

Reproduction on Multiple Versions:

• 3.14 branch: 4.5 MB leaked (4517100 bytes in 20 allocations)
• 3.15 (main): 4.5 MB leaked (4516460 bytes in 20 allocations)

Additional Context:

The leak appears to be related to subinterpreter lifecycle management. The stack trace shows memory is allocated through:

• interp_create (Modules/_interpretersmodule.c)
• _PyXI_NewInterpreter (Python/crossinterp.c:3204)
• Py_NewInterpreterFromConfig (Python/pylifecycle.c)
• new_interpreter (Python/pylifecycle.c)
• _PyInterpreterState_New (Python/pystate.c)
• alloc_interpreter (Python/pystate.c)

This issue appears to be related to #110411, which remains open with the latest report from August 2025 stating "This is still a problem in 3.13.7 and 3.14.0rc2". Issue #113055 was closed as resolved in February 2025, but the leak still occurs on current 3.14 and main branches.

Complete AddressSanitizer Output (3.15 main branch)

➜  main git:(main) sudo ./build/main/python -m test test_sys
Using random seed: 1335876681
0:00:00 load avg: 25.72 Run 1 test sequentially in a single process
0:00:00 load avg: 25.72 [1/1] test_sys
0:00:20 load avg: 27.57 [1/1] test_sys passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 20.6 sec
Total tests: run=92 skipped=6
Total test files: run=1/1
Result: SUCCESS

=================================================================
==544756==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4290637 byte(s) in 19 object(s) allocated from:
    #0 0x5c153a3eef9d in calloc (/home/shamil/oss/cpython/main/build/main/python+0x33bf9d) (BuildId: c97dc91acec24b8d7d7a9344676b1bb0726ae568)
    #1 0x5c153a777a5e in _PyMem_DebugRawAlloc /home/shamil/oss/cpython/main/build/main/../../Objects/obmalloc.c:2884:24
    #2 0x5c153a777a5e in _PyMem_DebugRawCalloc /home/shamil/oss/cpython/main/build/main/../../Objects/obmalloc.c:2929:12
    #3 0x5c153abd8578 in alloc_interpreter /home/shamil/oss/cpython/main/build/main/../../Python/pystate.c:462:17
    #4 0x5c153abd8578 in _PyInterpreterState_New /home/shamil/oss/cpython/main/build/main/../../Python/pystate.c:660:18
    #5 0x5c153ab8e818 in new_interpreter /home/shamil/oss/cpython/main/build/main/../../Python/pylifecycle.c:2425:14
    #6 0x5c153ab8e592 in Py_NewInterpreterFromConfig /home/shamil/oss/cpython/main/build/main/../../Python/pylifecycle.c:2521:12
    #7 0x5c153aab1b58 in _PyXI_NewInterpreter /home/shamil/oss/cpython/main/build/main/../../Python/crossinterp.c:3204:23
    #8 0x726f325c23ae in _interpreters_create_impl /home/shamil/oss/cpython/main/build/main/../../Modules/_interpretersmodule.c:880:13
    #9 0x726f325c23ae in _interpreters_create /home/shamil/oss/cpython/main/build/main/../../Modules/clinic/_interpretersmodule.c.h:91:20
    #10 0x5c153a5e91c2 in _PyObject_VectorcallTstate /home/shamil/oss/cpython/main/build/main/../../Include/internal/pycore_call.h:169:11
    #11 0x5c153aa11ff6 in _PyEval_EvalFrameDefault /home/shamil/oss/cpython/main/build/main/../../Python/generated_cases.c.h:3188:35
    [... rest of stack trace truncated for brevity ...]

Direct leak of 225823 byte(s) in 1 object(s) allocated from:
    #0 0x5c153a3eef9d in calloc (/home/shamil/oss/cpython/main/build/main/python+0x33bf9d) (BuildId: c97dc91acec24b8d7d7a9344676b1bb0726ae568)
    #1 0x5c153a777a5e in _PyMem_DebugRawAlloc /home/shamil/oss/cpython/main/build/main/../../Objects/obmalloc.c:2884:24
    #2 0x5c153a777a5e in _PyMem_DebugRawCalloc /home/shamil/oss/cpython/main/build/main/../../Objects/obmalloc.c:2929:12
    #3 0x5c153abd8578 in alloc_interpreter /home/shamil/oss/cpython/main/build/main/../../Python/pystate.c:462:17
    #4 0x5c153abd8578 in _PyInterpreterState_New /home/shamil/oss/cpython/main/build/main/../../Python/pystate.c:660:18
    [... rest of stack trace truncated for brevity ...]

SUMMARY: AddressSanitizer: 4516460 byte(s) leaked in 20 allocation(s).

Complete AddressSanitizer Output (3.14 branch)

➜  3.14 git:(3.14) sudo ./python -m test test_sys
Using random seed: 641571276
0:00:00 load avg: 2.97 Run 1 test sequentially in a single process
0:00:00 load avg: 2.97 [1/1] test_sys
0:00:06 load avg: 2.66 [1/1] test_sys passed

== Tests result: SUCCESS ==

1 test OK.

Total duration: 6.7 sec
Total tests: run=97 skipped=7
Total test files: run=1/1
Result: SUCCESS

=================================================================
==570353==ERROR: LeakSanitizer: detected memory leaks

Direct leak of 4291245 byte(s) in 19 object(s) allocated from:
    #0 0x5ea8303c6f8d in calloc (/home/shamil/oss/cpython/3.14/python+0x325f8d) (BuildId: bd4e3657446f867dffe5f140bfa56307be2c714a)
    #1 0x5ea83074b5be in _PyMem_DebugRawAlloc /home/shamil/oss/cpython/3.14/Objects/obmalloc.c:2869:24
    #2 0x5ea83074b5be in _PyMem_DebugRawCalloc /home/shamil/oss/cpython/3.14/Objects/obmalloc.c:2914:12
    #3 0x5ea830b8f5ec in alloc_interpreter /home/shamil/oss/cpython/3.14/Python/pystate.c:570:17
    #4 0x5ea830b8f5ec in _PyInterpreterState_New /home/shamil/oss/cpython/3.14/Python/pystate.c:757:18
    #5 0x5ea830b5a2c8 in new_interpreter /home/shamil/oss/cpython/3.14/Python/pylifecycle.c:2313:14
    #6 0x5ea830b5a042 in Py_NewInterpreterFromConfig /home/shamil/oss/cpython/3.14/Python/pylifecycle.c:2409:12
    #7 0x5ea830a7cd78 in _PyXI_NewInterpreter /home/shamil/oss/cpython/3.14/Python/crossinterp.c:3204:23
    #8 0x7be02a35a2a8 in interp_create /home/shamil/oss/cpython/3.14/./Modules/_interpretersmodule.c:862:13
    [... rest of stack trace truncated for brevity ...]

Direct leak of 225855 byte(s) in 1 object(s) allocated from:
    #0 0x5ea8303c6f8d in calloc (/home/shamil/oss/cpython/3.14/python+0x325f8d) (BuildId: bd4e3657446f867dffe5f140bfa56307be2c714a)
    #1 0x5ea83074b5be in _PyMem_DebugRawAlloc /home/shamil/oss/cpython/3.14/Objects/obmalloc.c:2869:24
    #2 0x5ea83074b5be in _PyMem_DebugRawCalloc /home/shamil/oss/cpython/3.14/Objects/obmalloc.c:2914:12
    #3 0x5ea830b8f5ec in alloc_interpreter /home/shamil/oss/cpython/3.14/Python/pystate.c:570:17
    #4 0x5ea830b8f5ec in _PyInterpreterState_New /home/shamil/oss/cpython/3.14/Python/pystate.c:757:18
    [... rest of stack trace truncated for brevity ...]

SUMMARY: AddressSanitizer: 4517100 byte(s) leaked in 20 allocation(s).

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

• gh-140067: Fix memory leak in sub-interpreter creation  #140111
• [3.14] gh-140067: Fix memory leak in sub-interpreter creation (GH-140111) #140118
• Revert "gh-140067: Fix memory leak in sub-interpreter creation (#140111)" #140140
• gh-140067: Fix memory leak in subinterpreter creation #140148
• gh-140067: Fix memory leak in sub-interpreter creation (#140111)  #140261

[picnixz]

@ericsnowcurrently It appears we still have a leak in subinterpreters. It also looks like similar to #110411 but not really a duplicate considering the traceback.

[ashm-dev]

I'll try to fix this leak, but no promises.

[picnixz]

Let us first try to check if this is something that is known to Eric or not before jumping onto a PR. It would avoid having to make unnecessary work. It's fine to report an issue, wait for an expert to have a look at it, and only then consider a PR.

[ashm-dev]

Alright, got it, I’ll do that. Thanks!

[kumaraditya303]

I think this is a duplicate of #109700, I was looking into this a while ago I'll check if I had reached a fix.

[kumaraditya303]

I had git bisected it and I remember that #132428 was the first bad commit in my testing.

[ashm-dev]

@kumaraditya303 I solved this problem with these changes.
ashm-dev@42ff41a

[kumaraditya303]

Ah yes, the problem is that malloced pointer is being overwritten here, I came up with a different solution just now but it does the same as yours.

diff --git a/Include/internal/pycore_interp_structs.h b/Include/internal/pycore_interp_structs.h
index 2124e76514f..badc97808c6 100644
--- a/Include/internal/pycore_interp_structs.h
+++ b/Include/internal/pycore_interp_structs.h
@@ -769,12 +769,6 @@ struct _is {
      * and should be placed at the beginning. */
     struct _ceval_state ceval;
 
-    /* This structure is carefully allocated so that it's correctly aligned
-     * to avoid undefined behaviors during LOAD and STORE. The '_malloced'
-     * field stores the allocated pointer address that will later be freed.
-     */
-    void *_malloced;
-
     PyInterpreterState *next;
 
     int64_t id;
diff --git a/Python/pystate.c b/Python/pystate.c
index dbed609f29a..fb7f8177ab2 100644
--- a/Python/pystate.c
+++ b/Python/pystate.c
@@ -457,16 +457,19 @@ _PyInterpreterState_Enable(_PyRuntimeState *runtime)
 static PyInterpreterState *
 alloc_interpreter(void)
 {
+    // Aligned allocation for PyInterpreterState.
+    // the first word of the memory block is used to store
+    // the original pointer to be used later to free the memory.
     size_t alignment = _Alignof(PyInterpreterState);
-    size_t allocsize = sizeof(PyInterpreterState) + alignment - 1;
+    size_t allocsize = sizeof(PyInterpreterState) + sizeof(void*) + alignment - 1;
     void *mem = PyMem_RawCalloc(1, allocsize);
     if (mem == NULL) {
         return NULL;
     }
-    PyInterpreterState *interp = _Py_ALIGN_UP(mem, alignment);
-    assert(_Py_IS_ALIGNED(interp, alignment));
-    interp->_malloced = mem;
-    return interp;
+    void *ptr = _Py_ALIGN_UP((char *)mem + sizeof(void*), alignment);
+    ((void **)ptr)[-1] = mem;
+    assert(_Py_IS_ALIGNED(ptr, alignment));
+    return ptr;
 }
 
 static void
@@ -481,7 +484,7 @@ free_interpreter(PyInterpreterState *interp)
             interp->obmalloc = NULL;
         }
         assert(_Py_IS_ALIGNED(interp, _Alignof(PyInterpreterState)));
-        PyMem_RawFree(interp->_malloced);
+        PyMem_RawFree(((void **)interp)[-1]);
     }
 }
 

[kumaraditya303]

@ashm-dev I have slight preference to my fix as it avoids the malloced pointer altogether so in future there would not be such mistake of overwriting the pointer. Would you like to create a PR based on the changes?