Data race on a shared types.GenericAlias iterator in ga_iter_reduce under free-threading (follow-up to gh-154043)

Bug report

Bug description:

This is a follow-up to #154043 (segfault in ga_iternext when a types.GenericAlias iterator is shared across threads), which was fixed by gh-154108 by consuming gi->obj atomically in ga_iternext:

cpython/Objects/genericaliasobject.c

Lines 942 to 963 in 8b048eb

    
           ga_iternext(PyObject *op) 
        
           { 
        
               gaiterobject *gi = (gaiterobject*)op; 
        
           #ifdef Py_GIL_DISABLED 
        
               PyObject *obj = _Py_atomic_exchange_ptr(&gi->obj, NULL); 
        
           #else 
        
               PyObject* obj = gi->obj; 
        
               gi->obj = NULL; 
        
           #endif 
        
               if (obj == NULL) { 
        
                   PyErr_SetNone(PyExc_StopIteration); 
        
                   return NULL; 
        
               } 
        
               gaobject *alias = (gaobject *)obj; 
        
               PyObject *starred_alias = Py_GenericAlias(alias->origin, alias->args); 
        
               Py_DECREF(obj); 
        
               if (starred_alias == NULL) { 
        
                   return NULL; 
        
               } 
        
               ((gaobject *)starred_alias)->starred = true; 
        
               return starred_alias; 
        
           }

That fix only covers ga_iternext. The iterator's __reduce__ implementation, ga_iter_reduce, still reads the same gi->obj field with plain (non-atomic) loads and no critical section:

cpython/Objects/genericaliasobject.c

Lines 990 to 1004 in 8b048eb

    
           static PyObject * 
        
           ga_iter_reduce(PyObject *self, PyObject *Py_UNUSED(ignored)) 
        
           { 
        
               PyObject *iter = _PyEval_GetBuiltin(&_Py_ID(iter)); 
        
               gaiterobject *gi = (gaiterobject *)self; 
        
               /* _PyEval_GetBuiltin can invoke arbitrary code, 
        
                * call must be before access of iterator pointers. 
        
                * see issue #101765 */ 
        
               if (gi->obj) 
        
                   return Py_BuildValue("N(O)", iter, gi->obj); 
        
               else 
        
                   return Py_BuildValue("N(())", iter); 
        
           }

So sharing one iter(list[int]) across threads and calling __reduce__ concurrently with next() is still a data race on gi->obj, the same field the fix made atomic in ga_iternext. ga_iternext does _Py_atomic_exchange_ptr(&gi->obj, NULL) (atomic write) while ga_iter_reduce does if (gi->obj) return Py_BuildValue("N(O)", iter, gi->obj); (plain read).

from threading import Thread, Barrier
_shared_iter = iter(list[int])

def chain1_thread():
    for _ in range(20000):
        try:
            next(_shared_iter)
        except StopIteration:
            pass

def chain2_thread():
    for _ in range(20000):
        try:
            _shared_iter.__reduce__()
        except Exception:
            pass

N_C1 = 4
N_C2 = 4
barrier = Barrier(N_C1 + N_C2)

def _c1():
    barrier.wait()
    chain1_thread()

def _c2():
    barrier.wait()
    chain2_thread()

threads  = [Thread(target=_c1) for _ in range(N_C1)]
threads += [Thread(target=_c2) for _ in range(N_C2)]
for t in threads: t.start()
for t in threads: t.join()

TSAN Report:

WARNING: ThreadSanitizer: data race (pid=650295)
  Write of size 8 at 0x7fffb6161af0 by thread T1:
    #0 _Py_atomic_exchange_ptr /home/fuzz_cpython/cpython-latest/cpython/./Include/cpython/pyatomic_gcc.h:195:10 
    #1 ga_iternext /home/fuzz_cpython/cpython-latest/cpython/Objects/genericaliasobject.c:946:21 
    #2 builtin_next /home/fuzz_cpython/cpython-latest/cpython/Python/bltinmodule.c:1776:11 
    #3 cfunction_vectorcall_FASTCALL /home/fuzz_cpython/cpython-latest/cpython/Objects/methodobject.c:449:24 
    #4 _PyObject_VectorcallTstate /home/fuzz_cpython/cpython-latest/cpython/./Include/internal/pycore_call.h:144:11 
    #5 PyObject_Vectorcall /home/fuzz_cpython/cpython-latest/cpython/Objects/call.c:327:12 
    #6 _Py_VectorCallInstrumentation_StackRefSteal /home/fuzz_cpython/cpython-latest/cpython/Python/ceval.c:768:11 
    #7 _PyEval_EvalFrameDefault /home/fuzz_cpython/cpython-latest/cpython/Python/generated_cases.c.h:1906:35
...

  Previous read of size 8 at 0x7fffb6161af0 by thread T8:
    #0 ga_iter_reduce /cpython/Objects/genericaliasobject.c:1000:13 
    #1 _PyEval_EvalFrameDefault /cpython/Python/generated_cases.c.h:4330:35 
...

SUMMARY: ThreadSanitizer: data race /cpython/./Include/cpython/pyatomic_gcc.h:195:10 in _Py_atomic_exchange_ptr

CPython versions tested on:

CPython main branch

Operating systems tested on:

Linux

Linked PRs

Thanks for your report! I created #155476 to fix the issue.

gh-154916: Fix data races in GenericAlias using critical sections (#1…

pythongh-154916: Fix data races in GenericAlias using critical sectio…

[3.15] gh-154916: Fix data races in GenericAlias using critical secti…

Navigation Menu

Repository navigation

Data race on a shared `types.GenericAlias` iterator in `ga_iter_reduce` under free-threading (follow-up to gh-154043) #154916

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

	ga_iternext(PyObject *op)
	{
	gaiterobject gi = (gaiterobject)op;
	#ifdef Py_GIL_DISABLED
	PyObject *obj = _Py_atomic_exchange_ptr(&gi->obj, NULL);
	#else
	PyObject* obj = gi->obj;
	gi->obj = NULL;
	#endif
	if (obj == NULL) {
	PyErr_SetNone(PyExc_StopIteration);
	return NULL;
	}
	gaobject alias = (gaobject )obj;
	PyObject *starred_alias = Py_GenericAlias(alias->origin, alias->args);
	Py_DECREF(obj);
	if (starred_alias == NULL) {
	return NULL;
	}
	((gaobject *)starred_alias)->starred = true;
	return starred_alias;
	}

	static PyObject *
	ga_iter_reduce(PyObject self, PyObject Py_UNUSED(ignored))
	{
	PyObject *iter = _PyEval_GetBuiltin(&_Py_ID(iter));
	gaiterobject gi = (gaiterobject )self;

	/* _PyEval_GetBuiltin can invoke arbitrary code,
	* call must be before access of iterator pointers.
	* see issue #101765 */

	if (gi->obj)
	return Py_BuildValue("N(O)", iter, gi->obj);
	else
	return Py_BuildValue("N(())", iter);
	}

Data race on a shared types.GenericAlias iterator in ga_iter_reduce under free-threading (follow-up to gh-154043) #154916

Description

Bug report

Bug description:

CPython versions tested on:

Operating systems tested on:

Linked PRs

Activity

IvyXu420 commented on Aug 10, 2026

Metadata

Metadata

Assignees

Labels

Projects

Milestone

Relationships

Development

Issue actions

Data race on a shared `types.GenericAlias` iterator in `ga_iter_reduce` under free-threading (follow-up to gh-154043) #154916