Maximum recursion depth exceeded in __getattr__().

[felixxm]

Bug report

We're hitting RecursionError: maximum recursion depth exceeded in Django test suite with Python 3.12.0a7 when accessing an attribute with a custom __getattr__() method:

ERROR: test_access_warning (deprecation.test_storages.DefaultStorageDeprecationTests.test_access_warning)
----------------------------------------------------------------------
Traceback (most recent call last):
  File "/django/tests/deprecation/test_storages.py", line 128, in test_access_warning
    settings.DEFAULT_FILE_STORAGE
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  File "/django/django/conf/__init__.py", line 83, in __getattr__
    if (_wrapped := self._wrapped) is empty:
                    ^^^^^^^^^^^^^
  [Previous line repeated 790 more times]
RecursionError: maximum recursion depth exceeded

See affected test and LazySettings.__getattr__().

Bisected to the aa0a73d.

I'd try to prepare a small regression test.

Your environment

• CPython versions tested on: Python 3.12.0a7
• Operating system and architecture: x86_64 GNU/Linux

Linked PRs

• gh-103272: add regression test for getattr that raises #103336
• [3.11] gh-103272: regression test for getattr exception in property (GH-103336) #103351

[AlexWaygood]

Bisected to the aa0a73d.

Cc. @wangxiang-hz and @Fidget-Spinner

[Fidget-Spinner]

Thanks for finding this. I'll try to debug once there's a minimal reproducer. Sorry I'm a little swamped for the next 2 weeks.

[sunmy2019]

NB: when running this test with debug build, it crashes.

Testing against Django installed in '/home/ubuntu/django-main/django' with up to 8 processes
Found 6 test(s).
Creating test database for alias 'default'...
System check identified no issues (0 silenced).
.python: Python/ceval.c:821: _PyEval_EvalFrameDefault: Assertion `!_PyErr_Occurred(tstate)' failed.
Fatal Python error: Aborted

Current thread 0x00007f2f2750e740 (most recent call first):
  File "/home/ubuntu/django-main/django/conf/__init__.py", line 81 in __getattr__
  File "/home/ubuntu/django-main/tests/deprecation/test_storages.py", line 128 in test_access_warning
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 589 in _callTestMethod
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 634 in run
  File "/home/ubuntu/cpython-main/Lib/unittest/case.py", line 690 in __call__
  File "/home/ubuntu/django-main/django/test/testcases.py", line 292 in _setup_and_call
  File "/home/ubuntu/django-main/django/test/testcases.py", line 257 in __call__
  File "/home/ubuntu/cpython-main/Lib/unittest/suite.py", line 122 in run
  File "/home/ubuntu/cpython-main/Lib/unittest/suite.py", line 84 in __call__
  File "/home/ubuntu/cpython-main/Lib/unittest/runner.py", line 240 in run
  File "/home/ubuntu/django-main/django/test/runner.py", line 972 in run_suite
  File "/home/ubuntu/django-main/django/test/runner.py", line 1044 in run_tests
  File "/home/ubuntu/django-main/tests/./runtests.py", line 429 in django_tests
  File "/home/ubuntu/django-main/tests/./runtests.py", line 770 in <module>

Extension modules: pywatchman.bser, markupsafe._speedups (total: 2)
Aborted (core dumped)

[sunmy2019]

I got a minimal repo!

class A:
    def __init__(self) -> None:
        self.bar = 0

    def __getattribute__(self, name):
        return super().__getattribute__(name)

    def __getattr__(self, name):
        if self.bar == 0:
            raise ValueError

    @property
    def foo(self):
        return self.__getattr__("foo")


A().foo

Running in older Python version, we got

Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 17, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
  File "/home/ubuntu/django-main/tests/../a.py", line 14, in foo
    return self.__getattr__("foo")
  File "/home/ubuntu/django-main/tests/../a.py", line 10, in __getattr__
    raise ValueError
ValueError

Call flow:

__getattribute__ called with name='foo'
property `foo` called

__getattribute__ called with name='__getattr__'
__getattr__ called with name='foo'
__getattribute__ called with name='bar'

But with the new python in main branch, (with out pydebug)

Exception ignored in tp_clear of: <class 'type'>
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
SystemError: <method-wrapper '__getattribute__' of A object at 0x7fc8af29e090> returned a result with an exception set
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 17, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    if self.bar == 0:
       ^^^^^^^^
  [Previous line repeated 71 more times]
  File "/home/ubuntu/django-main/tests/../a.py", line 10, in __getattr__
    raise ValueError
ValueError

More precuriously, when I try to add some debug lines like this:

class A:
    def __init__(self) -> None:
        self.bar = 0

    def __getattribute__(self, name):
        return super().__getattribute__(name)

    def __getattr__(self, name):
        print(name)   ######### <----------- Here
        if self.bar == 0:
            raise ValueError

    @property
    def foo(self):
        return self.__getattr__("foo")


A().foo

The new python outputs,

foo
Traceback (most recent call last):
  File "/home/ubuntu/django-main/tests/../a.py", line 18, in <module>
    A().foo
  File "/home/ubuntu/django-main/tests/../a.py", line 9, in __getattr__
    print(name)
  File "/home/ubuntu/django-main/tests/../a.py", line 6, in __getattribute__
    return super().__getattribute__(name)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 15, in foo
    return self.__getattr__("foo")
           ^^^^^^^^^^^^^^^^^^^^^^^
  File "/home/ubuntu/django-main/tests/../a.py", line 11, in __getattr__
    raise ValueError
ValueError

Everything just works fine? Looks like something in C is corrupting ❓ ❓ ❓

[sunmy2019]

Now with pydebug on, I got the output from new python: (crashes with/without debug line)

foo
python: Python/ceval.c:821: _PyEval_EvalFrameDefault: Assertion `!_PyErr_Occurred(tstate)' failed.
Aborted (core dumped)

I can confirm, for this minimum repo, crash starts at aa0a73d