Virtual environment breaks `test_sysconfig` and `test_venv` tests on macOS

[artemmukhin]

Bug report

1. Build CPython on macOS
2. Activate a virtual environment as described in https://devguide.python.org/testing/coverage/#install-coverage
3. Run python -m test

Results in:

== Tests result: FAILURE ==

2 tests failed:
    test_sysconfig test_venv

$ python -m test test_venv --verbose
...
AssertionError: Lists differ

First differing element 0:
'/.../bin/python.exe'
'/.../bin/python'

TODO

• Fix test_sysconfig gh-103224: Resolve paths properly in test_sysconfig #103292
• Fix test_venv GH-103224: Use the realpath of the Python executable in test_venv #103243

Your environment

• CPython versions tested on: main
• Operating system and architecture: macOS 13.2.1

Linked PRs

• GH-103224: Use the realpath of the Python executable in test_venv #103243
• gh-103224: Resolve paths properly in test_sysconfig #103292
• Revert "gh-103224: Use the realpath of the Python executable in test_venv (GH-103243)" #108667
• [3.12] gh-103224: Resolve paths properly in test_sysconfig (GH-103292) #115100
• [3.11] gh-103224: Resolve paths properly in test_sysconfig (GH-103292) #115101

[dtrodrigues]

What if you do python.exe -m test or python.exe -m test test_venv --verbose?

[artemmukhin]

@dtrodrigues Both python.exe -m test and python.exe -m test test_venv --verbose finish without failures. test_sysconfig passes only when I run ./python.exe and not the symlink python.exe.

I have opened a PR regarding test_venv.

[artemmukhin]

Regarding test_sysconfig, I found out that replacing sys.prefix with sys.base_prefix makes test_sysconfig passes with both ./python.exe and python.exe symlink.

Although I'm not entirely sure if this is the correct way to fix the test. So before opening the next PR regarding test_sysconfig, I would like to confirm this is the appropriate solution.

@hroncok Could you please take a look if you have a chance? Here is a patch:

diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py
index b6dbf3d52c..a95e7f4367 100644
--- a/Lib/test/test_sysconfig.py
+++ b/Lib/test/test_sysconfig.py
@@ -151,7 +151,7 @@ def test_posix_venv_scheme(self):

         # Resolve the paths in prefix
         binpath = os.path.join(sys.prefix, binpath)
-        incpath = os.path.join(sys.prefix, incpath)
+        incpath = os.path.join(sys.base_prefix, incpath)
         libpath = os.path.join(sys.prefix, libpath)

         self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='posix_venv'))
@@ -171,7 +171,7 @@ def test_nt_venv_scheme(self):

         # Resolve the paths in prefix
         binpath = os.path.join(sys.prefix, binpath)
-        incpath = os.path.join(sys.prefix, incpath)
+        incpath = os.path.join(sys.base_prefix, incpath)
         libpath = os.path.join(sys.prefix, libpath)

         self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='nt_venv'))

[hroncok]

Will do! It has been a while since I wrote that, but I'll try to figure it out.

[hroncok]

The include path is indeed defined via installed_base and the default for insatlled_base is sys.base_prefix, so the diff makes the test pass but arguably makes the test more fragile for potential future posix_venv changes.

I suggest instead of "Resolve the paths in prefix" we resolve the paths in "venv/" and then we set vars to a dict similar to the one in

cpython/Lib/venv/__init__.py

Lines 97 to 102 in f513d5c

	vars = {
	'base': env_dir,
	'platbase': env_dir,
	'installed_base': env_dir,
	'installed_platbase': env_dir,
	}

when calling get_path.

Something like this (untested):

diff --git a/Lib/test/test_sysconfig.py b/Lib/test/test_sysconfig.py
index b6dbf3d52cb..1137c2032b9 100644
--- a/Lib/test/test_sysconfig.py
+++ b/Lib/test/test_sysconfig.py
@@ -149,17 +149,21 @@ def test_posix_venv_scheme(self):
                                'python%d.%d' % sys.version_info[:2],
                                'site-packages')
 
-        # Resolve the paths in prefix
-        binpath = os.path.join(sys.prefix, binpath)
-        incpath = os.path.join(sys.prefix, incpath)
-        libpath = os.path.join(sys.prefix, libpath)
+        # Resolve the paths in an imaginary venv/ directory
+        binpath = os.path.join('venv', binpath)
+        incpath = os.path.join('venv', incpath)
+        libpath = os.path.join('venv', libpath)
 
-        self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='posix_venv'))
-        self.assertEqual(libpath, sysconfig.get_path('purelib', scheme='posix_venv'))
+        # Mimic the venv module, set all bases to the venv directory
+        bases = ('base', 'platbase', 'installed_base', 'installed_platbase')
+        vars = {base: 'venv' for base in bases}
+
+        self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='posix_venv', vars=vars))
+        self.assertEqual(libpath, sysconfig.get_path('purelib', scheme='posix_venv', vars=vars))
 
         # The include directory on POSIX isn't exactly the same as before,
         # but it is "within"
-        sysconfig_includedir = sysconfig.get_path('include', scheme='posix_venv')
+        sysconfig_includedir = sysconfig.get_path('include', scheme='posix_venv', vars=vars)
         self.assertTrue(sysconfig_includedir.startswith(incpath + os.sep))
 
     def test_nt_venv_scheme(self):
@@ -169,14 +173,19 @@ def test_nt_venv_scheme(self):
         incpath = 'Include'
         libpath = os.path.join('Lib', 'site-packages')
 
-        # Resolve the paths in prefix
-        binpath = os.path.join(sys.prefix, binpath)
-        incpath = os.path.join(sys.prefix, incpath)
-        libpath = os.path.join(sys.prefix, libpath)
+        # Resolve the paths in an imaginary venv\ directory
+        venv = 'venv'
+        binpath = os.path.join(venv, binpath)
+        incpath = os.path.join(venv, incpath)
+        libpath = os.path.join(venv, libpath)
+
+        # Mimic the venv module, set all bases to the venv directory
+        bases = ('base', 'platbase', 'installed_base', 'installed_platbase')
+        vars = {base: 'venv' for base in bases}
 
-        self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='nt_venv'))
-        self.assertEqual(incpath, sysconfig.get_path('include', scheme='nt_venv'))
-        self.assertEqual(libpath, sysconfig.get_path('purelib', scheme='nt_venv'))
+        self.assertEqual(binpath, sysconfig.get_path('scripts', scheme='nt_venv', vars=vars))
+        self.assertEqual(incpath, sysconfig.get_path('include', scheme='nt_venv', vars=vars))
+        self.assertEqual(libpath, sysconfig.get_path('purelib', scheme='nt_venv', vars=vars))
 
     def test_venv_scheme(self):
         if sys.platform == 'win32':

I think that this should have been done initially and my test was buggy.

[artemmukhin]

@hroncok Thank you! Your patch fixes the tests for me, so I have opened a PR #103292