sys.path[0] Is Set Differently From the Rest of sys.path

[ericsnowcurrently]

Feature or enhancement

Currently sys.path[0] is set by pymain_run_python() (in Modules/main.c). This happens after pymain_init(), which initializes the runtime, including the rest of sys.path (via getpath.py and site.py). This makes it harder to reason about and introduces extra complexity for subinterpreters. (See gh-109793 and gh-109794.)

We should consider calculating sys.path[0] and setting it to its own PyConfig field via getpath.py, when the rest of the base sys.path is calculated. We may need a later check to verify that there is a matching importer, as pymain_run_python() does. (FWIW, it isn't clear that there's any value to storing the sys.path[0] value on the global _PyPathConfig.)

Also, we currently wait to actually set sys.path[0] (for the main interpreter) until after the readline/rlcompleter modules are imported in pymain_run_python(). We'd need to factor that in.

CC @zooba, @vstinner, @ncoghlan

Linked PRs

• gh-109853: Fix sys.path[0] For Subinterpreters #109994
• [3.12] gh-109853: Fix sys.path[0] For Subinterpreters (gh-109994) #110701

[vstinner]

We should consider calculating sys.path[0] and setting it to its own PyConfig field via getpath.py

There are two cases:

• (A) PyConfig.run_filename is set: pymain_get_importer() computes sys.path[0]
• (B) Otherwise, _PyPathConfig_ComputeSysPath0() computes sys.path[0]

For the case (A), do you want to execute PyImport_GetImporter() twice? Once in Py_InitializeFromConfig(), then again in pymain_run_python()? It's to decide if pymain_run_module() or pymain_run_file() should be used.

For case (B), this code path can be easily moved to Py_InitializeFromConfig(). When I designed and implemented PyConfig, I tried to minimize changes. But apparently, now the dust has settled, and we can go further :-)

[zooba]

I think we should move most of the default sys.path calculation into python.c, including the running of getpath.py (we'd need to expose the ability to create and then close a runtime that can't import anything).

If we're able to fully initialise the search path using only our public APIs, we'll have a much better interface for embedders to use.

[vstinner]

I would love that sys.path would be fully initialized before the site module is loaded. Currently, sys.path is still modified by the site module in many ways, and so python -S gives a different sys.path :-(

site changes:

• Make paths absolute (why not doing that earlier?)
• Add user site directory (is it complicated to move the logic to getpath?)

[zooba]

I expect most of the site module can move into getpath. Venv and pth sure can (though we'd have to defer code execution in pth files until after initialization finishes). Some of the interactive mode features probably can't, but I'd also like to treat those as something specific to python.c and separate from libpython (i.e. part of the Python program not the Python interpreter).