Un-deprecate functional API for importlib resources & add subdirectory support

[encukou]

Feature or enhancement

Proposal:

The importlib.resources functions {open,read}_{text,binary}, path, is_resource and contents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.
They provide a simple API for simple tasks, while the full-featured Traversable API is better suited for complex ones -- especially for implementing new resources-aware loaders.

I'm now in a position where I can add these functions back and support them.

Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:

importlib.resources.read_text('modulename', 'subdirectory', 'subsubdir', 'resource.txt')

The additional arguments (encoding and errors) would become keyword-only.

---

There is a wrinkle in this: in Python 3.9-3.11, the above would mean:

importlib.resources.read_text(
    'modulename', 'subdirectory',
    encoding='subsubdir',
    errors='resource.txt',
)

I believe that this is acceptable, since:

• pragmatically: typical file names do not match typical encoding/errorhandler names
• lawyerly: the functions have already been deprecated for 2 releases; no one is using them now, right?

However, if this is a problem, I can

[edit: This is solved by:]

• make the encoding argument required if a text-reading function more than one path component is given.
• plan to lift this limitation around 3.15.

importlib.resources.read_text(
    'modulename', 'subdirectory', 'subsubdir', 'resource.txt',
    encoding='utf-8',
)
importlib.resources.read_text('modulename', 'resource.txt')  # OK
importlib.resources.read_text('modulename', 'subdirectory', 'utf-8')  # error

Has this already been discussed elsewhere?

I have already discussed this feature proposal on Discourse

Links to previous discussion of this feature:

https://discuss.python.org/t/deprecating-importlib-resources-legacy-api/11386/29

Linked PRs

• gh-116608: Bring back importlib.resources functional API #116609
• gh-116608: Ignore UTF-16 BOM in importlib.resources._functional tests #117569
• gh-116608: Apply style and compatibility changes from importlib_metadata. #123028
• [3.13] gh-116608: Apply style and compatibility changes from importlib_resources. (GH-123028) #123051
• [3.13] gh-116608: Revert "[3.13] gh-116608: Apply style and compatibility changes from importlib_resources. (GH-123028) (#123051)" #123072
• [3.13] gh-116608: Re-apply style and compatibility changes from importlib_resources. (GH-123028) #123074
• [3.12] gh-116608: undeprecate functional importlib.resources API #132206

[FFY00]

Wouldn't this be a bit late for that? We already went through the deprecation period, and removed the feature in the alpha releases, bringing them back now would be a bit confusing.

The importlib.resources functions {open,read}_{text,binary}, path, is_resource and contents, deprecated in 3.11 and removed in 3.13 alphas, are, anecdotally, missed by quite a few users.

Can you actually show a couple examples of this affecting users downstream? I think that's the most viable argument to bring that API back.

[zooba]

taking multiple path components as positional arguments

Why not just take multiple components with separators in a single argument? It's easy enough to require forward slash, disallow .. and even to normalise backslashes on Windows if you feel like it.

If they didn't allow subdirectories before (I never noticed, tbh), then presumably using a slash here would have either failed completely or worked. Either way, we can enable them in a new release.

(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.)

[jaraco]

At least one audience that would like to keep the legacy APIs is in mesonbuild/meson#12401.

I admit, I prefer this approach over keeping the legacy APIs with the cruft that it still had lying around. It adds a mostly-compatible layer and restores these wrappers in a supported way.

On one hand, this approach violates the "preferably one way" to do things; users will need to decide which way works best for them, creating a variety of supported approaches. On the other hand, I do appreciate that it offers a friendlier interface for certain operations (esp. path(...)).

FFY00 and I put a lot of work into this deprecation process, so it'll be disappointing to now see this reversed at the last minute, but it does feel like the right thing to do, especially since someone else is willing to own the implementation (thanks encukuo!). We will have to backport the change to importlib_resources, but that should be fairly straightforward.

Overall, I'm +0 on the change. I'd really like to see more vocal support from other core devs before committing to this approach.

[encukou]

I've made the encoding argument mandatory for _text functions when multiple path names are given.

Wouldn't this be a bit late for that?

Yes, sorry. Previously I couldn't commit to supporting this API.

Why not just take multiple components with separators in a single argument?

I'd rather not derail discussion on this issue. Support for separators can be added later if necessary. If they will, allowing multiple arguments will still be useful.

FFY00 and I put a lot of work into this deprecation process

Sorry to hear that. Sunk costs suck :(
This makes it seem that implementing the deprecation process was similarly (or more) time-consuming as keeping the API working. That's not a good situation to be in, especially considering all the work users need to put in to update their code.

[pfmoore]

(And add me to the anecdotal list of people who missed them. It's easy enough to add a few lines of code to bring them back, which is how I have been handling it so far, but I'd be happier to have those few lines in the stdlib.)

I’ll add a “me too” here as well. Being able to do simple things simply is an advantage.

[pradyunsg]

+1 from me on the (updated) proposed API as well as un-deprecating these -- for reasons that have been discussed on the d.p.o thread as well as mentioned here by others.

[eli-schwartz]

Their main drawback -- not allowing subdirectories -- can be solved by taking multiple path components as positional arguments, for example:

importlib.resources.read_text('modulename', 'subdirectory', 'subsubdir', 'resource.txt')

I've always sort of wondered why this is a drawback at all, compared to simply doing this:

with importlib.resources.path('modulename.subdirectory.subsubdir', 'resources.txt') as f:
    ...

I'm not objecting to the new API! It's more ergonomic than pretending everything is a namespace module. But for backwards compatibility with python < 3.13 it seems practical to use the two-argument form, and the lack of a new API doesn't seem like it should have been a killer problem before now.

[zooba]

I don't understand the reason we can't reimplement it as:

def read_text(module, filename, *args, **kwargs): #use proper args if you want here, I just don't know them all off the top of my head
    with (path(module) / filename).open("r", *args, **kwargs) as f:
        return f.read()

Why do we need the module and filename as multiple args instead of just two?

[zooba]

I'd rather not derail discussion on this issue.

How is it derailing this issue? You're bringing back an API, which I like, and changing the design in a potentially backwards-incompatible way in the process, which I don't. Why is it derailing to ask why it has to have a different design now?

[barneygale]

Why do we need the module and filename as multiple args instead of just two?

I'm catching myself up on this. I think the answer is (somewhere) in this thread: https://gitlab.com/python-devs/importlib_resources/-/issues/58

[jaraco]

Why do we need the module and filename as multiple args instead of just two?

I'm catching myself up on this. I think the answer is (somewhere) in this thread: https://gitlab.com/python-devs/importlib_resources/-/issues/58

Which was migrated to python/importlib_resources#58.