`types-requests>=2.31.0.7` can't be installed alongside `urllib3<2`

[jessemyers-lettuce]

This PR creates a breaking change because it adds:

urllib3>=2

As of this writing, requests itself has this dependency declaration:

urllib3>=1.21.1,<3

This means that there are scenarios where requests can be installed, but types-requests cannot.

In my case, I am using boto3 and botocore for AWS interactions and it defines its own urllib3 dependency range that produces this incompatiblity:

urllib3>=1.25.4,<1.27

Please consider aligning the types-requests version ranges to the same bounds that requests itself uses.

[AlexWaygood]

If you have a requirements file that also includes a package that pins urllib3>=1.25.4,<1.27, pip (or any other standards-based installer) should be able to determine that the maximum version of types-requests that can be installed in types-requests==2.31.0.6, since the latest version depends on urllib3>=2. If that doesn't work for some reason, would pinning types-requests to <2.31.0.7 be an option for you?

[AlexWaygood]

urllib3<2 will unfortunately not work for our types-requests stubs, as they have to have a version of urllib3 with typing information.

[jessemyers-lettuce]

@AlexWaygood My goal is always to use the most recent version of every dependency that I can without breaking my systems. I use an automated process (renovate) to continuously integrate the latest.

So yes, I am pinning a specific version.

But... it seems to me that you've taken a step to optimize your development process -- I don't know what these stubs do so I can't comment on the value added -- and this comes at the expense of the users of your library.

Put another way: it seems obvious to me that the latest version of types-requests should be 100% compatible with the latest version of requests OR types-requests should have an upper bound version for requests itself (which is probably not something you want).

[AlexWaygood]

But... it seems to me that you've taken a step to optimize your development process -- I don't know what these stubs do so I can't comment on the value added -- and this comes at the expense of the users of your library.

To be clear, without this change, it was painful for our users to use types-requests alongside the latest version of urllib3 (see #10786). This change resolved that problem. It's hard for me to see a way that we could solve that problem and your problem simultaneously here.

[jessemyers-lettuce]

It seems like there are two obvious alternatives:

1. Figure out how to get widely used libraries (e.g. botocore) to use newer versions of requests; this isn't my choice and I am sure that there are a large number of users out there in a similar boat.
2. Write your types to detect the version of urllib3 and fallback to what you had before for older versions.

[AlexWaygood]

2. Write your types to detect the version of urllib3 and fallback to what you had before for older versions.

Stubs are never executed at runtime. In a runtime package you might do something like this to resolve this problem...

try:
    import urllib3
except ImportError:
    urllib3 = None
else:
    if int(urllib3.__version__.split(".")[0]) < 2:
        urllib3 = None

if urllib3 is None:
    import types_urllib3 as urllib3

...but that's out of the question for a stubs package, unfortunately, since a stubs package entirely consists of "data files" that are to be read by a static type checker. All of that stuff is way too dynamic for a type checker to understand; types-requests really has to depend on either urllib3>=2 or types-urllib3 (which is a stubs package for urllib3<2). I don't see a way of falling back to types-urllib3 if urllib3>=2 isn't installed.

[jessemyers-lettuce]

It seems like your implementation choices are getting in the way. I'm going to assume that these choices aren't up for debate at this time.

A few options to consider:

1. Don't make this kind of change without some sort of major version indicator.
2. Offer the old behavior under a different dependency name (e.g. types-requests-old-stubs); this would allow a user to more obviously declare a dependency without an auto-upgrade path introducing breakages.
3. Document the expected usage. I'm not the only person who will experience this issue.

[setu4993]

Just ran into this change and I do agree with the @jessemyers-lettuce above on:

it seems obvious to me that the latest version of types-requests should be 100% compatible with the latest version of requests OR types-requests should have an upper bound version for requests itself (which is probably not something you want).

If that's not the expected behavior, I wouldn't expect such a major change to be introduced as a part of a patch to a patch (v2.31.0.7). This should at the very least, be a minor version update, and ideally a breaking change.

[grutts]

We are also encountering an issue with this change. Semgrep does not support urllib3 > 1.26 in their latest version. The ensuing pip conflict means we are unable to build a number of services which use both semgrep and types-requests. We will pin types-requests to an older version for the time being.

[hauntsaninja]

Thanks everyone for chiming in!

Stub version numbers primarily reflect the version of the upstream package that they represent. This is important as it lets users more clearly type check their code based on the versions of upstream packages they're using.

Therefore, changes to stubs that are not retargeting to a different upstream version end up changing later digits in the version (also basically every change to a stub can change type checker behaviour, so hard to ascribe too much semantic meaning to them)

I agree it would have been nice if we'd had the foresight to bundle this change in with a version bump in upstream requests, but as it stands, there's not too much to be done. Dependency resolvers should be able to handle this and figure out a compatible set of packages just fine. You're welcome to manually pin to older types-requests as well.

[srittau]

I agree with @AlexWaygood and @hauntsaninja here. There is not much that can be done on typeshed's side. The underlying issue is that boto depends on an old version of urllib3. (The underlying underlying issue is the flat namespace of Python imports, which is not something that will change in the short run or ever.) While I understand that it's frustrating to deal with this issue, this is worked around by using proper dependency management (which should prevent newer versions of types-requests to be installed) and/or pinning the types-requests dependency.