Clarify the valid use locations of `typing.Concatenate`

[johnslavik]

In python/cpython#142965, it was reported that the documentation of typing.Concatenate is "incorrect" (documentation that originates back to python/cpython#24000).

@A5rocks, in your example, Concatenate[int, P_2] somewhat finally lands as a first argument to Callable, just indirectly.
This snippet previously defined:

P_2 = ParamSpec("P_2")

class X(Generic[T, P]):
  f: Callable[P, int]
  x: T

Similar example is included in the typing spec:

https://typing.python.org/en/latest/spec/generics.html#user-defined-generic-classes

class X[T, **P]:
    f: Callable[P, int]
    x: T

# (...)
def accept_concatenate[**P](x: X[int, Concatenate[int, P]]) -> str: ...  # Accepted

However, it seems that the current situation (confirmed with mypy, pyright and ty) is that you can use Concatenate in all valid locations of ParamSpec except directly in a Concatenate. I.e., you can't do Concatenate[int, Concatenate[str, P]]), but you can (besides passing Concatenate form as the first argument to Callable):

1. Accumulate Concatenates as ParamSpecs

from collections.abc import Callable
from typing import Concatenate

type Y[**P] = Callable[Concatenate[int, P], None]
type X[**P] = Y[Concatenate[int, P]]

def foo(f: X[str]) -> None:
    reveal_type(f)
    # mypy: def (builtins.int, builtins.int, builtins.str)
    # pyright: (int, int, str) -> None
    # ty: (...) -> None

2. Bind Concatenates to user-defined generics as ParamSpecs

class X[T, **P]:
    f: Callable[P, int]
    x: T

def accept_concatenate[**P](x: X[int, Concatenate[int, P]]) -> str: ...

3. Use Concatenate as a type argument to tuple -- is this correct?

from typing import Concatenate

def c(t: tuple[Concatenate[int, ...]]) -> None:
    reveal_type(c)
    # mypy: def (t: tuple[[builtins.int, *Any, **Any]])
    # pyright: (t: tuple[Concatenate[int, ...]]) -> None
    # ty: def c(t: tuple[@Todo]) -> None

I've found this mostly by poking around -- I haven't analyzed the actual implementations (yet).
I'll continue to investigate this from these searches.

I think that the valid use locations of Concatenate should be clarified in the typing spec and then in the CPython docs.

Is there anything else I overlooked? CC @JelleZijlstra @AlexWaygood

[A5rocks]

is that you can use Concatenate in all valid locations of ParamSpec except directly in a Concatenate

To be clear, this is what the PEP specifies. It notes that:

parameters_expression ::=
  | "..."
  | "[" [ type_expression ("," type_expression)* ] "]"
  | parameter_specification_variable
  | concatenate "["
                   type_expression ("," type_expression)* ","
                   parameter_specification_variable
                "]"

and:

defining a class as inheriting from Generic[P] makes a class generic on parameters_expressions (when P is a ParamSpec).

[A5rocks]

and mypy accepting Concatenate in a tuple is a bug. I've filed a report: python/mypy#20452

The only other point I see is that Concatenate[int, ...] is allowed despite not being in the PEP, but this is something that changed in the typing docs:

• Add support for using ellipsis as last argument to Concatenate microsoft/pyright#5715
• Accept Ellipsis as last argument for Concatenate. mypy#14656