Hacker News new | ask | show | jobs
by posix_monad 877 days ago
In Python it's common to have functions whose return types depend on the run-time arguments.

For example:

    def fooify(x):
      if isinstance(x, list):
        map(fooify, x)
      else:
        x * 2
I guess this is to make scripting more forgiving.

In typical typed languages, you would have two functions instead:

    fooify : number -> number

    fooifyMany : [number] -> [number]
But in the Python community, it's common to have a big function with many behaviors. However, then the type annotations cannot be so precise:

    fooify : any -> any
Not an experienced Python dev, so curious how this works in practice.
5 comments

In practice, this would be solved with `typing.overload`[0].

Using you example:

    from typing import overload
 
    @overload
    def fooify(x: int) -> int:
      ...
 
    @overload
    def fooify(x: list[int]) -> list[int]:
      ...
 
 
    def fooify(x: list[int] | int) -> list[int] | int:
      if isinstance(x, list):
        return [fooify(_x) for _x in x]
      return x * 2
 

[0] https://docs.python.org/3/library/typing.html#overload
And for an example of the practical limits of @overload, take a look at the Pandas type hints: https://github.com/pandas-dev/pandas/blob/dc5586baa9e4731805...

Meanwhile it's not even possible to express such things in other static type systems. So I'm not exactly an unhappy customer, but it does put certain things tantalizingly close, but still out of reach without a ton of clunky boilerplate and LoC explosion.

> it's not even possible to express such things in other static type systems

what do you mean? It seems relatively straightforward.

Or a bound typevar:

    T = TypeVar("T", bound=int | list[int])

    def(x: T) -> T:
That's a recursive function, so to make it fully general a recursive type can be used:

  type CompoundInt = int | list[CompoundInt]
  
  def fooify(x: CompoundInt) -> CompoundInt:
      if isinstance(x, int):
          return x * 2
      else:
          return list(map(fooify, x))
  
  print(fooify([1, [3, 4, 5], [6, 7, 8], [[[4, 4, 4]]]]))
This uses the `type` keyword introduced in 3.12. Unfortunately Mypy doesn't support it yet :( so this workaround can be used instead:

  from __future__ import annotations
  
  from typing import TYPE_CHECKING
  
  if TYPE_CHECKING:
      CompoundInt = int | list[CompoundInt]
That could be a Union[float, list[float]]. Union types are very common!

In fact, I think TypeScript will, for your given example, with the 'else' clause accurately identify the type of x to be a float if it's a union like the one I wrote down above.

Sorta? The function does return a union type, in isolation. At most callsites, you would know which of the two you are getting. This is much closer to generic invocation, if I remember the name correctly. Was very common in a lot of older dynamic languages.

In fact, in a lot of languages, you can't tell this is a float, statically. It would work with whatever type was passed in that supports multiplication. And return the appropriate type. Right?

This is function overloading, you can do the same thing in C++. In Python you type this using the overload decorator from the typing module.
You can annotate OR types in python so in this case you could do

def fooify(x: int | list[int])

But if x is an int, the result is an int

If x is a list, the result is a list

I don’t think that the type system can describe this.