Python Development Best Practices — Using TypeVar
Type annotations are perhaps one of the strongest tools of Python to make code easier to understand, as well as promote guidance on how to use specific third-party libraries you may come across. The Python Azure SDK, for instance, is an example of how type annotations can help us understand code faster and also in the process of debugging code, in some cases. Although Python is not a statically typed programming language, those type annotations can guide us to spot places in which we may be using wrong types, thus leading to weird behavior in functions we used throughout our code. This post will be relatively simple and fast to read, and will focus on how we can use TypeVar, a feature of the typing Python library, to make type annotations a little bit more “dynamic”.
Let us begin with a simple example of a function in which we pass one single argument which can be a string or an integer, and return the same argument to the user. The code for this would be:
# example.py
from typing import Union
def identity(arg: Union[int, str]) -> Union[int, str]:
return arg
print(identity(5.18))As usual, by running Python for the code above, we would have no errors, and the result would be 5.18 printed in the screen. However, if we run mypy example.py, we get the following message:
error: Argument 1 to “identity” has incompatible type “float”; expected “int | str” [arg-type]
This is a classic type of error we only see by using specific static code checking tools, exactly like mypy. So, if we wanted to make the code work AND pass the mypy inspection, we’d need to change it to:
# example.py
from typing import Union
def identity(arg: Union[int, float, str]) -> Union[int, float, str]:
return arg
print(identity(5.18))The code above would both run successfully and pass the mypy checking procedure. However, note the following: since the function is basically just returning the given input argument, any time that we change our allowed types for the arg input, we need to manually change the type annotation of the function’s output, as we did in the code above. How can we sort of dynamically define the type annotations of this function, so that no matter the type of the input, we can declare that the type of the output should match the type of the input?
The answer to this question is by using TypeVar. This class, which is provided by the typing Python library, allows us to define new type variables. Check out the official docs of typing and the PEP 484 official page for more info on type hints for Python.
Note: TypeVar and NewType are different concepts. While the first is generally used to refer to the same type more than once without explicitly telling the type, the second is actually used to create new types, sometimes by inheriting the methods of existing types (see this discussion for a more thorough differentiation betweent the two concepts, and this post for a more detailed explanation of some use cases of TypeVar).
Let us go back to our code. Now we want to explicitly say, by using type annotations, that no matter what the input type is, the output type is the same as the input. The new code is given below.
# example.py
from typing import TypeVar
TypeForIdentity = TypeVar("TypeForIdentity")
def identity(arg: TypeForIdentity) -> TypeForIdentity:
return arg
print(identity(5.18))The code above passes successfully the mypy static type checker, which you can verify by running mypy example.py. No matter the input type, the above code basically says that the output type matches the input type. To check how exactly this “passing of types from input to output” works, try to change the return statement line, and put return [arg] in place of return arg. You will get an error at mypy. That’s because the type of the output is a List of arguments of type TypeForIdentity, while the type annotation says that the allowed type is only TypeForIdentity.
Finally, let us suppose that we want to both constrain the output’s type to the input’s type, and declare explicitly the allowed types of the I/O of the function (which would be the case if we wanted to allow integers to be passed, but not floats, for instance). Then, the function body would be:
from typing import TypeVar
TypeForIdentity = TypeVar("TypeForIdentity", int, str)
def identity(arg: TypeForIdentity) -> TypeForIdentity:
return arg
print(identity(5.18))
print(identity(5))
print(identity('5'))By analysing the above code with mypy, we would get:
error: Value of type variable “TypeForIdentity” of “identity” cannot be “float”
The error above is a consequence of the fact that we declared the new TypeForIdentity type and only allowed int or str types. Thus, we have seen that we can use TypeVar to both 1) constrain the new type and 2) recycle the type in multiple places, like the case in which the output type should match the input type “dynamically”. If we wanted to fix the above code to pass the mypy check, we would just need to add float to the TypeVar definition in line 3, and all the three function calls types would be obeyed, thus passing the mypy type checking test.
References:
