Python: Cool features in python 3.11

Its time for Python 3.11

python 3.11

Hi everyone, In this article I will brief about what new updates/changes are there in python 3.11 and I think you going to love them as they will make your life easier. lets dive in -

1. Exception Notes

Now you can also add your notes to the Exception. You can attach extra info to your existing exception and re raise it.

This help you to deal with it differently and it also helpful in testing packages as well(like hypothesis).

Lets see an example —

def main() -> None:
    try:
        raise TypeError("The Type Error")
    except TypeError as error:
        error.add_note("Other Information")
        raise


if __name__ == '__main__':
    main()

2. Exception Group

With the use of base Exception and ExceptionGroup, You can easily aggregate/combine exceptions together.

The traceback of ExceptionGroup contains a tree of information about all individual exceptions.

Lets see an example —

async def func1():
    raise ValueError

async def func2():
    raise TypeError

async def main_func():
    results = await asyncio.gather(
        func1(),
        func2(),
        return_exceptions=True,
    )
    exceptions = [exc for exc in results if isinstance(exc, Exception)]
    if exceptions:
        raise ExceptionGroup("Couldn't execute", exceptions)

Below is how the traceback looks -

ExceptionGroup: Couldn't execute (2 sub-exceptions)
+-+----------------------1------------------------------
| Traceback (most recent call last):
|   File "/hone/pravash/Projects/python311-release/exception-groups-py", line 5, in func1
|        raise ValueError
| ValueError
+------------------------2------------------------------
| Traceback (most recent call last):
|   File "/home/pravash/Projects/python311-release/exception-groups-py", line 9, in func2
|   raise TypeError
| TypeError

3. Asyncio TaskGroup

As from the above example in Exception Group, you can also see asyncio.gather() provide a cleaner and nicer syntax for asyncio tasks.
But, there is a more modern way to run tasks concurrently and wait for their completion — asyncio.TaskGroup.

Instead of using gather(), you use a context manager to define when tasks will be awaited:

async with asyncio.TaskGroup() as tg:
    for param in params:
        tg.create_task(run_some_task(param))

4. Precise Traceback

Apart from printing the message, Now you can see in which line of code the error occurs. Traceback now gives the precise location of the error where it occurs.

This is really helpful when you are dealing with more complex objects or multiple functions.

Here’s an Example -

Traceback (most recent call last):
    File distance.py", line 11, in <module>
        print manhattan distance pl, p2))
    File "distance.py", line 6, in manhattan_distance
        return abs (point_l.x - point_2.x) + abs (point_l.y - point_2.y)
AttributeError: NoneType object has no attribute 'X'

5. Tomllib library

This is now the part of python’s standrad library. Its used for defining configurations files. tomllib configuration is more readable than json or yaml.

Here’s an example -

import tomllib

def main () -> None:
    with open("settings.toml", "rb") as f:
        data = tomllib. load (f)
    print(data)

if __name__ == '__main__':
    main()

It will print the result in dictionary (Used the Tomllib config file from documentation) —

{'title': 'TOML Example', 'owner': {'name'; 'Tom Preston-Werner', 'dob': datetime.datetime(1979, 5, 27, 7 ,32, tzinfo=datetime. timezone(datetime.timedelta(days=-1, seconds=57600)))}, 
'database': {'server': '192.168.1.1', 'ports': [8000, 8001, 80021], 'connection_max': 5000, 'enabled': True'}, 
'servers': {'alpha': {'ip': '10.0.0.1', 'dc': 'eqdc10}, 'beta': {'ip': '10.0.0.2', 'dc': 'eqdc10'}},
'clients': {'data': [['gama', 'delta'], [1,2]], 'hosts': ['alpha', 'omega']}}

6. LiteralString type

This accepts any literal string value. Its really helpful when using sql or shell commands.

When you are constructing the SQL query using f-string, it might allows to cause the injection attacks. So SQL apis offers parameterized queries to avoid this issue.

Since with this new python version, we can use the LiteralString types to make Python more secure. For a secure API, it’s best to accept only literal strings or string constructed from literal values.

This allows IDE to remind you that the query should only involves literal text. For Example -

from typing import LiteralString

def execute_query(conn, query: LiteralString, *params):
    ...

def your_code(conn, arbitrary_string:str, string_val: LiteralString):
    execute_query(conn, f"select * from table where id =?", arbitrary_string)
    execute_query(conn, arbitrary_string)   # type check error
    execute_query(conn, f"SELECT * FROM students WHERE name = {arbitrary_string}")   # type check error

7. Self Type

Now you have Self type identifier. This way you can annotate methods that return an instance of their class. So it is much more readable.

For example —

class MyLock:
    def __enter__(self) -> Self:
        self.lock()
        return self

    ...

class MyInt:
    @classmethod
    def fromhex(cls, s: str) -> Self:
        return cls(int(s, 16))

This is useful, in case where you need to change the class name, then you don’t have to modify the methods.

8. dataclass transform

This is used to decorate a class, metaclass, or a function that is itself a decorator.

The @dataclass_transform() decorator (a custom or specialized decorator), could perform similar transformations on a class to provide dataclass-like behaviors. This means that when using a static type checker (such as mypy), it will understand that the class being decorated has these additional behaviours and can perform type checking accordingly.

For Example -

# The create_model decorator is defined by a library.
@typing.dataclass_transform()
def create_model(cls: Type[T]) -> Type[T]:
    cls.__init__ = ...
    cls.__eq__ = ...
    cls.__ne__ = ...
    return cls

# The create_model decorator can now be used to create new model classes:
@create_model
class CustomerModel:
    id: int
    name: str

c = CustomerModel(id=327, name="Eric Idle")

9. Improved Performance

Python 3.11 is between 10–40% faster than Python 3.10. This is possible because of the faster — cpython project. The cpython interpreter itself was optimized greatly for both startup and runtime. Bytecode objects are now statically allocated and stored in the __pycache__.

Few builtins like lists and dicts were optimized.

In the new version, whenever the interpreted detects a function call from another python function, it sets a new frame for that function and executes it without calling the C function, which effectively inlines the function call. This results in a significantly bigger recursion limit as well as faster-running recursive functions.

The biggest wins with Python 3.11 are the improvements to the developer experience: the better error messages and the faster code execution.

Whether you should upgrade to 3.11 or not, the answer is: “it depends”, And also sooner or later, you will not be able to access security updates and bug fixes. Therefore, you will eventually need to upgrade. Migration can be a hassle if you’re using older versions of Python because you’ll frequently have to deal with more major cross-version changes in both Python and libraries at the same time.

And if you are really excited to switch right now then you should wait, at a minimum, after a major new release you will need to wait until: 1) All your libraries explicitly support the new Python release. 2) All the tools you rely on explicitly support the new Python release.

Being said that, It’s worth giving it a try.