Capture warnings

pytask captures warnings during the execution.

Here is an example with the most infamous warning in the world of scientific Python.

from pathlib import Path
from typing import Annotated

import pandas as pd

from pytask import Product


def _create_df() -> pd.DataFrame:
    df = pd.DataFrame({"a": range(10), "b": range(10, 20)})
    df[df["a"] < 5]["b"] = 1
    return df


def task_warning(path: Annotated[Path, Product] = Path("df.pkl")) -> None:
    df = _create_df()
    df.to_pickle(path)

Running pytask produces

$ pytask
────────────────────────── Start pytask session ─────────────────────────
Platform: win32 -- Python 3.12.0, pytask 0.5.3, pluggy 1.3.0
Root: C:\Users\pytask-dev\git\my_project
Collected 1 task.

┏━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━┳━━━━━━━━━┓
┃ Task                                              ┃ Outcome ┃
┡━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━╇━━━━━━━━━┩
│ <span class="termynal-dim">task_warning.py::</span>task_warning                     │ <span class="termynal-success">.</span>       │
└───────────────────────────────────────────────────┴─────────┘

<span class="termynal-dim">─────────────────────────────────────────────────────────────────────────</span>
<span class="termynal-success">╭───────────</span> <span style="font-weight: bold;">Summary</span> <span class="termynal-success">────────────╮</span>
<span class="termynal-success">│</span> <span style="font-weight: bold;"> 1  Collected tasks </span>           <span class="termynal-success">│</span>
<span class="termynal-success">│</span> <span class="termynal-success-textonly"> 1  Succeeded        (100.0%) </span> <span class="termynal-success">│</span>
<span class="termynal-success">╰────────────────────────────────╯</span>
<span class="termynal-success">─────────────────────── Succeeded in 0.24 seconds ──────────────────────</span>
<span class="termynal-skipped">╭─────────────────────────────── Warnings ──────────────────────────────╮</span>
<span class="termynal-skipped">│ task_warning.py::task_warning                                         │</span>
<span class="termynal-skipped">│     ...\my_project\task_warning.py:8: SettingWithCopyWarning:         │</span>
<span class="termynal-skipped">│     A value is trying to be set on a copy of a slice from a DataFrame.│</span>
<span class="termynal-skipped">│     Try using .loc = value instead                                    │</span>
<span class="termynal-skipped">│                                                                       │</span>
<span class="termynal-skipped">│     See the caveats in the documentation:                             │</span>
<span class="termynal-skipped">│     https://pandas.pydata.org/pandas-docs/stable/user_guide/          │</span>
<span class="termynal-skipped">│     indexing.html#returning-a-view-versus-a-copy                      │</span>
<span class="termynal-skipped">│       df[df[&quot;a&quot;] &lt; 5][&quot;b&quot;] = 1                                        │</span>
<span class="termynal-skipped">│                                                                       │</span>
<span class="termynal-skipped">│ </span><font color="#cd3131">♥</font><span class="termynal-skipped"> https://pytask-dev.rtfd.io/en/stable/how_to_guides/capture_warnings.html │</span>
<span class="termynal-skipped">╰───────────────────────────────────────────────────────────────────────╯</span>

Controlling warnings

You can use the filterwarnings option in pyproject.toml to configure pytask’s behavior when it comes to warnings.

The syntax for specifying warnings filters is the same as in the Python standard library, i.e., a sequence of fields separated by colons:

action:message:category:module:line

You can specify a list of such filters. When a warning matches more than one option, the action for the last matching option is performed.

For example, the configuration below will ignore specific deprecation warnings matching a regex, all user warnings, and transform all other warnings into errors.

[tool.pytask.ini_options]
filterwarnings = [
    "error",
    "ignore::UserWarning",
    # note the use of single quote below to denote "raw" strings in TOML
    'ignore:function ham\(\) is deprecated:DeprecationWarning',
]

The syntax is explained in more detail in this section of the Python documentation and there are also more examples.

@pytask.mark.filterwarnings

You can use the @pytask.mark.filterwarnings to add warning filters to specific test items, allowing you to have finer control of which warnings should be captured at the test, class or even module level:

from pathlib import Path
from typing import Annotated

import pandas as pd

import pytask
from pytask import Product


def _create_df() -> pd.DataFrame:
    df = pd.DataFrame({"a": range(10), "b": range(10, 20)})
    df[df["a"] < 5]["b"] = 1
    return df


@pytask.mark.filterwarnings("ignore:.*:pandas.errors.SettingWithCopyWarning")
def task_warning(path: Annotated[Path, Product] = Path("df.pkl")) -> None:
    df = _create_df()
    df.to_pickle(path)

Filters applied using a mark take precedence over filters passed on the command line or configured by the filterwarnings configuration option.

Important

Note that the type of warning needs to be importable. For example, UserWarning is a built-in warning with no module specified. SettingWithCopyWarning though needs to be imported from pandas.errors.

Disabling warnings summary

Although not recommended, you can use the --disable-warnings command-line option to suppress the warning summary entirely from the test run output.

Debugging warnings

Sometimes it is not clear which line of code triggered a warning. To find the location, you can turn warnings into exceptions and then use the pytask build --pdb flag to enter the debugger.

You can use the configuration to convert warnings to errors by setting

[tool.pytask.ini_options]
filterwarnings = ["error:.*"]

and then run pytask.

Or, you use a temporary environment variable. Here is an example for bash.

$ PYTHONWARNINGS=error pytask --pdb