test

Adding Tests to a Multi-File Project

Standard Project Layout with Tests

To maintain a clean and organized codebase, it is standard practice to separate your application code from your test code. This is typically achieved by creating a dedicated tests directory at the project's root level.

• The project root contains the main application package (e.g., devops_utils) and the tests directory as siblings.
• The tests directory houses all the test files. It is common for the structure inside tests to mirror the structure of the application package to keep tests organized as the project grows.
• It is also good practice to place an empty __init__.py file inside the tests directory to ensure it can be treated as a package if needed, although pytest's discovery mechanism is powerful enough to work without it in most cases.

Importing Application Code into Tests

To test your application code, your test files must import the functions, classes, and variables that need to be verified. The standard and most robust way to do this is by using absolute imports that start from the project root.

• A test file, such as tests/test_file_ops.py, will use an import statement like from devops_utils.file_ops import check_file_extension.
• This import syntax assumes that the project's root directory is on Python's module search path (sys.path).
• Attempting to run a test file directly as a Python script will fail with a ModuleNotFoundError, because the project root is not automatically added to the path in that context. This is why a test runner like pytest is necessary.

Running pytest for Project Discovery

The pytest framework is designed to handle the complexities of testing structured projects. When you run pytest from your project's root directory, it intelligently prepares the environment for test execution.

• Pytest begins by scanning upwards from the current directory to find a configuration file (like pyproject.toml or pytest.ini), which it uses to identify the project's root.
• Once the root is identified, pytest automatically adds this directory to sys.path.
• With the path correctly configured, the absolute imports within your test files will resolve successfully, allowing your tests to find and execute the application code.

Robust Test Execution with python -m pytest

While running pytest directly is often sufficient, an even more reliable method is to invoke it as a module using the python -m flag. This approach is highly recommended, especially in automated environments like CI/CD pipelines.

• The command python -m pytest guarantees that the current working directory is added to sys.path before pytest begins its execution.
• This method eliminates many path-related ambiguities that can arise in complex project structures or different operating environments.
• It is considered the most robust and explicit way to run your test suite, ensuring consistent behavior across different machines and setups.

Common Pitfalls & How to Avoid Them

When setting up tests for a multi-file project, there are several common issues that can be easily avoided.

• ModuleNotFoundError during test runs: This is the most frequent problem and is almost always caused by running pytest from the wrong directory. To avoid this, always run pytest from the project root, or preferably use python -m pytest. An editable install (pip install -e .) provides the most robust solution.
• Tests importing other tests: This is considered an anti-pattern as it creates implicit dependencies between tests. If you need to share test logic or data, use conftest.py to define shared fixtures.
• Using relative imports in tests: Imports like from ..devops_utils import ... are fragile and should be avoided in test files. Always use absolute imports from the project root (e.g., from devops_utils.file_utils import ...).

Editable Installs with pyproject.toml

The Python interpreter doesn't automatically know about our project's structure. The modern and most robust solution is to formally define our project as an installable package. By creating a standard pyproject.toml metadata file, we can perform an "editable install" using the command pip install -e .. This seamlessly links our source code into the virtual environment, making our packages importable from anywhere without manual path hacks or special commands. This is the standard, professional workflow for developing Python applications.

The pyproject.toml File

The pyproject.toml file is the modern, unified standard for configuring Python projects, replacing older files like setup.py.

• It is written in the simple and readable TOML (Tom's Obvious, Minimal Language) format.
• It defines the project's build system, such as setuptools, in the [build-system] table.
• It specifies essential project metadata, like the package name, version, and dependencies, in the [project] table.
• It can also serve as a central location for configuring development tools like formatters and linters.

Editable Installs: pip install -e .

An "editable" or "development" install is a special mode for installing packages with pip.

• The command pip install -e . installs the project from the current directory in "editable" mode.
• Instead of copying files, it creates a special link from the virtual environment's site-packages directory back to the original source code.
• The main benefit is that any changes made to the source .py files are immediately reflected in the installed package without needing to run pip install again.

Solving the Import Problem Permanently

Performing an editable install provides a definitive solution to the path and import issues encountered during development.

• After an editable install, the project is effectively on sys.path for the entire activated virtual environment.
• There is no longer any need to manually set the PYTHONPATH environment variable.
• You can now run scripts that are inside packages directly, and they will be able to use absolute imports from the project root.
• This approach creates a self-contained and reproducible environment, which is the standard for professional Python development.

Console Scripts vs. python -m

While a script with an if __name__ == "__main__" block can be run with python -m, defining a console script in pyproject.toml is the preferred professional approach.

• User Experience: A console script (ping-check) is short and intuitive. A python -m command is long, exposes the internal module path, and is easy to mistype.
• Abstraction: A console script hides your project's internal structure. You can refactor your code internally, and as long as you update the pyproject.toml file, the command remains the same for the user. The python -m command breaks if you rename or move the target file.
• Clarity of Intent: Declaring a script in pyproject.toml clearly marks it as a public, supported command-line interface for your package.

[project]
name = "devops_utils"
version = "0.1.0"

[project.scripts]
check_host = "devops_utils.network_utils.check_host:main"

Running Scripts: python -m vs. python file.py

The Core Difference: What is "Entry Point Zero"?

The key to understanding the difference lies in the first entry of sys.path. When Python initializes, it needs to know where to start looking for modules. The way you call it determines this "entry point zero".

• When you run a script directly using python path/to/script.py, the interpreter's main task is to execute that specific file. It sets the first entry of sys.path to be the directory that contains the script.

• When you run a script as a module using python -m package.module, the interpreter's goal is to locate and run a module within an importable package. It sets the first entry of sys.path to be the current working directory from which the command was executed. This allows absolute imports from the project root to succeed.

Best Practice: Separating Library Code from Scripts

While you can run any file with python -m, it can lead to a RuntimeWarning if the file is both a library (meant to be imported) and a script (meant to be run). The best practice is to separate these roles.

• Library Modules: These files (like our file_ops.py and network_ops.py) should contain only reusable functions and classes. They should not contain an if __name__ == "__main__": block for complex script logic.
• Runner Scripts: For any action you want to make runnable from the command line, create a new, separate script.

Common Pitfalls & How to Avoid Them

• Running scripts inside packages directly with python file.py will often cause ModuleNotFoundError for absolute imports. Avoid this by always running packaged scripts from the project root using python -m.
• Making a single file both a complex library and a runnable script can lead to RuntimeWarning. Avoid this by separating concerns: create dedicated runner scripts that import from your library modules.
• Forgetting the module path when using -m. The command must be the full dotted path to the script from the project root (e.g., python -m package.subpackage.script).

Introduction to Packages (__init__.py)

What is a Package?

A Python package provides a way to structure a project's module namespace by using directories. It enables a hierarchical organization of modules, mirroring the file system's structure.

• Any directory that contains a file named __init__.py is recognized by the Python interpreter as a package.
• A package directory can contain not only module files (ending in .py) but also other subdirectories that are themselves packages.

The Role of __init__.py

• Its primary role is to serve as a marker, signaling to Python that the directory it resides in should be treated as a package. In older versions of Python, a directory without this file would not be recognized as a package. While newer versions have more lenient rules, explicitly including __init__.py is the standard and most compatible method.
• The second purpose is for package initialization. Any code written inside the __init__.py file is executed once when the package or any of its modules are first imported, which can be useful for setting up package-level resources. For many packages, this file can simply be left empty.

Importing from a Package

• You can import the entire module using the full path, such as import devops_utils.file_ops. Accessing its contents would then require the full prefix, like devops_utils.file_ops.check_file_extension().
• Alternatively, you can use the from keyword to import the module more directly, as in from devops_utils import file_ops, which then allows access via file_ops.check_file_extension().
• For even more direct access, you can import specific functions or variables, for example, from devops_utils.file_ops import check_file_extension. This makes the function available to be called directly as check_file_extension().

Using __init__.py to Control Imports

• By importing members from the package's modules into the __init__.py file itself, you can make them appear as if they belong to the top-level package namespace.
• This technique can create a simpler, more user-friendly API for your package, but it can also obscure the underlying module structure.

Importing from Subpackages

Project Structure with Subpackages

To improve organization, we can group related modules into their own dedicated subpackages. A subpackage is a directory inside a parent package that contains its own __init__.py file.

• A top-level package, like devops_utils, can contain multiple subpackages.
• For example, we can create a file_utils subpackage for file-related modules and a network_utils subpackage for networking modules.
• Each of these subdirectories must contain an __init__.py file (which can be empty) to be recognized by Python as a package.

Absolute Imports

An absolute import provides the complete, explicit path to a module starting from a top-level directory that is on Python's search path (sys.path).

• The syntax follows the project's directory structure, such as from package.subpackage.module import function.
• This is the most recommended and readable way to import modules, particularly in top-level scripts that execute the application's logic.
• Absolute imports are unambiguous and clearly state the origin of the imported code, which greatly improves code maintainability. For example, a script outside the devops_utils package would use from devops_utils.file_utils.file_ops import check_file_extension to access a function.

Relative Imports

A relative import specifies the path to a module based on the location of the file performing the import. This is done using dot notation.

• The syntax uses dots to navigate the package hierarchy: . refers to the current package, while .. refers to the parent package.
• An import like from .sibling_module import name brings in a name from a module in the same directory. An import like from ..parent_sibling.module import name navigates one level up and then down into a sibling package.
• Relative imports are primarily used for communication within a package. Their main advantage is that they make the package self-contained; if you rename the top-level package, the internal relative imports will not break.

The ImportError Trap with Relative Imports

A significant pitfall arises when you attempt to directly execute a Python file that contains relative imports. This action will almost always result in an error.

• Running a script like python devops_utils/network_utils/network_ops.py will raise an ImportError: attempted relative import with no known parent package.
• This happens because when a file is run directly, Python sets its name (__name__) to "__main__" and does not recognize it as being part of a package. Consequently, it cannot resolve relative paths like . or ...
• The rule of thumb is that relative imports should only be used for intra-package imports, and the application should be started from a top-level script that uses absolute imports to access the package's functionality.

Python Modules and the import System

What is a Module?

• A module in Python corresponds directly to a single file containing Python code.
• The module's name is derived from its filename.
• Any file with a .py extension can be treated as a module.
• The name used to import the module is the filename without the .py suffix. For example, a file named file_ops.py is imported as the file_ops module.

The import Statement

• The most straightforward way to import a module is with import module_name
• To use functions, variables, or classes from the imported module, you must prefix them with the module name and a dot, such as module_name.function_name.
• This method creates a dedicated namespace for the imported module, which is highly effective at preventing name collisions. For instance, a variable named CONFIG in your script will not conflict with module_name.CONFIG.
• It enhances code clarity by making it obvious where each function or variable originates, which is especially helpful in larger projects.

The from...import Statement

• It's possible to bring only specific objects from a module with from ... import ....
• You can use the as keyword to rename an imported object, for example, from file_ops import parse_yaml_file as parse_yaml.
• This approach can make code more concise because it requires less typing (parse_yaml() instead of file_ops.parse_yaml_file()).
• The primary drawback is the increased risk of name collisions. If you import a function named my_function and later define your own function with the same name, the original import will be overwritten.
• Using from module import * is strongly discouraged because it imports all public names from the module, which can pollute the local namespace and make the code difficult to read and debug.

How Python Finds Modules: sys.path

When you execute an import statement, Python needs to locate the corresponding module file. It does this by searching through a specific list of directories.

• The search path is stored in a list of strings called sys.path, which is part of the standard sys module.
• The sys.path list is automatically populated and typically includes the directory of the script that is currently running, directories specified in the PYTHONPATH environment variable, and the default locations where Python and third-party packages are installed.
• The fact that the script's own directory is the first entry on this path is why a script like main.py can seamlessly import utils.py when both files are located in the same folder.

Example of main.py

print("Main script starting...")

from devops_utils import (
    check_file_extension,
    is_host_up,
    check_hosts_from_config,
)
import sys

print(sys.path)

filenames = ["config.yaml", "script.sh"]

for filename in filenames:
    print(f"Checking {filename}")
    print(f"Result: {check_file_extension(filename)}")

print(f"\nIs localhost up? {is_host_up("localhost")}")
print(
    f"Is nonexistenthost12345 up? {is_host_up("nonexistenthost12345")}"
)

print(
    f"\nAre all hosts from servers_config.yaml up? {check_hosts_from_config("servers_config.yaml")}"
)

Example of file_ops.py

print("Module file_ops is being imported")

from typing import Any

try:
    import yaml
except (ModuleNotFoundError, ImportError):
    print(
        "Warning: PyYAML not found, parse_yaml_file will not work."
    )
    yaml = None

SUPPORTED_EXTENSIONS: list[str] = [".json", ".yaml", ".txt"]

def check_file_extension(filename: str) -> bool:
    """Checks if a file has a supported extension"""
    print(
        f"  - file_ops.check_file_extension called for {filename}"
    )
    return any(
        filename.endswith(ext) for ext in SUPPORTED_EXTENSIONS
    )

def parse_yaml_file(path_str: str) -> dict[str, Any]:
    """Parses a YAML file and returns its contents."""
    print(f"  - file_ops.parse_yaml_file called for {path_str}")
    if yaml:
        with open(path_str, "r") as file:
            return yaml.safe_load(file)
    else:
        return {}

Mocking Fundamentals

Introduction

• When unit testing DevOps scripts that interact with external systems, tests can become slow, unreliable, difficult to set up, or even destructive.
• Mocking replaces these real dependencies with controlled, fake objects so that tests run quickly and deterministically.
• Python’s built-in unittest.mock module provides tools to create and configure these mock objects and to track interactions.

What is Mocking?

• Mocking involves creating objects that mimic the behavior of real functions or classes in a controlled environment.
• When your code calls a mocked object, you can specify what it returns, simulate exceptions, or inspect how it was called.
• This allows you to isolate the logic under test and avoid side effects from actual external calls.

Using unittest.mock.patch

• The patch function replaces a target object with a mock in a specified scope, either for the duration of a function (decorator) or within a context block (with).
• As a decorator, patch injects the mock into the test function’s parameters; as a context manager, it yields the mock within the with block.
• It’s important to patch the object where it is looked up in the module under test, not necessarily where it is originally defined.

MagicMock and Configuring Mock Objects

• When you patch an object, you typically receive a MagicMock instance that you can configure.
• Use mock.return_value to define what the mock will return when called.
• Use mock.side_effect to simulate an exception being raised by the mock when invoked, to pass different values to be returned by each execution, or to pass a calable to replace the implemented function.
• Assertion methods like assert_called_with and assert_called_once let you verify interactions with the mock.

Common Mocking Scenarios in DevOps

• Network API Calls: Mock requests.get or requests.post to simulate successful responses, HTTP errors, or timeouts.
• Filesystem Operations: Mock functions like open() or os.path.exists() to simulate file presence or content.
• Subprocess Execution: Mock subprocess.run to avoid running real system commands and control return codes.
• Time-Dependent Code: Patch time.sleep or mock datetime.now() to remove delays and make time-based tests deterministic.

from unittest.mock import patch, Mock
from pytest_mock import MockerFixture
from dummy_functions import check_file_exists, get_user_data

# Section: Using unittest.mock.patch
def test_check_file_exists_manual_patch() -> None:
    filepath = "/path/to/some/file.txt"

    patcher = patch("dummy_functions.os.path.exists")
    mock_exists = patcher.start()

    mock_exists.return_value = True

    try:
        result = check_file_exists(filepath=filepath)
        mock_exists.assert_called_once_with(filepath)
        assert result is True
    finally:
        patcher.stop()

def test_check_file_exists_context_manager() -> None:
    filepath = "/path/to/some/file.txt"

    with patch("dummy_functions.os.path.exists") as mock_exists:
        mock_exists.return_value = True

        result = check_file_exists(filepath=filepath)
        mock_exists.assert_called_once_with(filepath)
        assert result is True

@patch("dummy_functions.os.path.exists")
def test_check_file_exists_decorator(mock_exists: Mock) -> None:
    filepath = "/path/to/some/file.txt"

    mock_exists.return_value = True

    result = check_file_exists(filepath=filepath)
    mock_exists.assert_called_once_with(filepath)
    assert result is True

def test_check_file_pytest_mocker(mocker: MockerFixture) -> None:
    filepath = "/path/to/some/file.txt"

    mock_exists = mocker.patch("dummy_functions.os.path.exists")
    mock_exists.return_value = True

    result = check_file_exists(filepath=filepath)
    mock_exists.assert_called_once_with(filepath)
    assert result is True

# Section: MagicMock and Configuring Mock Objects
def test_get_user_data_success(mocker: MockerFixture) -> None:
    mock_api_response: dict[str, str | int] = {
        "id": 1,
        "name": "test user",
    }

    mock_get = mocker.patch("dummy_functions.requests.get")
    mock_get.return_value.status_code = 200
    mock_get.return_value.json.return_value = mock_api_response

    data = get_user_data(user_id=1)

    mock_get.assert_called_once_with(
        "https://api.example.com/users/1"
    )
    assert data == mock_api_response

Example of dummy_function.py

import requests
import os
import subprocess
from typing import Optional, Any

def get_user_data(user_id: str | int) -> dict[str, str | int]:
    response = requests.get(
        f"https://api.example.com/users/{user_id}"
    )
    print(f"Status code: {response.status_code}")
    response.raise_for_status()
    return response.json()

def check_file_exists(filepath: str | os.PathLike[str]) -> bool:
    return os.path.exists(filepath)

def get_external_ip():
    """Fetches the current external IP from an external service."""
    try:
        response = requests.get(
            "https://api.ipify.org?format=json", timeout=5
        )
        response.raise_for_status()
        return response.json().get("ip")
    except requests.exceptions.RequestException:
        return None

def get_current_user() -> Optional[str]:
    try:
        result = subprocess.run(
            ["whoami"],
            capture_output=True,
            text=True,
            check=True,
            timeout=5,
        )
        return result.stdout.strip()
    except (
        subprocess.CalledProcessError,
        subprocess.TimeoutExpired,
        FileNotFoundError,
    ):
        return None

def fetch_both_endpoints() -> (
    tuple[dict[str, Any], dict[str, Any]]
):
    """
    Fetch data from two endpoints and return their JSON responses as a tuple.
    """
    response2 = requests.get("https://api.example.com/second")
    response2.raise_for_status()
    data2 = response2.json()

    response1 = requests.get("https://api.example.com/first")
    response1.raise_for_status()
    data1 = response1.json()

    return data1, data2

Advanced Mocking Concepts

Using side_effect

• The side_effect attribute on a mock allows you to control its behavior beyond a single return value.
• List of values: When side_effect is set to a list, each call to the mock returns the next item in that list, in order.
• Callable: When side_effect is a function, it is called with the same arguments as the mock, and its return value is used as the mock’s return.
• Exception: When side_effect is an exception, it will raise that exception when the original function is called.
• Use a list when you know the sequence and order of calls; use a function when behavior should vary based on arguments.

Choosing between Mock and MagicMock

• Mock: A simple replacement that only creates attributes when accessed, and raises errors for undefined methods or attributes.
• MagicMock: Inherits from Mock and implements Python’s magic methods (__len__, __enter__, etc.) by default.
• Use Mock by default for stubbing external dependencies to catch unintended interactions.
• Use MagicMock only when mocking objects that require special behavior, such as context managers or iterables.

import subprocess
import pytest
from unittest.mock import MagicMock
from pytest_mock import MockerFixture
from dummy_functions import (
    get_current_user,
    check_file_exists,
    fetch_both_endpoints,
)

# Section: Using side_effect - Exceptions

def test_get_current_user_command_fails(mocker: MockerFixture):
    mock_run = mocker.patch("dummy_functions.subprocess.run")
    mock_run.side_effect = subprocess.CalledProcessError(
        returncode=1, cmd=["whoami"]
    )

    result = get_current_user()

    assert result is None

# Section: Using side_effect - List for Multiple Calls

def test_check_file_exists_side_effect_list(
    mocker: MockerFixture,
):
    mock_exists = mocker.patch(
        "dummy_functions.os.path.exists",
        side_effect=[True, False],
    )

    assert check_file_exists("some/path/one") is True
    assert check_file_exists("some/path/two") is False

    assert mock_exists.call_count == 2

    assert [
        call.args for call in mock_exists.call_args_list
    ] == [("some/path/one",), ("some/path/two",)]

# Section: Using side_effect - Callable for Multiple Calls

def test_fetch_both_endpoints_by_url(mocker: MockerFixture):
    fake_responses: dict[str, MagicMock] = {}

    for url, data in [
        ("https://api.example.com/first", {"first": "data"}),
        ("https://api.example.com/second", {"second": "data"}),
    ]:
        resp = mocker.MagicMock()
        resp.status_code = 200
        resp.json.return_value = data

        fake_responses[url] = resp

    def _fake_get(url: str) -> MagicMock:
        return fake_responses[url]

    mocker.patch(
        "dummy_functions.requests.get", side_effect=_fake_get
    )

    result = fetch_both_endpoints()

    assert result == ({"first": "data"}, {"second": "data"})

# Section: Choosing between Mock and MagicMock

@pytest.mark.xfail(
    reason="Context managers do not work with Mock", strict=True
)
def test_context_manager_with_mock(mocker: MockerFixture):
    fake_cm = mocker.Mock()
    fake_cm.__enter__.return_value = fake_cm
    fake_cm.read.return_value = "file contents"

    mock_open = mocker.patch("builtins.open")
    mock_open.return_value = fake_cm

    with open("somefile.txt") as f:
        contents = f.read()

    mock_open.assert_called_once_with("somefile.txt")
    assert contents == "file contents"

def test_context_manager_with_magicmock(mocker: MockerFixture):
    fake_cm = mocker.MagicMock()
    fake_cm.__enter__.return_value = fake_cm
    fake_cm.read.return_value = "file contents"

    mock_open = mocker.patch("builtins.open")
    mock_open.return_value = fake_cm

    with open("somefile.txt") as f:
        contents = f.read()

    mock_open.assert_called_once_with("somefile.txt")
    assert contents == "file contents"

Parametrized Tests

Introduction

• Often, we need to test the same logic with different inputs and outputs, such as validating various IP address or hostname formats.
• Writing individual test functions for each case leads to repetitive code and a test suite that is harder to maintain.
• Parametrized tests allow a single test function to run multiple times with different data, adhering to the DRY principle and simplifying test maintenance.

The Problem: Duplicated Test Logic

• A function that checks valid hostname character codes must be tested across letters, digits, hyphens, and invalid symbols.
• Without parametrization, each input case requires its own test function, duplicating the assertion logic.
• This approach increases verbosity and makes the test suite more error-prone and tedious to update.

Solution: @pytest.mark.parametrize

• The @pytest.mark.parametrize(argnames, argvalues) decorator takes argument names and a list of value tuples to generate multiple test invocations.
• Argument names can be provided as a comma-separated string or as a list of strings.
• Each tuple in the argvalues list corresponds to a separate run of the test function, with tuple elements mapped to argument names.
• Running Pytest with -v shows each parametrized case as a distinct test, simplifying result interpretation.

The pytest.param Construct

• The pytest.param() function wraps a set of parameter values and allows you to attach metadata to that invocation:
  • id: a custom label shown in the test report.
  • marks: one or more markers (e.g., pytest.mark.xfail, pytest.mark.skip) applied only to that case.
• This is useful when you want to:
  • Give human-readable names to complex or ambiguous parameter sets.
  • Mark individual cases as expected failures or skip them selectively.

Customizing Test IDs

• By default, Pytest creates IDs from parameter values, which may be non-descriptive for complex data.
• You can use pytest.param(..., id="custom_id") to assign clear, human-readable names to individual cases.
• Alternatively, an ids list passed to parametrize can specify identifiers in the order of argvalues.
• Custom IDs make it easier to identify failing cases in test reports and improve overall readability.

import pytest

def is_valid_hostname_char(char: str) -> bool:
    if "a" <= char <= "z":
        return True
    if "0" <= char <= "9":
        return True
    if char == "-":
        return True
    return False

def check_url_status(url: str) -> tuple[int | str, str]:
    if url == "https://google.com":
        return (200, "OK")
    if url == "https://fakesite123.org/notfound":
        return (404, "HTTP_ERROR (404)")
    if url == "http://httpbin.org/status/503":
        return (503, "HTTP_ERROR (503)")
    if url == "http://localhost:1":
        return ("CONNECTION_ERROR", "CONNECTION_ERROR")
    return ("UNKNOWN", "UNKNOWN")

# Section: The Problem: Duplicated Test Logic

"""
a -> True
5 -> True
- -> True
A -> False
_ -> False
"""

def test_is_valid_lower_case_a():
    assert is_valid_hostname_char("a") is True

def test_is_valid_5():
    assert is_valid_hostname_char("5") is True

def test_is_valid_hyphen():
    assert is_valid_hostname_char("-") is True

def test_is_valid_upper_case_A():
    assert is_valid_hostname_char("A") is False

def test_is_valid_underscore():
    assert is_valid_hostname_char("_") is False

# Section: Solution: @pytest.mark.parametrize

@pytest.mark.parametrize(
    "input_char, expected_result",
    [
        ("a", True),
        ("5", True),
        ("-", True),
        ("A", False),
        ("_", False),
        ("!", False),
    ],
)
def test_is_valid_hostname_char(
    input_char: str, expected_result: bool
):
    assert is_valid_hostname_char(input_char) is expected_result

# Section: Customizing Test IDs with pytest.param construct

@pytest.mark.parametrize(
    "input_char, expected_result",
    [
        pytest.param("a", True, id="lowercase_letter_a"),
        pytest.param("z", True, id="lowercase_letter_z"),
        pytest.param("0", True, id="digit_0"),
        pytest.param("-", True, id="hyphen"),
        pytest.param("A", False, id="uppercase_A_invalid"),
        pytest.param("_", False, id="underscore_invalid"),
    ],
)
def test_is_valid_hostname_custom_params(
    input_char: str, expected_result: bool
):
    assert is_valid_hostname_char(input_char) is expected_result

@pytest.mark.parametrize(
    "url_to_check, expected_status_code, expected_status_text",
    [
        ("https://google.com", 200, "OK"),
        (
            "https://fakesite123.org/notfound",
            404,
            "HTTP_ERROR (404)",
        ),
        (
            "http://httpbin.org/status/503",
            503,
            "HTTP_ERROR (503)",
        ),
        (
            "http://localhost:1",
            "CONNECTION_ERROR",
            "CONNECTION_ERROR",
        ),
        pytest.param(
            "https://pending.retries.tests",
            503,
            "HTTP_ERROR (503)",
            marks=(
                pytest.mark.xfail(
                    reason="Retry logic for 503 is not yet implemented."
                ),
                pytest.mark.api,
            ),
        ),
    ],
    ids=[
        "google_ok",
        "site_not_found",
        "server_error_503",
        "connection_error",
        "xfail_retry_case",
    ],
)
def test_various_url_statuses(
    url_to_check: str,
    expected_status_code: int,
    expected_status_text: str,
):
    status_code, status_text = check_url_status(url_to_check)
    assert status_code == expected_status_code
    assert status_text == expected_status_text

Fixtures in Pytest

• As tests grow more complex, repeating setup and cleanup steps makes tests harder to read and maintain.
• Pytest fixtures allow centralizing shared setup and teardown logic, promoting DRY code.
• Fixtures prepare resources or data before tests and can optionally clean up after tests complete.

What is a Fixture?

• A fixture in Pytest is a function decorated with @pytest.fixture that provides a baseline environment or data for tests.
• Fixtures can supply test data, manage the test environment (e.g., temporary files, mock services), and provide reusable resources (e.g., client objects).
• Tests request fixtures by declaring them as function arguments, and Pytest handles executing the fixture and injecting its result.

Defining a Simple Fixture with @pytest.fixture

• Use @pytest.fixture above a function to mark it as a fixture that can return or yield resources.
• A fixture that returns a value runs its setup code, returns the value, and skips teardown logic.
• A fixture that yields a value transforms into a generator: code before yield is setup, the yielded value goes to the test, and code after yield is teardown.

Using Fixtures in Test Functions

• Tests receive fixture results by naming them as parameters; Pytest locates and executes matching fixtures automatically.
• This approach removes boilerplate setup code from tests and keeps test functions concise.

Fixture Scope and Lifecycle

• Fixture scope controls how often setup and teardown run: function (default), class, module, or session.
• A function-scoped fixture runs for each test function; a session-scoped fixture runs only once for the entire test session.
• Choosing the right scope balances resource isolation (function scope) against efficiency (session scope).

Fixture Teardown (Cleanup)

• To ensure cleanup logic executes after tests, define fixtures with yield rather than return.
• Code after yield always runs, even if the test fails or raises an error.
• This pattern mimics a try...finally block, ensuring resources like temporary files or connections are properly released.

conftest.py: Sharing Fixtures

• Defining fixtures in a conftest.py file makes them available across multiple test modules without explicit imports.
• Placing conftest.py in a test directory or its parent enables automatic discovery of shared fixtures.
• This structure keeps fixture definitions centralized and tests cleaner.

import pytest
from typing import Iterable

ManagedResource = dict[str, str]

@pytest.fixture
def managed_resource() -> Iterable[ManagedResource]:
    print("  [SHARED FIXTURE]: acquiring resource lock")
    resource = {"status": "lock_acquired"}
    yield resource
    print("  [SHARED FIXTURE]: releasing resource lock")
    resource["status"] = "lock_released"

Pytest Markers

• Markers are decorators (@pytest.mark.<markername>) applied to tests to attach metadata.
• Built-in markers like skip, skipif, xfail, and parametrize provide predefined behaviors.
• Custom markers (e.g., slow, integration, api) help categorize tests by project-specific criteria.
• Applying markers allows Pytest (and plugins) to filter, select, or modify tests during collection and execution.

Skipping Tests Unconditionally: @pytest.mark.skip

• The skip marker unconditionally prevents a test from running, useful when a feature is disabled or under refactor.
• Skipped tests show an s in the summary along with the provided reason.
• Use @pytest.mark.skip(reason="...") to disable tests without removing their code.

Skipping Tests Conditionally: @pytest.mark.skipif

• The skipif marker skips tests only when a specified condition evaluates to true at collection time.
• Conditions can be expressions like sys.platform != "linux" or checks for optional dependencies.
• This enables platform-specific, version-specific, or resource-dependent test filtering.

Expected Failures: @pytest.mark.xfail

• The xfail marker marks tests expected to fail due to known bugs or unimplemented features.
• Expected failures are reported as XFAIL and don’t cause the suite to fail; unexpected passes are reported as XPASS.

Custom Markers and Registration

• Custom markers (e.g., slow, api, integration) help organize and categorize tests by functionality or runtime.
• Tests can have multiple markers for fine-grained control (e.g., @pytest.mark.api and @pytest.mark.integration).
• To avoid warnings, register custom markers in pytest.ini or pyproject.toml under the markers option.

Running Tests by Marker (m option)

• The -m <expression> CLI option selects tests matching a marker expression.
• Expressions support logical operators: and, or, and negation with not.
• Examples: pytest -m slow, pytest -m "not slow", pytest -m "api and integration".

Common Pitfalls & How to Avoid Them

• Overusing skip hides failures; prefer xfail for known bugs to maintain visibility.
• Complex skipif conditions reduce readability; delegate logic to helper functions when needed.
• Typos in marker names create unregistered markers; register all custom markers to catch errors early.
• Forgetting to register markers leads to warnings; always define marker descriptions in configuration files.

import pytest
import time

try:
    import some_optional_library  # type: ignore
except ModuleNotFoundError:
    some_optional_library = None
# Section: Skipping Tests Unconditionally: @pytest.mark.skip

@pytest.mark.skip(
    reason="Skipping experimental feature until completion."
)
def test_new_experimental_feature() -> None:
    assert False

# Section: Skipping Tests Conditionally: @pytest.mark.skipif

@pytest.mark.skipif(
    some_optional_library is None,
    reason="Requires 'some_optional_library' to be installed.",
)
def test_with_optional_dependency() -> None:
    print(
        f"Running tests that depends on an optional library..."
    )
    assert some_optional_library

# Section: Expected Failures: @pytest.mark.xfail

@pytest.mark.xfail(
    reason="Bug #123: Division by zero not handled properly."
)
def test_divide_by_zero() -> None:
    _division = 1 / 0
    assert False

@pytest.mark.xfail  # Add strict=True to make XPASS lead to a failure
def test_expected_to_fail() -> None:
    assert True

# Section: Custom Markers and Registration

@pytest.mark.slow
def test_very_long_computations() -> None:
    time.sleep(5)
    assert True

@pytest.mark.api
@pytest.mark.smoke
def test_user_creation() -> None:
    assert True

# Section: Running Tests by Marker (m option)

Configuring Pytest

• As you start using Pytest extensively, typing -v or -m on the command line every time becomes tedious.
• Centralize your defaults in pyproject.toml under the [tool.pytest.ini_options] table.
• A single source of truth means every developer—and your CI system—runs tests with the same settings.
• Putting Pytest alongside other PEP 518 tools (Black, isort, Flake8) keeps your repo tidy and consistent.

Why a Configuration File?

• Consistency: Run pytest without remembering flags; everyone gets the same behavior.
• Simplicity: Remove boilerplate from docs and CI scripts.
• Project-specific discovery: Set testpaths, python_files and markers in one place.
• Cleaner output: Declare markers to silence PytestUnknownMarkWarning, enable color and rich tracebacks by default.

Configuration File Hierarchy

Pytest searches for settings in this order, using the first match from the current or a parent directory:

1. pyproject.toml under [tool.pytest.ini_options]
2. pytest.ini
3. tox.ini with a [pytest] section
4. setup.cfg under [tool:pytest]

Embrace pyproject.toml as the modern hub for all your tool configurations.

Creating pyproject.toml

1. Create or open pyproject.toml at your project root.
2. Add a [tool.pytest.ini_options] table.
3. Define your defaults using TOML syntax and inline strings.

Common Configuration Options

• addopts
  Defines default command-line flags that Pytest applies on every run (verbosity, reporting, color, etc.).

• markers
  Pre-registers custom markers with descriptions so that you can categorize tests and avoid unknown-marker warnings.

• testpaths
  Restricts test discovery to the listed directories, preventing Pytest from scanning other parts of the project.

• python_files
  Specifies filename patterns that Pytest treats as test files (e.g., test_*.py).

• python_classes
  Indicates class name patterns Pytest will consider when looking for test classes (e.g., classes starting with Test).

• python_functions
  Sets function name patterns Pytest uses to identify individual test functions (e.g., functions beginning with test_).

• Other options

  • norecursedirs: directories to skip during discovery
  • minversion: enforce a minimum Pytest version
  • filterwarnings: configure how warnings are handled
  • and many more built-in settings for fine-tuning

Example of pyproject.toml

from typing import TypedDict
import re

class TextAttributes(TypedDict):
    word_count: int
    unique_words: set[str]
    average_word_length: float
    longest_word: str

def calculate_text_attributes(input_text: str) -> TextAttributes:
    split_text = re.findall(r"\w+", input_text)
    word_length_sum = sum(len(word) for word in split_text)
    avg_word_length = (
        word_length_sum / len(split_text)
        if len(split_text)
        else 0
    )

    return {
        "word_count": len(split_text),
        "unique_words": set(text.lower() for text in split_text),
        "average_word_length": avg_word_length,
        "longest_word": (
            max(split_text, key=len) if split_text else ""
        ),
    }

Assertions in Pytest

• Pytest uses Python’s built-in assert statement to declare expected conditions in tests, making test code concise and readable.
• When an assert expression evaluates to True, execution continues; if it evaluates to False, an AssertionError is raised and Pytest marks the test as failed.
• Pytest intercepts assertion failures to provide detailed, introspective feedback on why an assertion failed.

The assert Statement

• The assert keyword checks that an expression is truthy; if it’s falsy, Python raises AssertionError.
• You can append an optional message: assert expression, "message", which will be shown if the assertion fails.
• In plain Python, assert x == 5 does nothing when true, while assert x == 10, "x should be 10" raises an error with that message if the condition is false.

Pytest and assert

• Pytest enhances the built-in assert by inspecting the expression’s values and rewriting the failure message to show variable states.
• Common assertion patterns include:
  • Equality and inequality checks to compare expected versus actual values.
  • Truthiness or falsiness checks to verify that objects are non-empty or evaluate to False.
  • Membership checks using in or not in to assert presence or absence in containers.
  • Comparison operators (<, >, <=, >=) to verify ordering conditions.

Pytest’s Rich Failure Output

• When an assertion fails, Pytest displays the values from the expression and highlights exactly where they differ.

Asserting Floating-Point Numbers (pytest.approx)

• Floating-point arithmetic can yield tiny precision errors, so direct equality comparisons may fail unexpectedly.
• Pytest provides pytest.approx to compare floats within a tolerance, supporting both relative and absolute tolerances.

Asserting Exceptions (pytest.raises)

• Use with pytest.raises(ExpectedException): as a context manager to assert that a block of code raises a specific exception.
• You can include match="regex" to verify that the exception message matches a given pattern.
• This allows testing both that the correct error type is raised and that its message contains expected details.

Common Pitfalls & How to Avoid Them

• Avoid overly complex expressions in a single assert; break them into multiple simpler assertions for clarity.
• Always use pytest.approx for floating-point comparisons to prevent false negatives from tiny precision differences.

from text_analysis import calculate_text_attributes
import pytest

# Section: The `assert` Statement

# Uncomment to play around with Python assertions

# x: int = 5

# assert x == 5  # Nothing will happen, because this is True
# assert (
#     x == 10
# ), "x should be 10, but it's not!"  # Raise an AssertionError

# Section: Pytest and `assert`

def test_string_equality() -> None:
    expected_status = "SUCCESS"
    actual_status = "success".upper()

    assert actual_status == expected_status

def test_word_count() -> None:
    text = "Deploying microservice to Kubernetes cluster."
    text_empty = ""

    assert (calculate_text_attributes(text)["word_count"]) == 5
    assert (
        calculate_text_attributes(text_empty)["word_count"]
    ) == 0

def test_unique_words() -> None:
    text = "Deploying microservice to Kubernetes cluster."
    text_with_duplicates = "Deploying deploying."
    text_empty = ""

    text_results = calculate_text_attributes(text)
    text_with_duplicates_result = calculate_text_attributes(
        text_with_duplicates
    )
    text_empty_results = calculate_text_attributes(text_empty)

    assert (len(text_results["unique_words"])) == 5
    assert (
        len(text_with_duplicates_result["unique_words"])
    ) == 1
    assert (len(text_empty_results["unique_words"])) == 0

def test_average_word_length() -> None:
    text = "Deploying microservice to Kubernetes cluster."  # 40 / 5 = 8
    text_with_duplicates = "Deploying deploying."  # 18 / 2 = 9
    text_empty = ""  # 0

    text_results = calculate_text_attributes(text)
    text_with_duplicates_result = calculate_text_attributes(
        text_with_duplicates
    )
    text_empty_results = calculate_text_attributes(text_empty)

    assert (text_results["average_word_length"]) == 8.0
    assert (
        text_with_duplicates_result["average_word_length"]
    ) == 9.0
    assert (text_empty_results["average_word_length"]) == 0.0

def test_longest_word() -> None:
    text = "Deploying microservice to Kubernetes cluster."  # microservice
    text_with_duplicates = "Deploying deploying."  # deploying
    text_empty = ""

    text_results = calculate_text_attributes(text)
    text_with_duplicates_result = calculate_text_attributes(
        text_with_duplicates
    )
    text_empty_results = calculate_text_attributes(text_empty)

    assert (
        text_results["longest_word"].lower()
    ) == "microservice"
    assert (
        text_with_duplicates_result["longest_word"].lower()
    ) == "deploying"
    assert (text_empty_results["longest_word"]) == ""

# Section: Pytest’s Rich Failure Output

@pytest.mark.xfail  # We're marking the test as an expected failure
def test_string_mismatch() -> None:
    expected = "HEllo WOrlD"
    actual = "hello world"

    assert expected == actual

# Section: Asserting Floating-Point Numbers (`pytest.approx`)

def test_float_with_approx() -> None:
    calculated_val = 0.1 + 0.2
    expected_val = 0.3

    assert calculated_val == pytest.approx(expected_val)  # type: ignore

# Section: Asserting Exceptions (`pytest.raises`)

def test_raises_exception() -> None:
    with pytest.raises(ZeroDivisionError):
        _division = 1 / 0

from typing import TypedDict
import re

class TextAttributes(TypedDict):
    word_count: int
    unique_words: set[str]
    average_word_length: float
    longest_word: str

def calculate_text_attributes(input_text: str) -> TextAttributes:
    split_text = re.findall(r"\w+", input_text)
    word_length_sum = sum(len(word) for word in split_text)
    avg_word_length = (
        word_length_sum / len(split_text)
        if len(split_text)
        else 0
    )

    return {
        "word_count": len(split_text),
        "unique_words": set(text.lower() for text in split_text),
        "average_word_length": avg_word_length,
        "longest_word": (
            max(split_text, key=len) if split_text else ""
        ),
    }

Adding Type Hints to Decorators and Generators

• Decorators and generators are advanced constructs that require specialized type hints to make their transformations and data flows explicit.
• Properly typed decorators allow MyPy to understand how they preserve or change function signatures.
• Typed generators clarify the types of values yielded, values accepted via .send(), and final return values.

Typing Decorators

• Decorators take a function (Callable) and return a new function; using Callable[..., Any] types them broadly but loses specific signature information.
• To preserve the original function’s signature, define a TypeVar bound to Callable[..., Any] and use it for both the decorator’s input and output types.
• Inside the decorator, the wrapper can use *args: Any, **kwargs: Any -> Any, while TypeVar ensures the decorated function’s overall type remains correct.

Typing Generators

• Use Generator[YieldType, SendType, ReturnType] to specify a generator’s yield type, the type accepted by .send(), and its return type on completion.
• If a generator does not use send(), set SendType to None; if it has no explicit return, set ReturnType to None.
• The count_up generator is typed as Generator[int, None, str], yielding integers and returning a string message.
• The accumulate_and_send generator is typed as Generator[float, float, None], yielding a running total, accepting floats via send(), and returning nothing.

Iterable & Iterator

• For functions that consume sequences of items, use Iterable[T] to accept any iterable of T (lists, tuples, generators).
• Use Iterator[T] when a function specifically expects an iterator object supporting __next__().

from typing import (
    Callable,
    Any,
    TypeVar,
    ParamSpec,
    Generator,
    Iterable,
)
import functools

# Section: Typing Decorators (simple_logging_decorator)

def simple_logging_decorator(
    func: Callable[..., Any],
) -> Callable[..., Any]:
    @functools.wraps(func)
    def wrapper(*args: Any, **kwargs: Any) -> Any:
        print(f"LOG: Calling {func.__name__}")
        result = func(*args, **kwargs)
        print(f"LOG: {func.__name__} returned {result}")

        return result

    return wrapper

@simple_logging_decorator
def add(x: int, y: int) -> int:
    return x + y

result_add = add(3, 5)

# Section: Typing Decorators (better_logging_decorator with TypeVar)

P = ParamSpec("P")
R = TypeVar("R")

def better_logging_decorator(
    func: Callable[P, R],
) -> Callable[P, R]:
    @functools.wraps(func)
    def wrapper(*args: P.args, **kwargs: P.kwargs) -> Any:
        print(f"LOG: Calling {func.__name__}")
        result = func(*args, **kwargs)
        print(f"LOG: {func.__name__} returned {result}")

        return result

    return wrapper

@better_logging_decorator
def subtract(x: int, y: int) -> int:
    return x - y

result_subtract = subtract(3, 5)

# Section: Typing Generators

def count_up_to(limit: int) -> Generator[int, None, str]:
    for i in range(limit):
        yield i

    return "Counting complete!"

def accumulate_and_send() -> (
    Generator[float, float | None, None]
):
    total = 0.0

    try:
        while True:
            sent = yield total

            if sent:
                total += sent
    except GeneratorExit:
        pass

test_accumulate = accumulate_and_send()
next(test_accumulate)
print(test_accumulate.send(1.0))
print(next(test_accumulate))
print(test_accumulate.send(2.0))
print(test_accumulate.send(3.0))
print(next(test_accumulate))

# Section: Iterable & Iterator

def process_items(items: Iterable[str]) -> list[str]:
    return [item.upper() for item in items]

print(process_items(["a", "b"]))
print(process_items(("a", "b")))
print(process_items({"a", "b"}))
print(process_items({"a": "b", "hello": "world"}))

Introduction to Generics

• Generic types let you write reusable, type-safe functions and classes that work uniformly across different data types.
• They preserve the relationship between input and output types, enabling MyPy to infer precise types instead of falling back to Any.
• The typing module’s TypeVar and Generic primitives unlock this capability.

The Need for Generics

• Annotating with Any sacrifices type information, so tools cannot guarantee correct usage of returned values.
• A generic abstraction retains knowledge of the specific type in each context, improving IDE support and static checks.
• For example, a "first-item" function should return str for a list[str] and int for a list[int], not just Any.

Defining Type Variables

• T = TypeVar('T') declares a placeholder type variable T that can stand for any type.
• A function annotated def get_first_item_generic(data: List[T]) -> Optional[T]: returns an element of the same type as the list elements.
• MyPy infers T from each call site, preserving specific return types like Optional[str] or Optional[int].

Constrained Type Variables

• When a generic should only accept certain types, constrain it: NumberType = TypeVar('NumberType', int, float).
• Functions like def add_generic_numbers(x: NumberType, y: NumberType) -> NumberType: then only accept int or float and return that same type.
• Constrained type variables combine flexibility with necessary restrictions for safe operations.

Bounded Type Variables

• When a generic should only accept subclasses of a specific superclass, we can use a type bound, constrain it: NumberType = TypeVar('NumberType', bound=Superclass).
• Functions like def add_generic_numbers(x: NumberType, y: NumberType) -> NumberType: then accept any subclass of Superclass, and they can be different subclasses for each argument.
• Like constrained type variables, bounded type variables provide useful functionalities combining flexibility and type safety.

Generic Classes

• Inherit from Generic[T] to define a class parameterized by a type variable T.
• A class like SimpleStack[T] can push, pop, and peek items of type T, and MyPy will enforce that only T instances are used.
• This pattern creates custom container types that maintain strong type guarantees for their contents.

Common Pitfalls & How to Avoid Them

• A class that uses T in its methods but does not inherit from Generic[T] is not recognized as generic by MyPy.
• Unconstrained TypeVar('T') can degrade type safety when operations require certain capabilities—use bounds or explicit type lists when appropriate.

from typing import Optional, TypeVar, Generic

# Section: Defining a generic function to get the first item of a list

T = TypeVar("T")

def get_first_item(
    input_list: list[T],
) -> Optional[T]:
    if input_list:
        return input_list[0]

    return None

first_number = get_first_item([1, 2, 3])
first_str = get_first_item(["abc", "def"])
first_mixed_list = get_first_item(["abc", "def", 1, 2, 3])

# Section: Constrained TypeVar for numeric addition

NumberType = TypeVar("NumberType", int, float)

def add_generic_numbers(
    x: NumberType, y: NumberType
) -> NumberType:
    return x + y

sum_int = add_generic_numbers(3, 5.0)

# Section: Bounded TypeVar with deployed filter for DevOps resources

class CloudResource:
    def __init__(self, name: str, cpu_usage: float) -> None:
        self.name = name
        self.cpu_usage = cpu_usage
        self.deployed: bool = False

    def deploy(self) -> None:
        print(f"Deploying {self.name}")
        self.deployed = True

class VirtualMachine(CloudResource):
    def reboot(self) -> None:
        print(f"Rebooting VM {self.name}")

class DockerContainer(CloudResource):
    def restart(self) -> None:
        print(f"Restarting container {self.name}")

ResourceType = TypeVar("ResourceType", bound=CloudResource)

def filter_deployed(
    resources: list[ResourceType],
) -> list[ResourceType]:
    return [
        resource for resource in resources if resource.deployed
    ]

vm1 = VirtualMachine("vm-01", cpu_usage=65.0)
vm2 = VirtualMachine("vm-02", cpu_usage=45.0)
container1 = DockerContainer("api-service", cpu_usage=85.0)
container2 = DockerContainer("worker", cpu_usage=55.0)

vm1.deploy()
container1.deploy()

all_resources = [vm1, vm2, container1, container2]
deployed_resources = filter_deployed(all_resources)

# Section: Generic class SimpleStack

G = TypeVar("G")

class SimpleStack(Generic[G]):
    def __init__(self) -> None:
        self._items: list[G] = []

    def push(self, item: G) -> None:
        self._items.append(item)

    def pop(self) -> G:
        if self.is_empty():
            raise IndexError("Stack is empty!")
        return self._items.pop()

    def peek(self) -> Optional[G]:
        if self.is_empty():
            return None

        return self._items[-1]

    def is_empty(self) -> bool:
        return not self._items

str_stack = SimpleStack[str](http://)
str_stack.push("str")

int_stack = SimpleStack[int](http://)
int_stack.push(12)

Introduction

• As our Python automation projects grow, defining custom classes helps model complex objects and should be reflected in type hints for clearer code and stronger static checking.
• Annotating functions and methods with user-defined classes lets MyPy verify correct usage of attributes and methods.

Classes as Type Hints

• Any class you define becomes a valid type; you can annotate parameters and return values with it.
• MyPy will ensure that calls to such functions pass instances of the expected class and that attribute access matches the class definition.

Hinting Methods Within a Class

• Inside class methods, self is implicitly the class type; annotate other parameters and return types normally.
• MyPy checks method bodies to ensure you only access attributes and call methods that exist on the class.
• New in Python 3.11: You can use typing.Self for methods that return the instance, for example def clone(self) -> Self:.

Forward References (Strings)

• Use string literals for type hints when referring to a class that is defined later in the file or in circular scenarios.
• Enabling from __future__ import annotations defers evaluation of all annotations, simplifying forward references.

from __future__ import annotations
from typing import Self, Optional

# Section: Classes as Type Hints

class Server:
    def __init__(
        self,
        hostname: str,
        ip_address: str,
        os_type: str = "Linux",
    ):
        self.hostname: str = hostname
        self.ip_address: str = ip_address
        self.os_type: str = os_type
        self.is_online: bool = False

    def connect(self) -> None:
        print(
            f"Connecting to {self.hostname} (IP address: {self.ip_address})"
        )
        self.is_online = True
        print(f"{self.hostname} is online.")

    def get_status(self) -> str:
        return "online" if self.is_online else "offline"

def deploy_app_to_server(
    target_server: Server, app_name: str
) -> bool:
    print(
        f"Deploying {app_name} to server: {target_server.hostname}"
    )

    if not target_server.is_online:
        target_server.connect()

    print(
        f"Deployment of {app_name} to {target_server.hostname} successful."
    )
    return True

web_server = Server(
    hostname="web01.dev.local", ip_address="10.0.1.10"
)
db_server = Server(
    hostname="db01.dev.local", ip_address="10.0.2.20"
)

deploy_app_to_server(web_server, "FrontendApp")
deploy_app_to_server(db_server, "UserDBApi")

# Section: Hinting Methods Within a Class

class Calculator:
    def __init__(self, initial_value: int | float = 0):
        self.total: int | float = initial_value

    def add(self, value: int | float) -> Self:
        self.total += value

        return self

    def subtract(self, value: int | float) -> Self:
        self.total -= value

        return self

    def multiply_by(self, value: int | float) -> Self:
        self.total *= value

        return self

    def divide_by(self, value: int | float) -> Self:
        self.total /= value

        return self

    def get_total(self) -> int | float:
        return self.total

my_calc = Calculator(1)

print(my_calc.add(2).subtract(1).multiply_by(10).get_total())

# Section: Forward References (Strings)

class Employee:
    def __init__(
        self, name: str, manager: Optional[Employee] = None
    ) -> None:
        self.name: str = name
        self.manager: Optional[Employee] = manager
        self.reports: list[Employee] = []

    def add_report(self, report: Employee) -> None:
        self.reports.append(report)

ceo = Employee("ceo")
manager1 = Employee("Alice", ceo)
ceo.add_report(manager1)

Introduction

• Python is a dynamically typed language, meaning you can assign values to variables without declaring their types, and type checking happens at runtime.
• While this offers rapid development and flexibility, it can lead to ambiguity and late discovery of type-related bugs in larger or collaborative projects.
• Type hints (PEP 484, introduced in Python 3.5) let you optionally annotate your code with expected types for variables, function parameters, and return values without changing Python’s runtime behavior.
• These annotations are leveraged by static type checkers (e.g., MyPy), IDEs for better autocompletion and error highlighting, and by developers for clearer, more maintainable code.

Why Use Type Hints?

• Type hints improve readability by making explicit what data types functions expect and return, which is invaluable when navigating unfamiliar or legacy code.
• Static type checkers like MyPy can catch mismatches between hinted and actual types before the code runs, surfacing bugs early in the development cycle.
• IDEs (e.g., VS Code, PyCharm) use hints to enhance autocompletion accuracy, provide inline type checking, and support safe refactoring.
• Explicit annotations act as a contract in collaborative environments, helping team members understand and correctly use each other’s code.
• For example, annotating a function as def process_user_data(user: dict) -> bool: makes it clear that the function expects a dict and returns a bool.

Basic Type Hint Syntax

• To annotate a variable, use variable_name: type = value. This syntax (variable annotations) was introduced in Python 3.6 (PEP 526).
• Example: config_path: str = "/etc/app.conf" indicates that config_path is intended to be a string.
• Function parameters are annotated with param_name: param_type, and the return type is specified after -> before the colon.
• Example: def get_server_status(hostname: str, port: int) -> str: declares that the function takes a str and an int, and returns a str.

Common Built-in Types for Hinting

• Standard built-ins such as int, float, bool, str, and bytes are directly usable in annotations.
• Collections can be hinted with list, tuple, set, and dict. For more precise element types:
  • In Python 3.9 and later (PEP 585), you can use built-in generics: list[int], dict[str, int].
  • In earlier versions, import from the typing module: from typing import List, Dict and use List[int], Dict[str, int].
• The special type None is used for functions that do not return a meaningful value (e.g., -> None).
• Advanced types like Optional, Union, and others will be covered when exploring the typing module in a later lecture.

Python Remains Dynamically Typed

• Type hints do not alter Python’s runtime behavior; passing arguments of the wrong type won’t raise a hint-related error unless an operation in the code fails for the actual type.
• For instance, calling process_id("user-123") on a function annotated as def process_id(user_id: int) -> None: runs without a hint-triggered error, though passing a string where an integer is expected may lead to a TypeError later if arithmetic is attempted.
• Static analysis tools flag these mismatches before execution, but Python itself enforces types only when invalid operations occur at runtime.

Common Pitfalls & How to Avoid Them

• Believing hints enforce types at runtime: Hints guide tools and developers, but Python ignores them unless you use a runtime checking library.
• Over-hinting or incorrect hints: Overly complex or wrong annotations can confuse readers and static checkers; start simple and use Any for truly dynamic values.
• Forgetting typing imports: When using List[int], Optional[str], etc., remember to import them from the typing module (unless you rely on built-in generics in Python 3.9+).
• Relying on hints for untyped libraries: If a third-party library lacks type hints or has them in separate stub files, static analysis may be limited—consult documentation or stub packages.

# Section: Basic Type Hint Syntax - Variable Annotations
config_path: str = "/etc/app.conf"
retry_count: int = 3
is_enabled: bool = bool(1)
servers: list[str] = ["web01", "web02"]
settings: dict[str, int | str] = {"port": 8080, "user": "admin"}

# Section: Basic Type Hint Syntax - Function Argument and Return Type Annotations
def get_server_status(hostname: str, port: int) -> str:
    print(f"Checking {hostname}:{port}")
    if port == 80:
        return "Online"
    else:
        return "Unknown"

# Section: Python Remains Dynamically Typed
def process_id(user_id: int) -> None:
    print(
        f"Processing user ID: {user_id} (type: {type(user_id)})"
    )

# Demonstration of dynamic typing
process_id(1234)
# process_id("user-1234") # Uncommenting will lead to a static type checking error.

Common Types in Python

• Python’s built-in dynamic typing allows rapid development without declaring variable types, but it can lead to ambiguous code and late discovery of type errors in larger projects.
• The typing module provides specialized type constructors to precisely describe the contents of collections (list, dict, tuple, set) and other complex scenarios.
• By using these constructors, you gain clearer documentation, stronger static analysis with tools like MyPy, and richer IDE support without changing Python’s runtime behavior.

The typing Module

• On Python 3.9+, built-in generics (list[int], dict[str, str], tuple[int, ...], set[str], frozenset[int]) are available via PEP 585, deprecating typing.List etc. for these cases.
• Import specific constructors from typing, for example: List, Dict, Tuple, Set, FrozenSet, Optional, Union, Any.
• Using typing remains necessary for compatibility with older versions (Python 3.7/3.8) and for constructs like Optional, Union, Literal, and TypedDict.

Typing Lists

• Use list[X] (or List[X] in Python < 3.9) to indicate a list whose elements are of type X.
• This makes it explicit if a function expects a list of strings (list[str]) or integers (list[int]), enabling static checkers to catch mismatches.

Typing Dictionaries

• Use dict[K, V] (or Dict[K, V] in Python < 3.9) to specify a dictionary with keys of type K and values of type V.
• You can nest generics, for example dict[int, list[str]], to model complex structures like mapping user IDs to role lists.

from typing import TypedDict, NotRequired

class User(TypedDict):
    id: int
    name: str
    email: str
    phone: NotRequired[str]

user: User = {
    "id": 123,
    "name": "Alice",
    "email": "alice@example.com",
    "phone": "+123456789",
}

print(f"User data: {user.get("email")}")

Typing Tuples

• Fixed-length tuples with heterogeneous types use tuple[T1, T2, ...] (or Tuple[T1, T2, ...] in Python < 3.9).
• Variable-length tuples of a uniform type use tuple[T, ...] (or Tuple[T, ...]), though lists are often more natural for that use case.

Typing Sets

• Use set[X] (or Set[X] in Python < 3.9) to indicate a set containing elements of type X.
• This clarifies that operations like membership checks (in) will compare values of the declared type.
• Note: For immutable sets, use frozenset[X] (or FrozenSet[X] in Python < 3.9).

Union[X, Y, ...] for Multiple Possible Types

• Use Union[...] when a value may be exactly one of several types (excluding None unless explicitly included).
• As of Python 3.10 you can write int | str instead of Union[int, str].

Optional[X] for Values That Can Be None

• Optional[X] is shorthand for Union[X, None], indicating a value may be of type X or None.
• Static checkers will warn if you use an Optional value without first checking for None.

Any for Unrestricted Types

• Any disables type checking for the annotated part, useful during gradual typing of legacy code or when truly dynamic types are needed.
• Overuse negates the benefits of static analysis, so prefer specific types whenever possible.

Common Pitfalls & How to Avoid Them

• Built-in Generics on Older Python: Syntax like list[int] only works on Python 3.9+; use typing.List[int] for Python 3.7/3.8 compatibility.
• Subtle Optional Defaults: def func(arg: Optional[str] = None) clearly allows None as a default, whereas def func(arg: str = None) may confuse static checkers.
• Excessive Any: Reserving Any for truly dynamic cases preserves the value of static checking elsewhere in your code.

from typing import Optional, Any

# Section: Typing Lists

hostnames: list[str] = ["web01.example.com", "db01.example.com"]
open_ports: list[int] = [80, 443, 22]

def process_hostnames(hosts: list[str]) -> None:
    for host in hosts:
        print(f"Processing host: {host.upper()}")

process_hostnames(hostnames)
# process_hostnames(open_ports) # Uncommenting will lead to type error

# Section: Typing Dictionaries

server_config: dict[str, str] = {
    "hostname": "app01.prod",
    "ip_address": "10.0.5.20",
    "os_type": "Linux",
}

user_roles: dict[str, list[str]] = {
    "user-123": ["admin", "editor"],
    "user-456": ["dev", "viewer"],
}

# Section: Typing Tuples

server_status: tuple[str, int, bool] = (
    "api.example.com",
    443,
    True,
)

ip_parts: tuple[int, ...] = (192, 168, 1, 100)

# Section: Typing Sets

admin_users: set[str] = {"alice", "bob", "charlie"}

def is_admin(username: str, admins: set[str]) -> bool:
    return username in admins

# Section: Union[X, Y, ...] for Multiple Possible Types

identifier: str | int = "abcde-1234"
identifier = 1234

def process_mixed_data(data: list[int | str]) -> None:
    for item in data:
        if isinstance(item, str):
            print(f"Processing string: {item.upper()}")
        else:
            print(f"Processing int: {item * 2}")

# Section: Optional[X] for Values That Can Be None

def find_user(user_id: str) -> Optional[dict[str, str]]:
    if user_id == "123":
        return {
            "id": "123",
            "name": "Admin user",
            "email": "admin@example.com",
        }

    return None

found_user = find_user("123")

if found_user:
    print(f"Found user: {found_user["name"]}")

# Section: Any for Unrestricted Types
def print_anything(item: Any) -> None:
    print(f"Item: {item}, type: {type(item)}")

print_anything(1)
print_anything("hello")

Implementing Retries and Timeouts

• External services can be slow or unreliable, causing scripts to hang or fail unexpectedly.
• Timeouts and retries help ensure your automation scripts remain responsive and resilient.

Timeouts

• By default, requests may wait indefinitely for a response, which is risky in automation.
• Use the timeout parameter with a single value for both connect and read, or a tuple (connect, read) for fine-grained control.
• A ConnectTimeout is raised if the connection can’t be established in time; a ReadTimeout is raised if data stops arriving within the read timeout.

HTTPBIN_ENDPOINT = "https://httpbin.org"

import requests
import time

delay_url = f"{HTTPBIN_ENDPOINT}/delay/5" # Simulate a 5-second delay

start = time.perf_counter()

try:
    res = requests.get(delay_url, timeout=2)
    print(f"Completed in {time.perf_counter() - start:.2f}s, status {response.status_code}")
except (
    requests.exceptions.ConnectTimeout,
    requests.exceptions.ReadTimeout
) as timeout_err:
    print(f"Timeout after {time.perf_counter() - start:.2f}s: {timeout_err}")

Retries

• Transient issues like network blips or server overloads may cause requests to fail temporarily.
• Implement a simple retry loop that catches errors, retries on server-side (5xx) errors or network exceptions, and breaks on success or client errors.
• Use a fixed delay between retries for simplicity, or an exponential backoff for a more robust approach.
• Avoid retrying non-idempotent operations.

import requests
import time

flaky_url = f"{HTTPBIN_ENDPOINT}/status/200,500,503"

max_retries = 3
delay = 2

for attempt in range(1, max_retries + 1):
    print(f"Attempt {attempt}/{max_retries}...")

    try:
        res = requests.get(flaky_url, timeout=10)
        res.raise_for_status()
        print(f"Succeeded with status {res.status_code}")
        break
    except requests.exceptions.HTTPError as err:
        if err.response.status_code < 500:
            print(f"Failed with client error code {err.response.status_code}. Skipping retry.")
            break
        else:
            print(f"Failed with server error code {err.response.status_code}.")
    if attempt < max_retries:
        print(f"Waiting {delay}s before retry...")
        time.sleep(delay)
else:
    print(f"All {max_retries} attempts failed!")

Exponential Backoff with Jitter

• Fixed delays can overwhelm a recovering server if many clients retry simultaneously.
• Exponential backoff increases the wait time after each failure (e.g., 1s, 2s, 4s...).
• Adding jitter (a small random offset) prevents synchronized retry spikes.

import requests
import time
import random

def get_with_backoff(url, max_retries=3):
    delay=1

    for attempt in range(1, max_retries + 1):
        print(f"Attempt {attempt}/{max_retries}...")

        try:
            res = requests.get(url, timeout=10)
            res.raise_for_status()
            print(f"Succeeded with status {res.status_code}")
            return res
        except requests.exceptions.HTTPError as err:
            if err.response.status_code < 500:
                print(f"Failed with client error code {err.response.status_code}. Skipping retry.")
                raise RuntimeError(f"Client error! Please review request.")
            else:
                jitter = random.uniform(-0.1 * delay, 0.1 * delay)
                # delay = 1 -> jitter [-0.1, 0.1] -> 0.9 and 1.1s
                # delay = 2 -> jitter [-0.2, 0.2] -> 1.8 and 2.2s
                # delay = 4 -> jitter [-0.4, 0.4] -> 3.6 and 4.4s
                wait = min(delay * 2, 30) + jitter
                print(f"  Failed with server error code {err.response.status_code}. Retrying in {wait:.2f}s")
                time.sleep(wait)
                delay = min(delay * 2, 30)
    raise RuntimeError(f"All retries to query {url} failed!")

try:
    res = get_with_backoff(
        f"{HTTPBIN_ENDPOINT}/status/503",
        max_retries=4
    )
except RuntimeError as e:
    print(e)

Common Pitfalls & How to Avoid Them

• Forgetting to set timeouts can cause scripts to hang indefinitely; always use timeout.
• Retrying client errors (4xx) usually won’t help; only retry transient server errors (5xx) or network issues.
• Retrying non-idempotent operations (e.g., POST) can cause duplicate actions; limit retries to safe methods.
• Fixed retry delays can lead to synchronized retry spikes; use exponential backoff with jitter for production scenarios.
  python

Handling Authentication

• APIs often require authentication to control access, rate limits, and auditing.
• Without authentication, requests to protected endpoints will fail with codes like 401 (Unauthorized) or 403 (Forbidden).
• This section demonstrates a simple GET to a protected endpoint, illustrating why auth is needed.

Why Authentication?

• Authentication tells the API who you are, enabling personalized data and higher rate limits.
• It prevents unauthorized access to private resources and supports auditing of actions.
• Authenticated requests often succeed where anonymous requests would be blocked or limited.

GITHUB_ENDPOINT = "https://api.github.com"
HTTPBIN_ENDPOINT = "https://httpbin.org"

import requests

urls = {
    "public_endpoint": f"{GITHUB_ENDPOINT}/zen",
    "protected_endpoint": f"{GITHUB_ENDPOINT}/user",
}

for description, url in urls.items():
    res = requests.get(url, timeout=5)
    print(f"{description} ({url}) : {res.status_code}")
    print(res.text[:200])

Basic Authentication

• Basic Auth sends a username and password with each request, encoded in the Authorization header.
• requests accepts an auth=(username, password) tuple and handles encoding automatically.
• Servers return 401 Unauthorized when credentials are missing or incorrect.

import requests
import json

url = f"{HTTPBIN_ENDPOINT}/basic-auth/myuser/myotherpwd"

try:
    res = requests.get(url, auth=("myuser", "mypasswd"), timeout=10)
    res.raise_for_status()
    print(f"Status code: {res.status_code}")
    print("Response JSON:")
    print(json.dumps(res.json()))
except requests.exceptions.HTTPError as err:
    print(err)

Token-Based Authentication

• Modern APIs use API keys or bearer tokens passed via the Authorization header.
• For GitHub PATs, use `Authorization: token - Always load tokens from environment variables to avoid hardcoding secrets.

import requests
import os
from dotenv import load_dotenv

load_dotenv(override=True)

token = os.getenv("GH_PAT", "")
print(f"Token: {token[:15]}")

urls = {
    "public_endpoint": f"{GITHUB_ENDPOINT}/zen",
    "protected_endpoint": f"{GITHUB_ENDPOINT}/user",
}

for description, url in urls.items():
    try:
        headers = {
            "Authorization": f"Bearer {token}"
        }
        res = requests.get(url, headers=headers, timeout=10)
        res.raise_for_status()
        print(f"Status code: {res.status_code}")
        print(f"Authenticated user: {res.json().get("login")}")
    except requests.exceptions.JSONDecodeError as err:
        print(f"Invalid JSON in response body. Defaulting to text:")
        print(res.text[:200])
    except requests.exceptions.HTTPError as err:
        print(err)

Common Pitfalls & How to Avoid Them

• Using the wrong header format (e.g., Bearer vs token) causes 401/403 errors. Follow API docs.
• Hardcoding secrets risks accidental exposure; always use environment variables or secret managers.
  python

Handling Errors and Status Codes

• HTTP status codes communicate the outcome of an API request, and handling them correctly is key to robust automation.
• A simple 200 OK means success, while codes like 404 Not Found or 500 Internal Server Error indicate different failure modes.
• In this lecture, we’ll learn how to check status codes, use response.ok, raise errors automatically, and inspect error details for troubleshooting.

Understanding HTTP Status Codes

• Status codes are grouped by their first digit: 1xx (informational), 2xx (success), 3xx (redirection), 4xx (client error), 5xx (server error).
• Examples include 200 OK, 201 Created, 301 Moved Permanently, 404 Not Found, and 500 Internal Server Error.
• Knowing these categories helps you decide how to handle each response in your scripts.

Checking response.status_code

• After a requests call, the integer response.status_code tells you the exact HTTP code returned.
• You can compare it directly (e.g., if resp.status_code == 404:) to implement custom logic based on the code.
• This explicit check is useful when you need fine-grained control over specific status codes.

GITHUB_ENDPOINT = "https://api.github.com"
HTTPBIN_ENDPOINT = "https://httpbin.org"

import requests

urls = {
    "ok": f"{GITHUB_ENDPOINT}/zen",
    "not_found": f"{GITHUB_ENDPOINT}/nonexistentendpoint"
}

for description, url in urls.items():
    response = requests.get(url, timeout=5)
    print(f"{description}: status {response.status_code}")

Using response.ok

• The boolean response.ok is True for any status code below 400 (1xx, 2xx, 3xx) and False for 4xx/5xx errors.
• This provides a quick success/failure check without examining the numeric code directly.
• It’s a handy shorthand when you only need to know if the request broadly succeeded.

import requests

urls = {
    "ok": f"{GITHUB_ENDPOINT}/zen",
    "not_found": f"{GITHUB_ENDPOINT}/nonexistentendpoint"
}

for description, url in urls.items():
    response = requests.get(url, timeout=5)
    print(f"{description}: ok? {"Yes" if response.ok else f"No. Failed with status {response.status_code}"}")

Automatic Error Raising with raise_for_status()

• Calling response.raise_for_status() will do nothing on 1xx, 2xx and 3xx codes but raise an HTTPError on 4xx/5xx.
• This follows the EAFP (“Easier to Ask Forgiveness than Permission”) style: try the request, and catch errors if they occur.
• The caught exception carries the original response in its response attribute, letting you inspect headers and body.

import requests
import json

urls = {
    "ok": f"{GITHUB_ENDPOINT}/zen",
    "not_found": f"{GITHUB_ENDPOINT}/nonexistentendpoint"
}

for url in urls.values():
    print(f"Requesting: {url}")
    try:
        res = requests.get(url, timeout=5)
        res.raise_for_status()
        print("  Success!")
    except requests.exceptions.HTTPError as err:
        print(f"  HTTPError: {err} (status {err.response.status_code})")
        try:
            details = err.response.json()
            print("  Error details:")
            print(json.dumps(details, indent=2))
        except ValueError:
            print(f"  Non-JSON response body: {err.response.text[:100]}")

Common Pitfalls & How to Avoid Them

• Not checking errors: Treating any response as success can mask failures. Always use ok or raise_for_status().
• Catching too broadly: A generic except Exception: hides HTTP errors. Catch HTTPError specifically.
• Ignoring error bodies: APIs often return JSON error messages; inspect response.text or response.json().

Making HTTP Requests

• The requests library simplifies HTTP interactions by abstracting raw HTTP details, making it ideal for DevOps automation tasks.
• It allows you to query web services, trigger CI/CD builds, manage cloud resources, and integrate with APIs like GitHub in a straightforward way.
• In this notebook, we'll demonstrate installing requests, performing GET and POST requests, inspecting response data, and customizing requests with parameters and headers.
• The requests library is a third-party package and must be installed in your active virtual environment.
• Use pip install requests==2.32.2 to add it to your project (pinning the version here so that we all work with the same version, but in other projects you can omit the version to install the latest), and consider pinning its version in requirements.txt.

GITHUB_ENDPOINT = "https://api.github.com"

Making GET Requests with requests.get()

• The GET method retrieves data from a specified URL; it’s the most common HTTP request type.
• Key parameters include url, optional params for query strings, headers for custom HTTP headers, and timeout to avoid hanging requests.
• The returned Response object provides .status_code, .headers, .text, .content, and .json(), plus .raise_for_status() to handle HTTP errors.

import requests
import json

response = requests.get(GITHUB_ENDPOINT, timeout=10)

print(f"Status code: {response.status_code}")
print(f"Content-Type: {response.headers.get("Content-Type")}")

"""
# Commenting out for brevity, but leaving for documentation

print(".text attribute:")
print(response.text)
print("\n")
print(".content attribute:")
print(response.content)
print("\n")
print(".json() method:")
print(response.json())
"""

data = response.json()
print("Available endpoints:")
print(json.dumps(data, indent=2))

Passing URL Parameters with params

• Query parameters are passed as a dictionary to the params argument, and requests handles URL-encoding automatically.
• This makes it easy to filter, sort, or paginate API results without manually constructing the query string.
• You can inspect the final URL via response.url to confirm your parameters were applied correctly.

import requests
import json

search_url = f"{GITHUB_ENDPOINT}/search/repositories"
query_params = {
    "q": "python devops",
    "sort": "stars",
    "order": "desc",
    "per_page": 5
}

response = requests.get(search_url, params=query_params, timeout=10)
response.raise_for_status()

print(f"Requested URL: {response.url}")
results = response.json()

print(f"Found {results.get("total_count")} repositories. Top 5:")
for repo in results.get("items", []):
    print(f"- {repo["name"]} (Stars: {repo["stargazers_count"]})")

print(json.dumps(results.get("items", [])[0], indent=2))

Making POST Requests with requests.post()

• Use requests.post() to send data to a server, choosing between data= for form-encoded bodies or json= for JSON payloads.
• Providing a dictionary to json= automatically serializes it and sets Content-Type: application/json.
• The response can be inspected similarly to GET responses, using .status_code, .json(), and error handling.

import requests
import json

post_echo_url = "https://httpbin.org/post"

payload = {
    "script_name": "devops_automation",
    "action": "trigger_deployment",
    "environment": "staging",
    "version": "v1.5.0"
}

response = requests.post(post_echo_url, json=payload, timeout=10)
response.raise_for_status()

print(json.dumps(response.json(), indent=2))

Common Pitfalls & How to Avoid Them

• Not setting timeouts can cause scripts to hang indefinitely; always include a timeout value.
• Ignoring HTTP errors means you might assume success when a request failed; use response.raise_for_status().
• Using data= instead of json= sends form-encoded data, which may be rejected by modern APIs expecting JSON.
• Hardcoding secrets in code is insecure; use environment variables (e.g., via python-dotenv) and pass them in headers.