Python Functions Are First‑Class Citizens

• In Python, functions behave like any other object (strings, ints, lists).

• Because they are "first‑class", we can:

  • Bind them to new variable names
  • Pass them around as arguments
  • Return them from other functions
  • Stash them in data structures.

• This flexibility is the foundation for patterns such as callbacks, plugin registries, and decorators.

• Assigning Functions to Variables

• A variable can reference the function object itself, not its return value.

• Any name that points to the function can be used to call it.

• This is handy for creating aliases or late‑binding a function into another module.

def greet(name):
    print(f"Hello, {name}!")

say_hello = greet
print(say_hello is greet)
say_hello("Alice")

Passing Functions as Arguments

• Higher‑order functions accept other callables to customize behavior.
• Classic examples: sorted(key=...), event callbacks, retry helpers.
• Lets you build flexible pipelines without hard‑coding every step.

def apply_operation(operation, *operands):
    print(f"Applying {operation.__name__} to {operands}")
    return operation(*operands)

def add(*numbers):
    return sum(numbers)

def mul(*numbers):
    result = 1

    for n in numbers:
        result *= n

    return result

print(apply_operation(add, 1, 2))
print(apply_operation(mul, 1, 2, 3, 4))

Returning Functions from Functions

• A factory function can create and return a new, customized function.
• The returned function “remembers” variables from the factory’s scope: this is a closure.
• Great for building tailored validators, loggers, or API clients on the fly.

def create_api_client(auth_token):
    def api_client(endpoint, method):
        return f"Hitting endpoint {endpoint} with method {method} and auth token {auth_token}"

    return api_client

alice_api_client = create_api_client("alice-token")
bob_api_client = create_api_client("bob-token")

print(alice_api_client("/users", "GET"))
print(bob_api_client("/health", "GET"))

Storing Functions in Data Structures

• Functions can live inside lists, dicts, sets, and other containers.
• Enables command dispatch tables, plugin registries, and processing pipelines.

def task_A():
    print("Running task A")

def task_B():
    print("Running task B")

def task_C():
    print("Running task C")

pipeline = [task_B, task_A, task_C]

for task in pipeline:
    task()

command_registry = {
    "start": task_A,
    "process": task_B,
    "stop": task_C
}
command_registry["process"](http://)

Why First‑Class Functions Matter for Decorators

• Decorators are simply functions that take another function, wrap it, and return a new function.
• That entire mechanism only works because Python lets us treat functions as data.
• With this groundwork, we’re ready to explore decorator syntax (@decorator) next.
  python

Generators and Lazy Pipelines

• You can chain generator functions to form multi-stage data pipelines that process items one at a time.
• No intermediate lists are built, so memory stays low even for very large streams.
• Each generator only holds its own minimal state and passes items downstream on demand.

Memory Efficiency

• Lazy iterables maintain only minimal state (like start, stop, step) regardless of total length.
• Eager collections (lists, tuples) grow in memory usage as you add items.
• Use sys.getsizeof() to inspect the in-memory size of objects themselves (not their contents).

# 1. DONE Ingest the log lines
# 2. DONE Filter log lines based on either level or message substring
# 3. DONE Extract and return only the message attribute of the logs

import sys
import json

def read_logs(filepath):
    """Reads the contents of a file line by line.

    Args:
        filepath (str): The path where the file is located.

    Returns:
        generator (dict(str)): The json dictionary for the log line.
    """
    with open(filepath, 'r') as file:
        for line in file:
            line = line.strip()
            if not line:
                continue
            yield json.loads(line)

def filter_logs(logs, level=None, message_substring=None):
    """Filters any iterable containing dictionaries by either level or message_substring (or both)

    Args:
        logs (iterable(dict)): Iterable containing the logs to be filtered.
        level (str): The log level to keep. Defaults to None.
        message_substring (str): The pattern to look for in messages. Defaults to None.

    Returns:
        generator (dict(str)): The json dictionary for the filtered log.
    """

    for log in logs:
        if (
            level is not None
            and log.get("level", "").lower() != level.lower()
        ):
            continue

        if (
            message_substring is not None
            and message_substring.lower() not in log.get("message", "").lower()
        ):
            continue

        yield log

def extract_field(logs, field="message"):
    """Extracts a specific field from any iterable containing dictionaries.

    Args:
        logs (iterable(dict)): Iterable containing the logs to be evaluated.
        field (str): The field to return. Defaults to 'message'.

    Returns:
        generator (str): The value of the extracted field.
    """
    for log in logs:
        yield log.get(field, "").strip()

def get_first_n(logs, n=10):
    """Extracts the first n items from the provided iterable.

    Args:
        logs (iterable(T)): Iterable from which items will be extracted.
        n (int): The number of items to extract.

    Returns:
        generator (T): The item from the iterable.
    """
    count = 0

    for log in logs:
        if count >= n:
            break

        yield log
        count += 1

logs_gen = read_logs("large_logs.txt")
filter_gen = filter_logs(logs_gen, message_substring="user")
extract_gen = extract_field(filter_gen, "message")

for log in get_first_n(extract_gen, 4):
    print(log)

print("Generator object sizes (in bytes):",
      sys.getsizeof(logs_gen),
      sys.getsizeof(filter_gen),
      sys.getsizeof(extract_gen)
     )

Functions: return vs yield

• Regular functions execute immediately, run to completion, and return a single value (or None).
• Generator functions return an iterator immediately; their body runs incrementally as values are requested.
• Understanding this distinction is critical for choosing between eager and lazy workflows.

Regular Function (return) Recap

• Calling a regular function runs its entire body before returning.
• A single return exits the function and discards all local state.
• Useful when you need to compute and return a complete result at once.

def get_list_of_servers():
    print("Regular function started.")
    servers = []

    for i in range(3):
        server_name = f"server-{i}"
        print(f"\tAdding {server_name}")
        servers.append(server_name)

    print("Regular function finished.")

    return servers

servers = get_list_of_servers()
print(f"Returned list: {servers}")

Generator Function (yield) Recap

• Calling a generator function returns a generator object without running its body.
• Each yield returns one value and pauses, preserving local variables until the next request.
• Ideal for producing sequences lazily, especially when the full list is large or unbounded.

def yield_servers(count):
    print("Generator function started.")

    for i in range(count):
        server_name = f"server-{i}"
        print(f"\tYielding {server_name}")
        yield server_name

    print("Generator function finished.")

servers_gen = yield_servers(3)

for server in servers_gen:
    print(f"Server received: {server}")

Generators

• Writing a class-based iterator requires __iter__() and __next__(), plus manual state management and StopIteration handling.
• Generator functions let you express the same logic in plain Python functions, using yield to produce values one at a time.
• Any function with yield becomes a generator: calling it returns a generator object (an iterator) without running its body immediately.

def count_up_to(limit):
    """Generates numbers from 1 up to (and including) the limit.

    Args:
        limit (int): The upper limit for counting.

    Returns:
        generator(int): The generator to lazily count up to limit.
    """
    print("Generator function started...")
    n = 1

    while n <= limit:
        print(f"Yielding {n}")
        yield n
        print(f"Resumed after yielding {n}.")
        n += 1

    print("Generator function finished.")

count_gen = count_up_to(3)
print(f"Returned object: {count_gen} of type {type(count_gen)}")

print("First call to next outside of for loop.")
next(count_gen)

print("Remaining output from for loop.")
for number in count_gen:
    print(number)

Generator Functions & the yield Keyword

• A function becomes a generator by including yield; no other boilerplate is needed.
• Calling a generator function returns an object that implements __iter__() and __next__().
• The code inside runs only when iteration begins (e.g., in a for loop or via next()).

def filter_evens(data):
    """Yield only the even items from the input sequence.

    Args:
        data (iterable(int or float)): The data to iterate through and filter.

    Returns:
        generator(int or float): A generator object that yields the even items.
    """
    print("filter_evens: starting")

    for item in data:
        if item % 2 == 0:
            print(f"filter_evens: yielding {item}")
            yield item

    print("filter_evens: finished")

evens_from_range = filter_evens(range(6))

print(f"Generator object created: {evens_from_range}")

for num in evens_from_range:
    print(f"Received even: {num}")

evens_from_list = filter_evens([0, 1, 2, 3, 4, 5])

print(f"Generator object created: {evens_from_list}")

for num in evens_from_list:
    print(f"Received even: {num}")

How yield Works: Pause and Resume

• On each next() (or loop iteration), execution runs until it hits yield, returns the value, then pauses with all local state intact.
• The next next() call resumes immediately after the yield, preserving variables and the instruction pointer.
• When the function ends (no more yield), a StopIteration is raised automatically.

def demo_three_yields():
    """Demonstrate how having multiple yield statements work."""
    print("Generator started")
    yield 1
    print("Generator resumed after yielding 1.")
    yield 2
    print("Generator resumed after yielding 2.")
    yield 3
    print("Generator finished.")

demo_gen = demo_three_yields()

print(next(demo_gen))
print(next(demo_gen))
print(next(demo_gen))
# print(next(demo_gen)) # Uncommenting will raise a StopIteration Exception because there are no more yields

Generator State

• Generators keep their local variables alive between yields, making explicit state objects unnecessary.
• This persistent state allows infinite or long-running sequences without full data storage.

count_gen = count_up_to(5)

print("First call to next outside of for loop.")
print(next(count_gen))

print("Second call to next outside of for loop - now the value yielded is 2.")
print(next(count_gen))

print("Remaining output from for loop - prints from 3 onwards.")
for number in count_gen:
    print(number)

count_gen = count_up_to(5)

# Since generators have state, using the same generator object in nested loops can lead to issues.
# The inner for loop will complete the iteration, and the outer for loop will have a sinle pass.
for num in count_gen:
    for num2 in count_gen:
        print(f" - {num}:{num2}")

# The solution to this is to use distinct generator objects.
for num in count_up_to(5):
    for num2 in count_up_to(5):
        print(f" - {num}:{num2}")

Exhaustion

• Once a generator’s code path completes (falls off the end or hits return), further next() calls immediately raise StopIteration.
• A for loop over an exhausted generator does nothing on subsequent passes—you must call the function again for a fresh iterator.

count_gen = count_up_to(2)

print(next(count_gen))
print(next(count_gen))

try:
    print(next(count_gen)) # Will raise StopIteration exception
except StopIteration:
    print("Generator finished")

# Nothing will happen because the generator is already exhausted
for number in count_gen:
    print(number)

The Iteration Protocol

We use for item in sequence: all the time. But how does Python get each item?

• Iterable: An object that can be looped over. It's anything you can put on the right side of the in keyword in a for loop. Examples include lists, tuples, strings, dictionaries, sets, files, and range objects.

  • An object is considered iterable if it implements the __iter__() special method.
  • The __iter__() method returns an iterator.

• Iterator: An object that produces the next value in a sequence when asked. It "remembers" its position in the sequence.

  • An object is an iterator if it implements the __next__() special method. When there are no more items, __next__() raises the StopIteration exception.
  • Iterators normally also implement the __iter__() method, which makes them iterables too.

Example: iterable returning an iterator

class CountTo:
    def __init__(self, max_value):
        self.max = max_value

    def __iter__(self):
        # Each new for-loop call
        # gets its own iterator
        return CountToIter(self.max)

class CountToIter:
    def __init__(self, max_value):
        self.max = max_value
        self.curr = 1

    def __iter__(self):
        # Iterators are iterable
        return self

    def __next__(self):
        if self.curr <= self.max:
            val = self.curr
            self.curr += 1
            return val
        else:
            raise StopIteration

Supports nested loops: Decoupling iterables from iterators allow us to instantiate a single iterable and use it in nested loops without consuming the values from a single iterator.

my_foods = ["apple", "banana", "cherry"]

for food in my_foods:
    for food2 in my_foods:
        if food == food2:
            print(f"Skipping duplicate food: {food}")
            continue
        print(f"Cooking {food} with {food2}")

class CountTo:
    def __init__(self, max_value):
        self.max = max_value

    def __iter__(self):
        return CountToIterator(self.max)

class CountToIterator:
    def __init__(self, max_value):
        self.max = max_value
        self.current = 1

    def __iter__(self):
        return self

    def __next__(self):
        if self.current <= self.max:
            val = self.current
            self.current += 1
            return val
        else:
            raise StopIteration

counter = CountTo(5)

for count in counter:
    for count2 in counter:
        print(f"Count: {count} and {count2}")

Lambda Functions

• Python functions defined with def allow multiple statements, clear naming, and support for docstrings, making them ideal for complex or reusable logic.
• In many cases, you need a simple, single-expression function to pass directly to another function without the ceremony of a full definition.
• The lambda keyword lets you create small, anonymous functions inline, avoiding the verbosity of a def for trivial operations.
• These one-line functions are particularly handy when you want to supply custom behavior to built-in higher-order functions without polluting your namespace with one-off function names.

Syntax of a lambda Function

• A lambda function follows the exact pattern: lambda <arguments>: <expression>, where the expression result is implicitly returned.
• The lambda keyword introduces the function, <arguments> lists its parameters, and a colon separates them from the single expression body.
• You cannot include multiple statements, loops, or traditional if/else blocks: only a single expression or a ternary expression.
• Compared to a def function, a lambda is nameless and concise, making it ideal for inline usage where defining a named function would be overkill.

square = lambda x: x * x
print(square(5))

print((lambda a, b: a + b)(3, 4))

Custom Sorting with sorted()

• The built-in sorted() function accepts an optional key parameter, which should be a function that returns a comparison key for each element.
• Using a lambda for the key argument lets you define the sorting logic inline without a separate function definition.
• This approach keeps your code concise and focused, especially when the key logic is a simple attribute extraction or computation.
• When you need more complex sorting logic, you can still fall back to a named def function for clarity.

services = [("web-app", 3), ("database", 1), ("cache", 5), ("api-gateway", 2)]

print(f"Default sort: {sorted(services)}")

def get_replica_count(svc_tuple):
    return svc_tuple[1]

print(f"Sorting by replica count - standard function: {sorted(services, key=get_replica_count)}")
print(f"Sorting by replica count - lambda function: {sorted(services, key=lambda svc: svc[1])}")

Transforming Data with map()

• The map(function, iterable) built-in applies a given function to each item in an iterable, producing an iterator of results.
• Using a lambda with map lets you specify simple transformations inline without an extra function definition.
• Although list comprehensions are often preferred for readability, map with a lambda can be concise when you already need an iterator or want to emphasize the function-application nature.
• Remember that map returns a lazy iterator; convert it to a list if you need to access all results at once.

my_numbers = [1, 2, 3, 4]
print(list(map(lambda num: num * 2, my_numbers)))

ports = [80, 443, 8080, 22]
port_descriptions = list(map(lambda port: f"Port {port} is open", ports))

print(port_descriptions)

Filtering Data with filter()

• The filter(function, iterable) built-in yields only those items for which the function returns True.
• A lambda in filter is perfect for inline tests, such as checking attributes or simple conditions.
• As with map, filter returns an iterator, so wrap it in list() to evaluate immediately if needed.
• While list comprehensions can express filtering more idiomatically in modern Python, filter remains a clear demonstration of higher-order function usage.

ports = [80, 443, 8080, 22, 5432]

privileged_ports = list(filter(lambda port: port < 1024, ports))
print(privileged_ports)

privileged_comprehension = [port for port in ports if port < 1024]
print(privileged_comprehension)

Flexible Functions: *args and **kwargs

• We can use the syntax *args and **kwargs to accept a variable number of both positional and keyword arguments.

def example_function(*args, **kwargs):
    print(f"Positional args: {args}")
    print(f"Keyword args: {kwargs}")

example_function(1, 2, 3, a="Value", b=True)

*args in Definition: Collecting Positionals

• Uses *args to gather extra positional parameters into a tuple
• Allows functions to accept any number of positional inputs
• Common in utilities like custom logging or aggregation functions

def apply_operator(operator, *operands):
    """Applies operator to a variable number of operands. Supports 'add' and 'mul'.

    Args:
        operator (str): The operator to apply. Must be either 'add' or 'mul'.
        *operands (int or float): Zero or more numbers to be combined.

    Returns:
        int or float: The result of applying the operator on the operands.

    Raises:
        ValueError: Raised when operator is not 'add' nor 'mul'.
    """

    if operator == 'add':
        result = sum(operands)
    elif operator == 'mul':
        result = 1
        for n in operands:
            result *= n
    else:
        raise ValueError(f"Unknown operator {operator}. Supported values are 'add' and 'mul'")

    return result

print(apply_operator('add', 1, 2, 3, 4))
print(apply_operator('add', 1, 2, 3, 4, 5, 6, 7))
print(apply_operator('add', 1, 2))

print(apply_operator('mul', 1, 2, 3, 4))
print(apply_operator('mul', 1, 2, 3, 4, 5, 6, 7))
print(apply_operator('mul', 1, 2))

# print(apply_operator('div', 1, 2)) # Uncommenting raises ValueError since div is not supported

**kwargs in Definition: Collecting Keywords

• Uses **kwargs to gather extra named parameters into a dictionary
• Ideal for optional configuration flags or settings
• Enables functions to accept flexible keyword arguments without predefining them

def set_options(**settings):
    print(f"Received dictionary: {settings}")
    for key, value in settings.items():
        print(f"\t{key} = {value}")

set_options(timeout=30, user="admin", retries=5)

Order in Definition Matters

• Standard positional parameters must come first, some might also have a default value
• Followed by *args to catch extra positionals
• Then keyword-only parameters, some might also have a default value
• Finally **kwargs to catch extra keyword arguments

def process_request(url, method="GET", *headers, timeout, **params):
    print(f"url={url}, method={method}, timeout={timeout}")
    print(f"headers={headers}")
    print(f"params={params}")

process_request("https://www.example.com", timeout=30)
process_request("https://www.example.com", "PUT", timeout=30)
# Equivalent to call above
process_request("https://www.example.com", timeout=30, method="PUT")

process_request(
    "https://www.example.com",
    "PUT",
    "Auth: xyz",
    "Content-Type: application/json",
    timeout=30
)

process_request(
    "https://www.example.com",
    "PUT",
    "Auth: xyz",
    "Content-Type: application/json",
    timeout=30,
    retries=5,
    log_level="DEBUG"
)

* in Call: Unpacking Positional Arguments

• Uses *sequence to expand a list or tuple into positional arguments
• Sequence length must match the function’s positional parameters
• Useful for dynamic argument lists built at runtime

def connect(host, port, timeout):
    print(f"Connecting to {host}:{port} with timeout {timeout}s.")

params = ["db.internal", 5432, 10]
params_with_extra_values = ["db.internal", 5432, 10, "a", True]
connect(*params)
connect(*params_with_extra_values[:3])

** in Call: Unpacking Keyword Arguments

• Uses **dict to expand key-value pairs into keyword arguments
• Dictionary keys must match the function’s parameter names
• Common in configuration-driven function calls

def configure_service(name, version, replicas=1):
    print(f"Setting up {name} v{version} with {replicas} replicas...")

config = {"name": "auth-service", "version": "2.1.0", "replicas": 3}
configure_service(**config)

Classes and Objects

• Beyond Built-ins: Python lets you define your own data types using class.
• Class: A blueprint or template for creating objects. Defines attributes (data) and methods (behavior). Convention: PascalCase names (MyClass).
• Object (Instance): A specific item created from a class blueprint. Each object has its own set of attribute values but shares the methods defined by the class. obj1 = MyClass(), obj2 = MyClass(). obj1 and obj2 are distinct objects.

Defining a Class & __init__ (The Constructor)

• __init__(self, ...): Special method for initialization. self is always the first parameter and represents the instance itself. Other parameters receive arguments passed during object creation.
• Instance Attributes (self.x = ...): Data attached to this specific object. Created inside methods (usually __init__) using self.attribute_name = value.

class ServiceMonitor:
    """Provides service checks for a single service"""
    def __init__(self, service_name, port):
        """Initializes the monitor for a specific service.

        Args:
            service_name (str): the name of the service.
            port (int): the port to use for checks.
        """
        print(f"Initializing monitor for service {service_name} on port {port}.")
        self.service = service_name
        self.port = port
        self.is_alive = False

Creating Instances (Objects)

• Mechanism: Call the class name as if it were a function, passing any arguments required by __init__ (after self).
• Python automatically creates the object and passes it as self to __init__.

nginx_monitor = ServiceMonitor("nginx", 80)
print(isinstance(nginx_monitor, ServiceMonitor))

redis_monitor = ServiceMonitor(service_name="redis", port=6379)
print(isinstance(redis_monitor, ServiceMonitor))

print(nginx_monitor.service)
print(redis_monitor.service)

Instance Methods: Object Behavior

• Definition: Functions defined inside a class definition.
• First Parameter: Always self (by strong convention), allowing the method to access and modify the instance's attributes (self.attribute_name).
• Calling: Use dot notation on an instance: instance.method_name(arguments). Python automatically passes the instance (instance) as the self argument.

class ServiceMonitor:
    """Provides service checks for a single service"""
    def __init__(self, service_name, port):
        """Initializes the monitor for a specific service.

        Args:
            service_name (str): the name of the service.
            port (int): the port to use for checks.
        """
        print(f"Initializing monitor for service {service_name} on port {port}.")
        self.service = service_name
        self.port = port
        self.is_alive = False

    def check(self):
        """Simulates checking the service status"""
        print(f"METHOD: Checking {self.service} on port {self.port}...")
        self.is_alive = True
        print(f"METHOD: Status for service {self.service}: {"Alive" if self.is_alive else "Down"}")
        return self.is_alive

nginx_monitor = ServiceMonitor("nginx", 80)
status = nginx_monitor.check()
print(f"Received status: {status}")

Basic Inheritance: Reusing and Extending

• Concept: Create a new class (Child/Subclass) that inherits properties (attributes and methods) from an existing class (Parent/Superclass). Promotes code reuse (DRY).
• Syntax: class ChildClassName(ParentClassName):
• Inherited Members: The Child automatically gets all methods and attributes defined in the Parent.
• Specializing: The Child can:
  • Add new attributes and methods.
  • Override parent methods by defining a method with the same name.
• super(): Inside the Child's methods, use super().method_name(...) to explicitly call the Parent's version of a method (very common in __init__).

class HttpServiceMonitor(ServiceMonitor):
    """Extends ServiceMonitor to add an HTTP endpoint check."""
    def __init__(self, service_name, port, url):
        super().__init__(service_name, port)
        self.url = url

    def ping(self):
        """Ping url provided when creating instance."""
        print(f"METHOD: Pinging url {self.url}")

    def check(self):
        alive = super().check()
        print(f"METHOD: Performing HTTP check on {self.url}")

http_monitor = HttpServiceMonitor("web", 8080, "http://localhost")
nginx_monitor = ServiceMonitor("nginx", 80)

http_monitor.ping()
http_monitor.check()
# nginx_monitor.ping() # Uncommenting will raise AttributeError since ping() is a method only of the subclass
nginx_monitor.check()

Efficient Looping: range

• Creating large lists for loops is memory-intensive (e.g., list(range(1_000_000))).
• range() stores only start, stop, and step values, not all numbers.
• Numbers are generated one at a time during iteration, reducing memory usage.
• Ideal for loops needing a fixed number of iterations without large allocations.

import sys

number_count = 10_000_000

numbers_list = list(range(number_count))
numbers_range = range(number_count)

list_mb = sys.getsizeof(numbers_list) / (1024**2)
range_mb = sys.getsizeof(numbers_range) / (1024**2)

print(f"List size: {list_mb:.2f}")
print(f"Range size: {range_mb:.6f}")
print(f"List uses {(list_mb / range_mb):.2f} more memory!")

Using range()

• range(stop): iterate from 0 up to (but not including) stop.
• range(start, stop): iterate from start up to stop.
• range(start, stop, step): iterate with a custom step increment.

for i in range(5):
    print(f"Retry #{i}")

for year in range(2020, 2024):
    print(f"Processing logs for {year}")

for server_id in range(10, 30, 5):
    print(f"Checking server {server_id}")

Getting Index + Value: enumerate()

• Use enumerate(iterable, start=0) to get (index, item) tuples.
• The start parameter sets the initial index value.

servers = ["web01", "web02", "web03"]

for idx, server in enumerate(servers, 1):
    print(f"#{idx}: Processing server {server}")

Parallel Iteration: zip()

• Use zip(*iterables) to pair items from multiple iterables.
• Iteration stops when the shortest iterable is exhausted.

hosts = ["hostA", "hostB", "hostC"]
ips = ["10.0.0.1", "10.0.0.2"]
azs = ["us-east-1a", "us-east-1b"]

for host, ip, az in zip(hosts, ips, azs):
    print(f"Host: {host}, IP: {ip}, AZ: {az}")

Functions

Functions package reusable code into named blocks, improving modularity, readability, and testability. They prevent duplication (DRY) and make scripts easier to maintain.

Defining a Function (def)

Use def name(params): followed by an indented block. An optional """docstring""" explains purpose, parameters, and return value.

def greet_user(name):
    """Greets the user by name.

    Args:
        name (str): The user to greet
    """

    print(f"Hello, {name}!")

greet_user("Alice")

Calling a Function

Invoke via name(args). Control jumps into the function body and (optionally) returns a value back.

import random

def random_number(min_val, max_val):
    """Generates an integer between min_val and max_val

    Args:
        min_val (int): The lower boundary of the interval
        max_val (int): The upper boundary of the interval

    Returns:
        int: The generated random number
    """
    return random.randint(min_val, max_val)

generated_number = random_number(0, 10)
print(f"Generated number: {generated_number}")

Parameters vs Arguments

Summary: Parameters are named in the def signature; arguments are the actual values passed when calling.

generated_number = random_number(-1, 100)

Positional vs Keyword Arguments

Positional args match by order; keyword args match by name and can be out of order. Positional arguments must come first.

def check_service_status(service_name, expected_status):
    print(f"Checking {service_name} for {expected_status}...")
    return True

check_service_status("nginx", "running")
check_service_status("running", "nginx")

check_service_status(service_name="nginx", expected_status="running")
check_service_status(expected_status="running", service_name="nginx")

# Positional arguments must come before keyword arguments
# check_service_status(service_name="nginx", "running") # Uncommenting raises a SyntaxError

Default Parameter Values

It's possible to give parameters default values in the signature (param=default), making them optional.

def connect(host, port=22, timeout=30):
    print(f"Connect to host {host} on port {port} (timeout {timeout})")

connect("web01")
connect("web02", 443, 60)

# When wanting to set the value of timeout but use the default value of port
# We need to use keyword arguments, since positional arguments would be
# incorrectly mapped

# Bad exaple - see how port is set to 60 and timeout remains 30
connect("web03", 60)

# Good example - both values are set as we expect
connect("web03", timeout=60)

Docstrings – Documenting Functions

The first string in a function is its docstring, explaining purpose, Args: and Returns:. Used by help() and IDEs. Observing the following conventions is considered good practice:

1. One-line summary
2. Blank line
3. Detailed description (optional)
4. Args: section for parameters
5. Returns: section for return values
6. Raises: section for exceptions

import socket

def check_port(host, port, timeout=5):
    """Checks if a TCP port is open on a given host.

    Args:
        host (str): Hostname or IP address.
        port (int): TCP port number.
        timeout (int, option): Connection timeout in seconds. Defaults to 5.

    Returns:
        bool: True if the port is open, False otherwise.
    """

    try:
        with socket.create_connection((host, port), timeout):
            return True
    except Exception:
        return False

print(check_port("www.google.com", 443))

# Port 22 is not open, should return False
print(check_port("www.google.com", 22))

# Host does not exist, should return False
print(check_port("www.afbdoaubfdoabdfoubaf.com", 22))

Concise Iteration: List Comprehensions

Simple for loops to create lists can be verbose. We can leverage list comprehensions to define the list contents directly within square brackets, obtaining a more compact syntax.

# Example: Double items using a for loop
old_items = [1, 2, 3, 4]
doubled_items = []

for item in old_items:
    doubled_items.append(item * 2)

print(doubled_items)

# Example: Double items using list comprehension
doubled_items_with_comprehension = [item * 2 for item in old_items]
print(doubled_items_with_comprehension)

List Comprehension Syntax

• Syntax: [<expression> for <item> in <iterable>]
• [] indicates a new list is created eagerly.
• <expression> is applied to each item.
• for <item> in <iterable> defines the loop.

servers = ["web", "db", "backend"]
uppercase_servers = [server.upper() for server in servers]
print(uppercase_servers)

Filtering with if in Comprehensions

• Purpose: Include only items meeting a condition.
• Syntax: [<expression> for <item> in <iterable> if <condition>].
• The condition filters items before expression is evaluated.

numbers = [1, 5, 10, 8, 2, 15]
even_numbers = [num + 1 for num in numbers if num % 2 == 0]
print(even_numbers)

Set and Dictionary Comprehensions

• Set comprehension uses {} and produces unique items.
• Dictionary comprehension uses {key: value ...}.
• Both evaluated eagerly like list comprehensions.

numbers = [1, 2, 3, 2, 4, 1, 3]
unique_squares = {x * x for x in numbers}
print(unique_squares)

servers = ["web", "backend"]
server_ips = {server: f"192.168.1.{i}" for i, server in enumerate(servers)}
print(server_ips)

Conditional Expression (Ternary Operator)

• Purpose: Apply different expressions based on a condition within the comprehension.
• Syntax: <value_if_true> if <condition> else <value_if_false> inside the comprehension.
• Places the ternary before the for clause.

numbers = [1, 5, 10, 8, 2, 15]
categories = ["PASS" if num >= 8 else "FAIL" for num in numbers]
print(categories)

For & While Loops

Python provides two main ways to repeat actions: for loops (for iterating over known sequences) and while loops (for repeating as long as a condition is true). These are essential for automating repetitive tasks in DevOps, such as processing lists of servers, retrying operations, or polling for status changes.

Automating Repetition: Loops

• for loop: Iterates through each item in a known sequence (list, tuple, string, dictionary items, range, file lines). Best when you know the items to process.
• while loop: Repeats as long as a condition remains True. Best when the number of repetitions isn't known beforehand, but a stopping condition is.

for Loops: Processing Each Item

for loops are used to process each item in a sequence. The loop variable takes on the value of each item, one at a time, and the indented block runs for each item.

servers = ["web01", "web02", "web03"]

for server in servers:
    print("Pinging server:", server)

for char in "SUCCESS":
    print(char)

for idx in range(10):
    print("Pinging server:", idx)

while Loops: Repeating While True

while loops repeat a block of code as long as a condition remains True. This is useful when you don't know in advance how many times you'll need to repeat the action.

connection_attempts = 0
max_attempts = 5
connected = False

while not connected and connection_attempts < max_attempts:
    print(f"Attempting to reach server: {connection_attempts + 1}")
    # Simulating for the purposes of demonstration - Succeeds on 4th attempt
    if connection_attempts == 3:
        connected = True

    connection_attempts += 1

if not connected:
    print("Failed to connect after maximum attempts.")

Important: The code inside the while loop must eventually make the condition False (e.g., by incrementing a counter or changing a flag), or you'll create an infinite loop.

Controlling Loop Flow: break and continue

• break: Immediately exits the innermost loop. Useful when you've found what you need or hit an error.
• continue: Skips the rest of the current iteration and moves to the next one. Useful for skipping items that don't meet criteria.

users = ["guest", "tester", "admin01", "admin02", "dev01"]
found_admin = None

for user in users:
    print(f"Checking user: {user}")
    if user.startswith("admin"):
        found_admin = user
        print(f"Admin user found: {found_admin}. Stopping search.")
        break

filenames = ["nginx.conf", "app.yaml", "db.yaml", "notes.txt"]

for file in filenames:
    if not file.endswith(".yaml"):
        print(f"Skipping non-yaml file: {file}") 
        continue
    print(f"Processing YAML config: {file}")

If / Elif / Else Logic

Control the flow of scripts based on conditions using if, elif, and else.

The if Statement

An if statement executes a block of code only if a condition is True.

• Syntax: if <condition>: followed by an indented block
• Comparison operators: ==, !=, <, >, <=, >=, in
• Combine conditions with and, or, not

server_status = "running"

if server_status == "running":
    print("Service is active.")

Truthiness

Python treats many values as truthy or falsy in conditionals.

• Falsy: False, None, 0, 0.0, '', [], {}
• Truthy: non-zero numbers, non-empty sequences/collections

servers = ["web01", "web02"]
error_message = ""
default_config = {}

if servers:
    print(f"Processing {len(servers)} servers.")
if error_message:
    print("Something went wrong:", error_message)
if not default_config:
    print("Default config not available, please provide the configuration values.")

The else statement

Use else to execute code when the if condition is false.

cpu_usage = 85.0

if cpu_usage > 90.0:
    print("ALERT: High CPU Usage")
else:
    print("CPU Usage is normal.")

The elif statement

Chain multiple checks; the first true block runs.

http_status = 503

if http_status == 200:
    print("Status OK")
elif http_status == 404:
    print("Resource not found")
elif http_status >= 500:
    print("Server error (5xx)")
else:
    print("Another status:", http_status)

Guard Clauses

Handle edge cases at the top of functions to avoid deep nesting of if conditions.

def process_data_guarded(data):
    if not data:
        print("No data provided")
    elif not isinstance(data, list):
        print(f"Invalid value type for 'data'. Provided {type(data)}; Required: list")
    else:
        print(f"Processing {len(data)} items...")
        print("Processed")

process_data_guarded(None)
process_data_guarded([])
process_data_guarded("abc")
process_data_guarded(10)
process_data_guarded([1, 2, 3])

Dictionaries (dict)

Dictionaries are mutable, insertion-ordered collections of key-value pairs. Keys must be unique and immutable; values can be of any type.

Characteristics and Use Cases

• Insertion-ordered (Python 3.7+)
• Mutable: add, remove, or change key-value pairs
• Fast lookups by key
• Ideal for configuration data, JSON-like structures, and lookups

Dictionary Operations Overview

Dictionaries in Python support a variety of operations for efficient data manipulation:

• Length: Use len(my_dictionary) to get the number of key-value pairs.
• Accessing Keys, Values, and Items: Use my_dictionary.keys(), my_dictionary.values(), and my_dictionary.items() to retrieve keys, values, or key-value pairs.
• Membership Test: Check if a key exists using 'key' in my_dictionary.
• Get with Default: Use my_dictionary.get('key', default) to safely retrieve a value with a fallback.
• Setdefault: Add a key with a default value if it doesn't exist using my_dictionary.setdefault(key, default).
• Pop and Popitem: Remove a specific key with my_dictionary.pop(key) or remove an arbitrary key-value pair with my_dictionary.popitem().
• Merging: Combine dictionaries using the | operator (Python 3.9+) or update() method.
• Fromkeys: Create a new dictionary with specified keys and a default value using dict.fromkeys(keys, value).
• Clear: Remove all items from the dictionary with my_dictionary.clear().

my_dictionary = {'a': 1, 'b': 2, 'c': 3}
print(my_dictionary)

print(f"Length: {len(my_dictionary)}")

# Keys, Values, and Items
print(f"Keys: {my_dictionary.keys()}")
print(f"Values: {my_dictionary.values()}")
print(f"Items: {my_dictionary.items()}")

for item in my_dictionary.items():
    print(type(item))

for key, value in my_dictionary.items():
    print(f"- {key}: {value}")

# Membership test
print(f"'b' is in my_dictionary? {"b" in my_dictionary}")
print(f"'d' is in my_dictionary? {"d" in my_dictionary}")
print(f"1 is in my_dictionary? {1 in my_dictionary}")
print(f"1 is in values of my_dictionary? {1 in set(my_dictionary.values())}")

# Accessing elements
print("'b':", my_dictionary["b"]) # Will raise KeyError if key is not present in the dictionary
print("'b':", my_dictionary.get("b")) # Will not raise KeyError
print("'e' without default:", my_dictionary.get("e"))
print("'e' with default:", my_dictionary.get("e", -1))

my_dictionary.setdefault("d", 4)
print(my_dictionary)

# Removing elements
removed = my_dictionary.pop("a")
print(f"Removed value: {removed}")
removed = my_dictionary.popitem()
print(f"Removed value: {removed}")
removed = my_dictionary.popitem()
print(f"Removed value: {removed}")

# Merging of dictionaries
default_tags = {
    "Environment": "Production",
    "Owner": "Finance",
    "CostCenter": "10000"
}

custom_tags = {
    "CostCenter": "12345"
}

merged_tags = default_tags | custom_tags
print(merged_tags)
default_tags.update(custom_tags)
print(default_tags)

# Creating new dictionary based on a set of keys
new_dict = dict.fromkeys(['one', 'two', 'one'], 0)
print(new_dict)

new_dict.clear()
print(new_dict)

Adding and Updating Items

• server_config['port'] = 8080 # Update existing key
• server_config['environment'] = 'production' # Add new key-value pair

tags = {
    "Environment": "Production",
    "Owner": "Finance",
    "CostCenter": "10000"
}

tags["CostCenter"] = "12345"
tags["Project"] = "Python for DevOps"

print(tags)

Tuples (tuple)

Tuples are ordered, immutable sequences defined with parentheses (). Once created, their contents cannot be changed.

Characteristics and Use Cases

• Ordered: items maintain position
• Immutable: cannot add, remove, or change after creation
• Useful for fixed records like coordinates, version numbers, or as dictionary keys

host_port = ("127.0.0.1", 3000)
red_rgb = (255, 0, 0)
tuple_single_value = ("only-value",) # To create a single-item tuple, add a trailing comma
print(type(host_port))
print(type(tuple_single_value))

print(f"Host: {host_port[0]}")
print(red_rgb[-2:])
print(type(red_rgb[-2:]))

# host_port[0] = "192.168.1.1" # Uncommenting will raise a TypeError because tuples are immutable

Sets (set)

• Characteristics: Unordered, Mutable, Unique items only (duplicates removed)
  • The items of a set must be immutable.
• Use Cases: Membership testing, removing duplicates, set operations (union, intersection, difference).

Set Operations

• Membership Testing: Check if an item exists in a set using the in keyword.
• Adding Items: Use add() to add an item to a set.
• Removing Items: Use remove() to remove an item (raises an error if the item doesn't exist) or discard() to remove an item (doesn't raise an error if the item doesn't exist).
• Set Operations:
  • Union: Combine all unique items from two sets using union() or |.
  • Intersection: Find common items between two sets using intersection() or &.
  • Difference: Find items in one set but not in another using difference() or -.

unique_ports = set([80, 443, 22, 80, 8080, 443])
server_names = {"web01", "web02"}

print(unique_ports)
print(22 in unique_ports)
print(22 in server_names)

unique_ports.add(3000)
print(unique_ports)
unique_ports.remove(22)
print(unique_ports)
# unique_ports.remove(22) # Will raise KeyError because item 22 is not in the set anymore
unique_ports.discard(22)
print(unique_ports)

# set_of_lists = set([[1, 2], [3, 4]]) # Will throw a TypeError, since lists are mutable
# set_of_sets = {{1, 2}, {3, 4}} # Will throw a TypeError, since sets are mutable
set_of_tuples = {(1, 2), (3, 4)}
print(set_of_tuples)
print((1, 2) in set_of_tuples)
print((1, 3) in set_of_tuples)

Lists (list)

Lists are ordered, mutable sequences defined with square brackets []. You can add, remove, or change items after creation.

Characteristics and Use Cases

• Ordered: items maintain position
• Mutable: .append(), .insert(), .pop(), .remove()
• Ideal for storing sequences where order matters and contents change (e.g., list of servers, deployment steps)

Accessing Items and Slicing

• Access single elements with my_list[index] (0-based). Use negative indices like my_list[-1] for the last item.
• Slice with my_list[start:stop] to get a sub-list from start up to (but not including) stop.
• Use three-parameter slicing my_list[start:stop:step] for stepping, e.g., my_list[::2] selects every other element.
• Omitting start or stop defaults to the beginning or end of the list respectively, and slicing returns a new list without modifying the original.

servers = ["web01", "web02", "web03"]
mixed_list = ["config.yaml", 8080, True]

for item in mixed_list:
    print(type(item))

print(servers[0])
# print(servers[3]) # Commenting this out will raise an IndexError Exception
print(servers[-1])
print(servers[-2])

# Slicing
print(servers[:2]) # Will print only elements at indexes 0 and 1
print(servers[1:]) # Will print only elements at indexes 1 and 2
print(servers[-2:]) # Will print only the second to last and last elements
# Slicing does not alter the original list
print(servers)

# Mutating lists
ports = [80, 443, 8080]
ports.append(5000)
print(ports)
ports.insert(1, 3000)
print(ports)
ports.remove(80)
print(ports)
removed_value = ports.pop(2)
print(ports)
print(removed_value)

# Example to show how mutating lists can lead to side-effects outside of
# the scope of the code that modifies the list.
def mutate_list(l):
    l.pop()

new_list = ["a", "b", "c"]
mutate_list(new_list)
print(new_list)

Numbers (int and float)

• int: Whole numbers (e.g., 10, 1024). No overflow due to arbitrary precision.
• float: Numbers with decimals (e.g., 3.14159). Uses IEEE 754 representation; small precision differences possible.

import math

print(type(1.0))

print("When comparing floats directly, we may run into precision issues:")
print("0.1 * 3 == 0.3: ", 0.1 * 3 == 0.3)
print("To tackle this, we can use the math.isclose() function:")
print("math.isclose(0.1 * 3, 0.3): ", math.isclose(0.1 * 3, 0.3))

Arithmetic Operations

• +, -, *, /
• / true division → float
• // floor division → integer or float
• % modulo → remainder
• ** → power

print(8/2)
print(type(8/2))
print(5/3)
print(8//2)
print(type(8//2))
print(5//3)
print(5//3.0)
print(5%3) # 1 as the result, and 2 remaining

Strings

String Manipulation

• Strings are ordered, immutable sequences of characters.
• Use single or double quotes consistently; triple quotes for multi-line strings or docstrings.

single_line_str = "Double quoted"
single_line_str2 = 'Single quoted'

command_template = """
I will not be indented
    I will be indented
"""

print(command_template)

Format Output using f-string

Tip: f-strings allow inline expression evaluation and formatting, making string construction concise and readable.

math_division = 7/2
print(f"Result: {math_division}")
print(f"Result: {7/2}")

Common Operations and Essential String Methods

• Concatenation (+): Joins strings.
• Length (len()): Gets the number of characters.
• Indexing ([]): Access a character by position (0-based).
• Slicing ([:]): Extract substrings.
• .lower() / .upper()
• .strip() / .lstrip() / .rstrip()
• .startswith() / .endswith()
• .split() / .join()
• .replace()

course_title = "     Python strings    "
print(course_title)
print(f"Result of .strip(): {course_title.strip()}")
print(f"Result of .lstrip(): {course_title.lstrip()}")
print(f"Result of .rstrip(): {course_title.rstrip()}")
print(f"Result of .upper(): {course_title.upper()}")
print(f"Result of .lower(): {course_title.lower()}")

filename = "file.yaml"
print(filename.startswith("file"))
print(filename.endswith("yaml"))

path = "/usr/local/bin"
path_parts = path.split("/")
print(f"path parts: {path_parts}")
print(f"joined path paths: {"\\".join(path_parts)}")

print(path + "/python")
print(len(path))
print(path[3])
print(path[3:10])
print(path[3:])
print(path[:10])

String Immutability

Strings are immutable, meaning you cannot change a string in place; operations that seem to modify a string actually create and return a new string object.

course_title = "     Python strings    "
print(course_title)
print(f"Result of .strip(): {course_title.strip()}")
print(f"Result of .lstrip(): {course_title.lstrip()}")
print(f"Result of .rstrip(): {course_title.rstrip()}")
print(f"Result of .upper(): {course_title.upper()}")
print(f"Result of .lower(): {course_title.lower()}")
print(course_title)

Variables: Naming Values

• Naming Guidelines:
  • Must start with a letter or underscore (_) and can contain letters, numbers, and underscores.
  • Use snake_case for readability (e.g., max_retries).
• Purpose: Store data like file paths, server counts, status messages, API keys, configurations.
• Typing: Python uses dynamic typing, which means we don't need to explicitly declare the variable type, and we can assign values with different types to the same variable (not recommended!)

var1 = "hello"

item = 101
print(type(item))

# Never do that! Don't assign a value of a different type to the same variable!
item = "Code 101"
print(type(item))

Comments

Python code may be readable, but comments and docstrings explain intent, rationale, and usage. Comments (#) are ignored by the interpreter; docstrings ("""...""") are accessible via __doc__ (we'll come back to docstrings later, when we discuss functions).

Single-Line Comments (#)

Use # to comment single lines or inline code. Best for explaining why, adding TODO/FIXME markers, or temporarily disabling code.

# Example of a single-line comment
error_code = 0

# TODO: handle case when argument is None 

Multi-Line / Block Comments

Prefix each line with # to comment out blocks of code. Useful for disabling sections or annotating complex logic. It's also possible to wrap multiline comments between triple single-quotes ('''...''') or between triple double-quotes ("""..."""), but this is not their original intended usage.

# if True:
#     print("I will execute")