Making HTTP Requests

• The requests library simplifies HTTP interactions by abstracting raw HTTP details, making it ideal for DevOps automation tasks.
• It allows you to query web services, trigger CI/CD builds, manage cloud resources, and integrate with APIs like GitHub in a straightforward way.
• In this notebook, we'll demonstrate installing requests, performing GET and POST requests, inspecting response data, and customizing requests with parameters and headers.
• The requests library is a third-party package and must be installed in your active virtual environment.
• Use pip install requests==2.32.2 to add it to your project (pinning the version here so that we all work with the same version, but in other projects you can omit the version to install the latest), and consider pinning its version in requirements.txt.

GITHUB_ENDPOINT = "https://api.github.com"

Making GET Requests with requests.get()

• The GET method retrieves data from a specified URL; it’s the most common HTTP request type.
• Key parameters include url, optional params for query strings, headers for custom HTTP headers, and timeout to avoid hanging requests.
• The returned Response object provides .status_code, .headers, .text, .content, and .json(), plus .raise_for_status() to handle HTTP errors.

import requests
import json

response = requests.get(GITHUB_ENDPOINT, timeout=10)

print(f"Status code: {response.status_code}")
print(f"Content-Type: {response.headers.get("Content-Type")}")

"""
# Commenting out for brevity, but leaving for documentation

print(".text attribute:")
print(response.text)
print("\n")
print(".content attribute:")
print(response.content)
print("\n")
print(".json() method:")
print(response.json())
"""

data = response.json()
print("Available endpoints:")
print(json.dumps(data, indent=2))

Passing URL Parameters with params

• Query parameters are passed as a dictionary to the params argument, and requests handles URL-encoding automatically.
• This makes it easy to filter, sort, or paginate API results without manually constructing the query string.
• You can inspect the final URL via response.url to confirm your parameters were applied correctly.

import requests
import json

search_url = f"{GITHUB_ENDPOINT}/search/repositories"
query_params = {
    "q": "python devops",
    "sort": "stars",
    "order": "desc",
    "per_page": 5
}

response = requests.get(search_url, params=query_params, timeout=10)
response.raise_for_status()

print(f"Requested URL: {response.url}")
results = response.json()

print(f"Found {results.get("total_count")} repositories. Top 5:")
for repo in results.get("items", []):
    print(f"- {repo["name"]} (Stars: {repo["stargazers_count"]})")

print(json.dumps(results.get("items", [])[0], indent=2))

Making POST Requests with requests.post()

• Use requests.post() to send data to a server, choosing between data= for form-encoded bodies or json= for JSON payloads.
• Providing a dictionary to json= automatically serializes it and sets Content-Type: application/json.
• The response can be inspected similarly to GET responses, using .status_code, .json(), and error handling.

import requests
import json

post_echo_url = "https://httpbin.org/post"

payload = {
    "script_name": "devops_automation",
    "action": "trigger_deployment",
    "environment": "staging",
    "version": "v1.5.0"
}

response = requests.post(post_echo_url, json=payload, timeout=10)
response.raise_for_status()

print(json.dumps(response.json(), indent=2))

Common Pitfalls & How to Avoid Them

• Not setting timeouts can cause scripts to hang indefinitely; always include a timeout value.
• Ignoring HTTP errors means you might assume success when a request failed; use response.raise_for_status().
• Using data= instead of json= sends form-encoded data, which may be rejected by modern APIs expecting JSON.
• Hardcoding secrets in code is insecure; use environment variables (e.g., via python-dotenv) and pass them in headers.