Skip to content
elephantoo

Standard library tour

Lesson 26 of 38 14 min read

pathlib, os, shutil, subprocess, datetime and zoneinfo, json, collections, itertools, re, random and more.

Python's motto is "batteries included": the standard library ships with hundreds of modules, available everywhere Python is installed, with no pip install required. Knowing what's in the box saves you from reinventing wheels (and from adding dependencies you don't need). This lesson is a guided tour of the modules working developers reach for every week. Bookmark the official docs at docs.python.org/3/library for the full list.

pathlib: files and folders#

pathlib.Path is the modern way to work with filesystem paths — objects with methods, instead of strings and scattered functions.

Python
from pathlib import Path

project = Path("demo_project")
(project / "src").mkdir(parents=True, exist_ok=True)
(project / "src" / "app.py").write_text("print('hi')\n", encoding="utf-8")
(project / "README.md").write_text("# Demo\n", encoding="utf-8")

p = project / "src" / "app.py"
print(p, p.name, p.stem, p.suffix)
print(p.parent, p.exists(), p.is_dir())
print(p.with_suffix(".txt").name)
print(sorted(str(f) for f in project.rglob("*") if f.is_file()))
print(Path.home().is_dir(), Path.cwd().is_absolute())
Output
demo_project/src/app.py app.py app .py
demo_project/src True False
app.txt
['demo_project/README.md', 'demo_project/src/app.py']
True True

Handy methods: glob("*.csv") (one folder), rglob("*.py") (recursive), read_text/write_text, unlink() (delete a file), rename(), stat().st_size, resolve() (absolute path). On Windows the paths print with backslashes — that's the point: your code stays the same.

os and shutil: the operating system#

os gives access to environment variables, processes and lower-level file operations; shutil does high-level file operations like copying and deleting whole trees.

Python
import os
import shutil
from pathlib import Path

os.environ["APP_MODE"] = "testing"            # set (for this process and its children)
print(os.environ.get("APP_MODE"))
print(os.environ.get("DATABASE_URL", "sqlite:///dev.db"))   # default if unset
print(os.getcwd() == str(Path.cwd()))
print(os.cpu_count() >= 1)

Path("src_dir").mkdir(exist_ok=True)
Path("src_dir/a.txt").write_text("A", encoding="utf-8")
shutil.copytree("src_dir", "backup_dir", dirs_exist_ok=True)
print(sorted(os.listdir("backup_dir")))
shutil.rmtree("backup_dir")                   # delete a folder and everything in it
print(Path("backup_dir").exists())
print(shutil.which("python3") is not None)    # find a program on PATH
Output
testing
sqlite:///dev.db
True
True
['a.txt']
False
True

Reading configuration and secrets from environment variables (rather than hard-coding them) is standard practice for anything you deploy. Be careful with shutil.rmtree — there is no recycle bin.

subprocess: running other programs#

Python
import subprocess
import sys

result = subprocess.run(
    [sys.executable, "-c", "print(6 * 7)"],
    capture_output=True, text=True, check=True,
)
print(result.returncode, result.stdout.strip())
Output
0 42

Pass the command as a list of arguments (not one string with shell=True) to avoid shell-injection bugs. check=True raises CalledProcessError if the program fails.

datetime and zoneinfo: dates and times#

Python
from datetime import date, datetime, timedelta, timezone
from zoneinfo import ZoneInfo

d = date(2026, 9, 30)
print(d, d.strftime("%A %d %B %Y"), d.isoweekday())

deadline = d + timedelta(days=45)
print(deadline, (deadline - d).days)

meeting = datetime(2026, 10, 1, 9, 30, tzinfo=ZoneInfo("Asia/Kolkata"))
print(meeting.isoformat())
print(meeting.astimezone(timezone.utc).isoformat())
print(meeting.astimezone(ZoneInfo("America/New_York")).strftime("%H:%M %Z"))

parsed = datetime.fromisoformat("2026-09-30T18:45:00+05:30")
print(parsed.hour, parsed.utcoffset())
print(datetime.strptime("30/09/2026", "%d/%m/%Y").date())
Output
2026-09-30 Wednesday 30 September 2026 3
2026-11-14 45
2026-10-01T09:30:00+05:30
2026-10-01T04:00:00+00:00
00:00 EDT
18 5:30:00
2026-09-30

Rules that save you from time-zone bugs:

  • A datetime without tzinfo is naive — Python has no idea which zone it's in. Prefer aware datetimes.
  • Store and compute in UTC (datetime.now(timezone.utc)); convert to local zones only for display.
  • Use zoneinfo.ZoneInfo("Asia/Kolkata") (Python 3.9+) rather than fixed offsets — it knows about daylight saving changes. (On Windows, pip install tzdata provides the zone database.)
  • ISO 8601 strings (isoformat() / fromisoformat()) are the best format for storage and APIs. strftime/strptime handle custom formats.

json: data interchange#

Python
import json

order = {"id": 7, "items": ["pen", "ink"], "paid": True, "coupon": None}
text = json.dumps(order)
print(text)
print(json.loads(text)["items"][1])
Output
{"id": 7, "items": ["pen", "ink"], "paid": true, "coupon": null}
ink

JSON gets its own lesson next, together with CSV.

collections: specialised containers#

Python
from collections import Counter, defaultdict, deque, namedtuple, OrderedDict, ChainMap

votes = Counter(["py", "js", "py", "go", "py", "js"])
print(votes.most_common(2), votes["rust"])          # missing → 0

graph = defaultdict(list)
for a, b in [("A", "B"), ("A", "C"), ("B", "C")]:
    graph[a].append(b)
print(dict(graph))

recent = deque(maxlen=3)                             # keeps only the last 3
for page in ["home", "docs", "blog", "pricing"]:
    recent.append(page)
print(list(recent))

defaults = {"theme": "light", "lang": "en"}
user = {"theme": "dark"}
settings = ChainMap(user, defaults)                  # look up user first, then defaults
print(settings["theme"], settings["lang"])
Output
[('py', 3), ('js', 2)] 0
{'A': ['B', 'C'], 'B': ['C']}
['docs', 'blog', 'pricing']
dark en

itertools and functools#

Python
from itertools import product, combinations, permutations, chain, islice
from functools import reduce, partial

print(list(product("AB", [1, 2])))
print(list(combinations("ABC", 2)))
print(len(list(permutations(range(4)))))
print(list(chain.from_iterable([[1, 2], [3]])))

print(reduce(lambda acc, n: acc * n, [1, 2, 3, 4, 5]))
to_binary = partial(int, base=2)          # pre-fill an argument
print(to_binary("1011"))
Output
[('A', 1), ('A', 2), ('B', 1), ('B', 2)]
[('A', 'B'), ('A', 'C'), ('B', 'C')]
24
[1, 2, 3]
120
11

product replaces nested loops, combinations and permutations solve many interview puzzles, and functools gives you partial, reduce, cache and wraps.

re: regular expressions (preview)#

Python
import re

text = "Call 98765-43210 or 91234-56789"
print(re.findall(r"\d{5}-\d{5}", text))
print(re.sub(r"\d", "#", "PIN 4821"))
Output
['98765-43210', '91234-56789']
PIN ####

Regular expressions get a full lesson in the advanced section.

random, secrets, math and statistics#

Python
import math
import random
import secrets
import statistics

random.seed(42)                       # reproducible results (tests, simulations)
print(random.randint(1, 6), random.choice(["red", "green", "blue"]))
deck = list(range(10))
random.shuffle(deck)
print(deck[:3], random.sample(range(100), 3))

token = secrets.token_urlsafe(16)     # cryptographically secure
print(len(token) > 20)

print(math.gcd(48, 18), math.comb(5, 2), math.isqrt(50))
scores = [72, 85, 90, 66, 85]
print(statistics.mean(scores), statistics.median(scores), statistics.mode(scores))
print(round(statistics.stdev(scores), 2))
Output
6 red
[7, 5, 2] [4, 3, 11]
True
6 10 7
79.6 85 85
10.11

Use random for games and simulations, but secrets for passwords, tokens and anything security-related.

A few more worth knowing#

ModuleWhat it's for
sysinterpreter info: sys.argv, sys.exit, sys.path, sys.version
argparsecommand-line interfaces
loggingapplication logs (own lesson later)
csvreading/writing CSV (next lesson)
sqlite3a built-in SQL database (databases lesson)
urllib.requestbasic HTTP (most people use requests instead)
hashlibSHA-256 and other hashes
uuidunique IDs: uuid.uuid4()
tempfiletemporary files and folders
zipfile, tarfile, gziparchives and compression
textwrap, stringtext utilities
enumnamed constants: class Color(Enum): RED = 1
timeitmicro-benchmarks
concurrent.futures, threading, asyncioconcurrency (own lesson)
Python
import hashlib
import uuid
from enum import Enum


class Status(Enum):
    PENDING = "pending"
    SHIPPED = "shipped"


print(hashlib.sha256(b"elephantoo").hexdigest()[:16])
print(len(str(uuid.uuid4())))
print(Status("shipped"), Status.PENDING.value)
Output
fc3ed5b9f9754006
36
Status.SHIPPED pending

Common mistakes#

  • Reinventing the wheel — before writing a helper, check itertools, collections, pathlib and functools.
  • Naive datetimes across time zones.
  • random for security — use secrets.
  • subprocess with shell=True and user input — command injection risk.
  • Naming your files after stdlib modules (random.py, json.py), which shadows them.

What's next#

Two data formats come up in almost every job: CSV spreadsheets and JSON from APIs. Next lesson: working with CSV and JSON.

Check your understanding

Quick quiz

0/3 answered

  1. 1.Which is the modern, cross-platform way to build the path data/reports/june.csv?

  2. 2.What's the safest way to store a timestamp for an event in a web app?

  3. 3.What does collections.Counter("banana").most_common(1) return?

Finished reading?

Mark this lesson complete to track your progress.