This roadmap is about Python Developer
Python Developer roadmap starts from here
Advanced Python Developer Roadmap Topics
By Sharvan K.
8 years of experience
My name is Sharvan K. and I have over 8 years of experience in the tech industry. I specialize in the following technologies: Python, CSS 3, React, node.js, MongoDB, etc.. I hold a degree in Bachelor of Technology (B.Tech.). Some of the notable projects I’ve worked on include: DappBnd, AI Chatbot with OpenAI API and Chat History in React, Effortless Order Foods, 🌐 TeleMed2U – Telemedicine Services for Health Plans, MikeToken.IO – Full-Stack Web3 Meme Token Platform, etc.. I am based in jalore, India. I've successfully completed 7 projects while developing at Softaims.
I am a business-driven professional; my technical decisions are consistently guided by the principle of maximizing business value and achieving measurable ROI for the client. I view technical expertise as a tool for creating competitive advantages and solving commercial problems, not just as a technical exercise.
I actively participate in defining key performance indicators (KPIs) and ensuring that the features I build directly contribute to improving those metrics. My commitment to Softaims is to deliver solutions that are not only technically excellent but also strategically impactful.
I maintain a strong focus on the end-goal: delivering a product that solves a genuine market need. I am committed to a development cycle that is fast, focused, and aligned with the ultimate success of the client's business.
key benefits of following our Python Developer Roadmap to accelerate your learning journey.
The Python Developer Roadmap guides you through essential topics, from basics to advanced concepts.
It provides practical knowledge to enhance your Python Developer skills and application-building ability.
The Python Developer Roadmap prepares you to build scalable, maintainable Python Developer applications.

What this section is about Python fundamentals are the habits and mental models you need before libraries and frameworks matter: how the interpreter runs code, how you install tool
Python fundamentals are the habits and mental models you need before libraries and frameworks matter: how the interpreter runs code, how you install tools safely, how names refer to objects, and how indentation shapes program structure.
Treat this block as your launchpad. The goal is not memorizing every builtin, but building reliable intuition so later topics like data structures, files, and errors feel predictable instead of magical.
You will move from environment setup into executable scripts and the REPL, then practice assigning values, inspecting types, and expressing logic with clear, readable syntax. Each topic reinforces the last: installation feeds into running code; variables feed into expressions and control flow.
As you study, keep a small scratch file open and retype examples by hand. Python rewards consistency: same indentation level means same block, and explicit is better than implicit when you are learning.
# python --version
python3 --version
Installing and running Python Installation is how you get a supported Python interpreter, the standard library, and pip so you can run scripts and install third-party packages.
Installation is how you get a supported Python interpreter, the standard library, and pip so you can run scripts and install third-party packages. On many systems you will use python3 explicitly because python may still point to an older release.
Virtual environments isolate dependencies per project so upgrades on one app do not break another. That discipline becomes essential once you collaborate or deploy.
Download a current CPython build from python.org or use your OS package manager, then verify python3 --version. Create a project folder, run python3 -m venv .venv, and activate it so pip installs land inside the environment instead of globally.
After activation, pip install requests (for example) only affects that venv. Commit requirements.txt or a lockfile strategy your team agrees on, and document the exact Python minor version your project expects.
python3 -m venv .venv
source .venv/bin/activate # Windows: .venv\Scripts\activate
python -m pip install --upgrade pip
Variables, objects, and built-in types In Python, a variable name is a label bound to an object in memory.
In Python, a variable name is a label bound to an object in memory. Types describe what you can do with a value: numbers support arithmetic, sequences support indexing, and mappings associate keys with values.
Dynamic typing means the name can refer to different object types over time, but each object still has a concrete type at runtime. Learning the core builtins early prevents confusion when you meet custom classes later.
You bind names with assignment (=), inspect types with type(x), and compare identity with is when you care about the exact object, not just equal values. Prefer snake_case for variable names to match community style (PEP 8).
Start with int, float, bool, str, list, tuple, dict, and set. Notice which are mutable (lists, dicts, sets) versus immutable (int-like small cases aside, str, tuple) because that affects aliasing bugs and dictionary keys.
count = 7
pi = 3.14159
active = True
label = "Softaims"
coords = (10, 20)
scores = {"ada": 98, "lin": 91}
Basic Python syntax you will use daily Syntax is the set of rules that make your file a valid program: comments, line structure, indentation for blocks, calls, imports, and simple
Syntax is the set of rules that make your file a valid program: comments, line structure, indentation for blocks, calls, imports, and simple expressions. Python trades braces for indentation, which enforces readable layout by default.
Imports pull names from modules; expressions compute values; statements perform actions. Getting comfortable with this skeleton lets you read real codebases without getting lost in punctuation.
Use # for comments, separate logical sections with blank lines, and keep indentation consistent (four spaces is standard). Import standard library modules first, then third party, then local packages, as teams often enforce with formatters or linters.
Experiment in the REPL (python -i or python) for one-liners, but move multi-step work into .py files so you can rerun, diff, and review. print() is fine for learning; logging wins in production code.
import math
radius = 5
area = math.pi * radius ** 2
print(f"Area: {area:.2f}")
Operators and expressions Operators are symbols and keywords that combine values: arithmetic (+, -, *, /, //, %, **), comparisons (==, !
Operators are symbols and keywords that combine values: arithmetic (+, -, *, /, //, %, **), comparisons (==, !=, <, <=, >, >=), membership (in, not in), identity (is, is not), and logical (and, or, not). Expressions are combinations that evaluate to a single object.
Understanding precedence and associativity keeps bugs out of conditionals and math. When in doubt, add parentheses—readability beats clever one-liners while you are learning.
Python evaluates sub-expressions, applies operators by precedence (e.g. ** before *, / before +), and short-circuits and / or once the result is known. The walrus operator := (3.8+) assigns inside an expression when you need a value and a name in one step.
Use // for floor division, % for remainder, and ** for powers. Chain comparisons like 0 <= x < 10 for readable ranges. Prefer is and is not for None checks, not ==.
a = 2 + 3 * 4
b = 10 // 3
ok = a == 14 and b == 3
chained = 1 <= len("py") <= 10
Type conversions and constructors Often you need to turn user input, JSON, or API payloads into the right Python types.
Often you need to turn user input, JSON, or API payloads into the right Python types. Builtin callables such as int(), float(), str(), bool(), list(), tuple(), and dict() construct or coerce values, sometimes with strict rules when input is invalid.
Conversion is not the same as parsing arbitrary text: int("42") works; int("4.2") raises ValueError. Learning which constructors accept which inputs saves hours of debugging forms and config files.
Call int(x, base) for strings in bases 2–36; float() and str() cover most numeric and display paths. bool() maps empty collections and zero to False; non-empty and non-zero to True.
For structured text use json.loads; for loose user input strip whitespace first, catch ValueError, and show friendly errors. round() rounds floats; it is not a type converter but pairs with numeric work.
age_str = "27"
age = int(age_str)
pi = float("3.14")
label = str(42)
active = bool(1)
items = list({1, 2, 2})
What this section is about Control flow decides which code runs and how often: branching with conditionals, repeating work with loops, and organizing logic into reusable functions.
Control flow decides which code runs and how often: branching with conditionals, repeating work with loops, and organizing logic into reusable functions. Functions are the primary tool for naming behavior, hiding detail, and testing small pieces in isolation.
This section connects syntax to structure. You will see how Python evaluates conditions, iterates over sequences and iterators, and passes data into callable objects including small anonymous functions.
Read each construct as a pattern: if handles decisions, for walks known iteration, while guards repetition with a predicate, def introduces a function with a local namespace, and lambda creates a lightweight function expression for short callbacks.
Practice by rewriting nested logic into smaller functions, naming intermediate results, and returning data instead of mutating globals. That habit scales directly into classes, decorators, and async code later.
def clamp(value, low, high):
if value < low:
return low
if value > high:
return high
return value
What is a conditional statement? Conditional statements let programs choose different paths based on Boolean tests.
Conditional statements let programs choose different paths based on Boolean tests. Python supports if, elif, and else chains, and expressions such as conditional expressions (value_if_true if test else value_if_false) for simple picks.
Truthiness means non-empty collections and non-zero numbers act like True in Boolean context, while None, zero, and empty containers act like False. Explicit comparisons (is None, ==) keep code obvious.
Python evaluates tests top to first match in an if / elif chain, then runs the indented suite for that branch only. Short-circuit operators and and or can combine predicates but should stay readable.
Prefer positive conditions and shallow nesting; extract complex predicates into named booleans or helper functions. Match / case (structural pattern matching, PEP 634) appears in 3.10+ for richer branching when you need it.
score = 88
if score >= 90:
grade = "A"
elif score >= 80:
grade = "B"
else:
grade = "C"
What is a for loop? A for loop in Python iterates over any iterable: the loop variable takes each item produced by the iterator protocol.
A for loop in Python iterates over any iterable: the loop variable takes each item produced by the iterator protocol. This is how you walk lists, characters in strings, keys in dicts (with .items() for pairs), and custom generators.
Unlike indexed C-style loops, Python encourages direct iteration. When you need an index anyway, enumerate() pairs values with positions without manual range(len(...)) everywhere.
Each iteration assigns the next value to the loop variable and runs the loop body. You can break to exit early, continue to skip to the next item, and else on a for loop runs if the loop completed without break (handy for search patterns).
range(stop) or range(start, stop, step) produces a memory-efficient sequence of integers for numeric iteration. Combine with zip to walk multiple iterables in parallel.
names = ["ada", "lin", "sam"]
for name in names:
print(name.upper())
for i, ch in enumerate("py"):
print(i, ch)
What is a while loop? A while loop repeats a block while its condition stays true.
A while loop repeats a block while its condition stays true. It is ideal when the number of iterations is not known upfront, such as consuming input until a sentinel or retrying until a service responds.
Because the condition is re-evaluated every iteration, ensure something inside the loop eventually makes the condition false, or you risk an infinite loop (sometimes intentional with break).
Python evaluates the Boolean test before each iteration; if it starts false, the body never runs. Use break to exit immediately and continue to jump back to the test.
For interactive menus or parsers, combine while with try/except for robustness, and consider moving complex loop bodies into functions so the loop reads as a high-level process.
n = 3
while n > 0:
print(n)
n -= 1
What is a Python function? A function is a named block you can call with arguments. It packages logic, documents intent, and enables reuse and testing.
A function is a named block you can call with arguments. It packages logic, documents intent, and enables reuse and testing. The def statement binds the function object to a name in the current scope.
Functions can take positional and keyword arguments, accept arbitrary *args and **kwargs, and annotate parameters for clarity (annotations are optional metadata consumed by tools).
Calling a function creates a new frame: parameters are local names, the body runs, and return sends a value back to the caller (or None implicitly). Docstrings immediately after def describe behavior for help() and Sphinx-style docs.
Pure functions that avoid mutating globals are easier to reason about. When you need side effects (I/O, logging), keep them at the edges and let the core return data structures.
def greet(name: str, excited: bool = False) -> str:
msg = f"Hello, {name}"
return msg + "!" if excited else msg
print(greet("Softaims", excited=True))
Arguments, parameters, and return values Parameters are names in the function signature; arguments are the values supplied at call time.
Parameters are names in the function signature; arguments are the values supplied at call time. Python matches positional arguments left to right, then applies keyword arguments by name, enforcing that required parameters receive values.
Return values are the function’s output object. Multiple results are usually packed in a tuple or a small dataclass; returning None is explicit or implicit when no return runs.
Default parameter values are evaluated at function definition time, so avoid mutable defaults like def append_item(x, bucket=[]): use None and assign inside the body instead. Use * to capture overflow positional args and ** for overflow keywords.
Unpacking at the call site with *iterable and **mapping forwards arguments cleanly. This pattern appears everywhere in wrappers and decorators.
def move(x, y, *, speed=1):
return (x * speed, y * speed)
point = {"x": 2, "y": 3}
print(move(**point, speed=2))
What is variable scope in Python? Scope determines where a name is visible without qualification.
Scope determines where a name is visible without qualification. Python uses lexical scoping: functions see names in enclosing defs and the module globals, with the builtin namespace as a final fallback.
The LEGB rule orders lookups: Local, Enclosing, Global, Builtins. Understanding this prevents accidental shadowing and explains why a loop variable still exists after the loop in Python 3.
Assignment to a name makes it local unless declared global or nonlocal. Use global sparingly for module-level singletons; prefer return values and parameters for clarity.
Closures capture variables from enclosing scopes by name, which matters when defining nested functions or decorators. nonlocal updates an outer function’s binding from an inner function.
count = 0
def outer():
count = 10
def inner():
nonlocal count
count += 1
return count
return inner
fn = outer()
print(fn(), fn())
What is a lambda function? A lambda expression creates a small anonymous function in line: lambda parameters: expression.
A lambda expression creates a small anonymous function in line: lambda parameters: expression. It is limited to a single expression body, so complex logic belongs in a normal def with a name.
Lambdas shine as short callbacks passed to sorted, map, filter, and GUI event handlers where brevity helps and the behavior is obvious.
Lambdas close over variables like nested defs; beware late binding in loops—capture loop variables in default parameters if you need fixed values per iteration.
Prefer named functions when stack traces should read clearly or the logic spans multiple steps. PEP 8 discourages assigning lambdas to variables; use def instead.
pairs = [("ada", 10), ("lin", 7), ("sam", 12)]
pairs.sort(key=lambda item: item[1])
print(pairs)
What this section is about Data structures organize values for access patterns you care about: fast membership, ordered sequences, key lookup, or uniqueness.
Data structures organize values for access patterns you care about: fast membership, ordered sequences, key lookup, or uniqueness. Python’s builtins cover most day-to-day needs before you reach specialized libraries.
Choosing the right structure affects readability and performance. Lists keep order and allow duplicates; tuples are immutable ordered records; dicts map keys to values; sets store unique hashable elements.
Study each type’s time complexity for typical operations (append, membership, key lookup) at a high level, then practice composing them: lists of dicts, dicts of sets, and so on.
Strings are sequences of Unicode code points with rich formatting methods; they are immutable like tuples. Slicing, concatenation, and f-strings appear constantly in real code.
from collections import Counter
words = ["python", "data", "python", "flow"]
print(Counter(words))
What is a Python list? A list is a mutable ordered sequence. You index, slice, append, extend, insert, pop, and sort in place.
A list is a mutable ordered sequence. You index, slice, append, extend, insert, pop, and sort in place. Lists can hold heterogeneous items, though homogeneous lists are easier to maintain.
List comprehensions and generator expressions build lists lazily or eagerly with concise syntax, replacing many manual loops.
Negative indices count from the end; slicing creates a shallow copy when you omit both bounds [:]. sort() orders in place; sorted() returns a new list.
Watch for aliasing: two names can refer to the same list object, so in-place methods affect both. Copy with list.copy(), slicing [:], or copy.deepcopy for nested structures.
nums = [3, 1, 4, 1, 5]
nums.append(9)
nums.sort()
squares = [n * n for n in nums if n % 2 == 1]
print(squares)
What is a tuple? A tuple is an immutable ordered sequence. Once created, you cannot assign to its elements (though mutable objects inside can still change).
A tuple is an immutable ordered sequence. Once created, you cannot assign to its elements (though mutable objects inside can still change). Tuples are ideal for fixed records like coordinates or rows from simple queries.
The comma forms tuples, not parentheses: 1, 2 is a tuple; (1) is just an int unless you write (1,).
Immutability lets tuples be dict keys and set members when all elements are hashable. Named tuples (collections.namedtuple or typing.NamedTuple) add field names for readability.
Unpacking tuples into variables (a, b = point) and starred unpacking (*rest) appear in function signatures and parallel assignment.
point = (10, 20)
x, y = point
records = {(42, "draft"): "pending"}
print(records[(42, "draft")])
What is a dictionary? A dict maps hashable keys to arbitrary values with average O(1) lookup. Insertion order is preserved in Python 3.
A dict maps hashable keys to arbitrary values with average O(1) lookup. Insertion order is preserved in Python 3.7+ as an implementation detail made official, which makes dicts pleasant for JSON-like data.
Common patterns include counting with .get(key, 0), setdefault, and collections.Counter or defaultdict for specialized defaults.
Iteration yields keys by default; .items() yields (key, value) pairs. Dict comprehensions build mappings concisely: {k: v for k, v in pairs}.
Avoid mutating a dict while iterating it; instead collect keys to change or rebuild a new dict. For nested structures, copy.deepcopy or explicit reconstruction prevents shared-reference bugs.
user = {"name": "Ada", "role": "engineer"}
user["email"] = "[email protected]"
for key, value in user.items():
print(f"{key}: {value}")
What is a set? A set stores unique hashable elements with fast membership testing.
A set stores unique hashable elements with fast membership testing. Sets support union, intersection, difference, and symmetric difference operations that mirror mathematical sets.
Use sets to deduplicate, test overlap between collections, or track visited items in graph-like problems.
Literal syntax uses braces with elements: {1, 2, 3}. An empty set is set(), not {}, which builds a dict. frozenset is an immutable variant usable as dict keys.
Set comprehensions {x for x in items if cond} mirror list comprehensions. Remember that elements must be hashable; lists cannot live inside sets.
a = {1, 2, 3, 3}
b = {3, 4, 5}
print(a | b, a & b, a - b)
What is a Python string? Strings are immutable sequences of Unicode code points.
Strings are immutable sequences of Unicode code points. You concatenate, slice, split, strip, and format with f-strings (formatted string literals, PEP 498) for readable interpolation.
Encoding matters at I/O boundaries: Python 3 strings are Unicode; bytes hold raw encoded data. Encode with .encode() and decode with .decode() when reading files or sockets.
Methods like str.lower, str.startswith, and str.replace return new strings rather than mutating. str.join(iterable) efficiently concatenates many pieces compared to repeated + in loops.
Regular expressions live in the re module for pattern matching beyond simple methods. For complex templates, consider Template strings or dedicated libraries.
title = " python roadmap "
slug = "-".join(title.split()).lower()
detail = f"Topic: {slug!r}"
print(detail)
What this section is about Files and exceptions are how programs interact with the outside world safely.
Files and exceptions are how programs interact with the outside world safely. Reading and writing data persists results; structured error handling keeps failures visible and recoverable instead of crashing silently.
Python emphasizes exceptions for error reporting rather than special return codes everywhere. Context managers (with) ensure files and locks clean up even when errors occur.
You will open files in text or binary mode, iterate line by line, and prefer pathlib for portable paths. Pair I/O with try/except/finally or with blocks to handle missing files, permission errors, and encoding issues.
Learn which exceptions are common, when to re-raise, and how assert documents invariants in development without replacing validation for users.
from pathlib import Path
path = Path("notes.txt")
text = path.read_text(encoding="utf-8")
print(text[:80])
Reading and writing files File I/O moves data between memory and storage.
File I/O moves data between memory and storage. Text mode decodes bytes to str with an encoding (UTF-8 by default in many contexts); binary mode handles bytes verbatim for images, pickles you trust, or network payloads.
For large files, iterate instead of read()ing everything at once to limit memory use. pathlib.Path offers high-level helpers like read_text and write_text for small files.
open(path, mode, encoding="utf-8") returns a file object; close it or use with to release OS handles. Modes include r, w, a, x, and + variants; add b for binary.
Newline handling differs by platform; text mode translates by default. When parsing CSV or JSON, prefer the csv and json modules over ad-hoc splitting.
with open("log.txt", "w", encoding="utf-8") as fh:
fh.write("startup\n")
with open("log.txt", "r", encoding="utf-8") as fh:
for line in fh:
print(line.strip())
What is the with statement? The with statement invokes a context manager’s __enter__ before the block and __exit__ afterward, even if the block raises.
The with statement invokes a context manager’s __enter__ before the block and __exit__ afterward, even if the block raises. Files, locks, and database connections commonly use this pattern for deterministic cleanup.
contextlib in the standard library offers helpers like contextmanager decorator and closing() for objects with close() but not full manager protocol.
Multiple managers can appear in one with line, separated by commas or parentheses, unwinding in reverse order of completion. Exceptions propagate unless __exit__ suppresses them by returning True (rare; use carefully).
Prefer with open(...) as f over manual try/finally for files—it is shorter and harder to misuse.
from contextlib import contextmanager
@contextmanager
def tag(name):
print(f"<{name}>")
try:
yield
finally:
print(f"</{name}>")
with tag("section"):
print("content")
What is an exception? An exception interrupts normal control flow when an error or special condition occurs.
An exception interrupts normal control flow when an error or special condition occurs. Python walks the stack looking for an except clause that matches the exception type, running cleanup in finally along the way.
BaseException sits above Exception; user code typically catches Exception or narrower types like ValueError, TypeError, or IOError subclasses.
try suites hold guarded code; except binds the exception instance; else runs if no exception occurred; finally always runs. Bare except: is discouraged because it hides KeyboardInterrupt and SystemExit.
Exception chaining (__cause__, __context__) preserves original tracebacks when you wrap errors—use raise NewError(...) from old in Python 3.
try:
value = int("42a")
except ValueError as exc:
print("bad int:", exc)
What is a finally block? A finally clause runs after try completes, whether it succeeded, raised, or returned from inside try or except.
A finally clause runs after try completes, whether it succeeded, raised, or returned from inside try or except. It is the right place to release external resources if you are not using a context manager.
If finally executes a return or raise, that overrides a pending return or exception from try/except unless suppressed carefully—usually you keep finally minimal.
Use finally to close sockets, flush buffers, or reset global state when context managers are unavailable. In modern code, prefer with for files and threading.Lock.
Be careful not to mask errors accidentally: logging in finally is fine; swallowing exceptions there can hide bugs.
def read_number(path):
fh = open(path, "r", encoding="utf-8")
try:
return int(fh.read().strip())
finally:
fh.close()
Raising exceptions intentionally raise signals an error or abnormal condition to callers.
raise signals an error or abnormal condition to callers. You can raise built-in exceptions with a message, re-raise the current exception with bare raise, or chain with raise NewError from old to preserve context.
Custom exception classes subclass Exception and often add fields for diagnostics. Libraries define hierarchies so callers can catch broad or specific types.
Validate inputs early and raise ValueError or TypeError with actionable messages. Avoid using exceptions for ordinary control flow when a simple if suffices.
Document which functions raise which exceptions so API users know what to catch. In async code, the same principles apply to awaitable failures.
def divide(a, b):
if b == 0:
raise ValueError("b must be non-zero")
return a / b
try:
divide(1, 0)
except ValueError as exc:
raise RuntimeError("division setup failed") from exc
Common exceptions you will see Certain exceptions appear constantly: ValueError for wrong values, TypeError for wrong types or arity, KeyError for missing dict keys, IndexError for
Certain exceptions appear constantly: ValueError for wrong values, TypeError for wrong types or arity, KeyError for missing dict keys, IndexError for out-of-range sequences, FileNotFoundError for missing paths, and PermissionError for OS denials.
Recognizing them speeds debugging: read the traceback bottom-up, identify the line, and reproduce with minimal input.
Catch the narrowest type that you can handle; let unexpected errors propagate to logs. For dict access when keys may be absent, prefer .get or defaultdict instead of blanket except KeyError unless you are translating errors.
ImportError / ModuleNotFoundError signal path or environment issues; SyntaxError points to parser problems; IndentationError is a specific syntax subclass for layout mistakes.
data = {"mode": "read"}
try:
print(data["write"])
except KeyError:
print("missing key: write")
What is the assert statement? assert condition [, message] evaluates condition and raises AssertionError if it is false.
assert condition [, message] evaluates condition and raises AssertionError if it is false. It documents invariants the author believes must always hold—internal consistency, not user input validation.
Python can strip asserts when running with optimization -O, so never rely on assert for security checks or required validation that must run in production.
Use asserts in development and tests to catch impossible states early. Pair with meaningful messages: assert x >= 0, f"expected non-negative, got {x}".
For public APIs, raise ValueError or custom exceptions with clear text instead of assert, because those paths remain under normal execution.
def average(nums):
assert len(nums) > 0, "nums must not be empty"
return sum(nums) / len(nums)
print(average([10, 20, 30]))
Object-oriented Python: classes, instances, and contracts Object-oriented programming models programs as collaborating objects: data lives in instances, behavior lives in methods,
Object-oriented programming models programs as collaborating objects: data lives in instances, behavior lives in methods, and classes describe shared structure. Python’s model is pragmatic: everything is an object, dunder methods hook into language features, and multiple inheritance is supported with the method resolution order (MRO).
This block connects syntax to design: you will name state clearly, encapsulate invariants, and reuse behavior through inheritance and composition. The goal is readable APIs, not clever hierarchies.
Start with simple classes, explicit __init__ for setup, and methods that operate on self. Learn when @dataclass or typing.NamedTuple reduces boilerplate without hiding intent.
Read the official tutorial on classes, then practice refactoring procedural scripts into small types. Prefer composition over deep inheritance trees unless a domain truly shares behavior.
class Greeter:
def __init__(self, name: str) -> None:
self.name = name
def hello(self) -> str:
return f'Hello, {self.name}'
print(Greeter('Softaims').hello())
What is a Python class and its instances? A class is a blueprint; an instance is a concrete object with its own attribute namespace.
A class is a blueprint; an instance is a concrete object with its own attribute namespace. The class statement creates a type object; calling the class constructs an instance and typically runs __init__ to initialize state.
Instance attributes attach per object; class attributes are shared unless shadowed. Understanding this distinction prevents surprise mutations when multiple instances share a mutable class-level default.
Use type hints on methods and attributes where they clarify contracts. isinstance() checks runtime types; abstract base classes in collections.abc document protocols like Iterable or Mapping.
Special methods (__str__, __repr__, __eq__) integrate instances with builtins and debugging. Keep __repr__ faithful enough to reconstruct or diagnose objects in logs.
class Counter:
kind = 'tally'
def __init__(self) -> None:
self.value = 0
def inc(self) -> None:
self.value += 1
c = Counter()
c.inc()
print(c.value, Counter.kind)
What is the __init__ initializer? __init__ is the instance initializer Python calls after the object is allocated.
__init__ is the instance initializer Python calls after the object is allocated. It is not the allocator—that is __new__ on the class—but most everyday classes only implement __init__ to set attributes and validate inputs.
Constructors express required dependencies and defaults. Clear parameters and docstrings make classes easier to test and compose.
Validate arguments early and raise ValueError or TypeError with actionable messages. Avoid heavy I/O in __init__; inject clients or use factory functions if setup is complex.
When subclasses extend initialization, call super().__init__(...) in a consistent order so cooperative multiple inheritance chains behave (see super() and MRO documentation).
class User:
def __init__(self, email: str, active: bool = True) -> None:
if '@' not in email:
raise ValueError('email looks invalid')
self.email = email
self.active = active
What is inheritance in Python? Inheritance lets a subclass reuse and extend a base class’s interface.
Inheritance lets a subclass reuse and extend a base class’s interface. Python supports multiple base classes; the MRO determines which method implementation runs when names overlap.
Not every “is-a” relationship needs inheritance; often a protocol or composed helper is simpler and easier to test.
Override methods by defining the same name on the subclass; call super().method() to extend rather than replace behavior. abstractmethod in abc enforces overrides on concrete subclasses.
Be cautious with super() in complex diamond graphs—linearize mentally with help(SubClass).mro(). Prefer shallow trees and explicit delegation when behavior diverges widely.
class Animal:
def speak(self) -> str:
return '...'
class Dog(Animal):
def speak(self) -> str:
return 'woof'
print(Dog().speak())
Modules, packages, and the import system Modules are namespaces backed by files (or extensions); packages are directories of modules with __init__.py (or namespace packages).
Modules are namespaces backed by files (or extensions); packages are directories of modules with __init__.py (or namespace packages). Imports make names available without loading duplicates unnecessarily thanks to sys.modules caching.
Organizing code into modules scales teams and reuse. Clear package boundaries, explicit public APIs in __all__, and stable entry points reduce coupling.
Learn absolute versus relative imports, package layout on PYTHONPATH, and python -m for runnable modules. Tools like ruff or isort keep import order consistent with PEP 8.
Publish internal libraries as installable packages when multiple apps share code; otherwise a src/ layout inside one repo often suffices.
# project/pkg/mathx.py
def double(x: int) -> int:
return x * 2
What is importing modules in Python? import module binds the name module to a loaded module object. from pkg import name pulls specific attributes.
import module binds the name module to a loaded module object. from pkg import name pulls specific attributes. Imports execute the module body once, then cache the module for subsequent imports in the same interpreter.
Circular imports happen when two modules load each other during initialization; refactor shared symbols into a third module or defer imports inside functions when necessary.
Prefer explicit imports over wildcard from module import * in application code—it obscures namespaces and frustrates static analysis.
Use importlib.reload sparingly in notebooks or hot reload; production services restart processes for deterministic upgrades.
import json
from pathlib import Path
data = json.loads(Path('config.json').read_text(encoding='utf-8'))
What is creating your own Python modules? Any .py file can act as a module when its directory is on sys.path. A package is a directory with __init__.
Any .py file can act as a module when its directory is on sys.path. A package is a directory with __init__.py (unless using namespace packages). Split large files by domain: models, services, cli.
Docstrings at module top describe purpose; if __name__ == "__main__": guards runnable demos without executing on import.
Expose a minimal surface: import heavy dependencies lazily if startup time matters. Keep side effects out of import time except logging configuration your team agrees on.
Add py.typed for typed packages; include tests that import the public API the way consumers will.
"""Small stats helpers for demos."""
def mean(nums: list[float]) -> float:
return sum(nums) / len(nums)
What is pip, the Python package installer? pip installs distributions from PyPI and other indexes into your active environment.
pip installs distributions from PyPI and other indexes into your active environment. It resolves dependencies, records metadata in site-packages, and pairs with pyproject.toml or requirements files for reproducible installs.
Treat pip as the low-level installer; many teams layer poetry, uv, or pip-tools for lockfiles and CI reproducibility.
Common flows: python -m pip install package, python -m pip install -r requirements.txt, and pip freeze for quick snapshots (prefer curated pins for production).
Use virtual environments so global site-packages stay clean and project versions do not clash.
python -m pip install --upgrade pip
python -m pip install requests
What is a Python virtual environment? A venv is an isolated directory with its own Python binary and site-packages.
A venv is an isolated directory with its own Python binary and site-packages. Activating it prepends that environment to PATH so pip and python refer to the project’s interpreter, not the system one.
Isolation prevents “works on my machine” drift and keeps deployment images smaller and predictable.
Create with python3 -m venv .venv; activate per OS (source .venv/bin/activate on Unix). Commit dependency manifests, not the whole venv, unless your workflow demands otherwise.
Document the supported Python minor version; use docker or CI matrices to test the range you claim to support.
python3 -m venv .venv
source .venv/bin/activate
python -m pip install -r requirements.txt
Web development with Python: frameworks and the request cycle Python powers many web stacks: microframeworks like Flask emphasize flexibility; Django ships batteries-included ORM,
Python powers many web stacks: microframeworks like Flask emphasize flexibility; Django ships batteries-included ORM, admin, and routing. Both sit on WSGI or ASGI servers that connect your app to HTTP.
This section frames how URLs become view functions or class-based views, how templates render HTML safely, and how you expose JSON APIs without mixing concerns.
Build a tiny Flask app locally, then contrast with Django’s project/app layout. Read each framework’s security checklist: CSRF, XSS, SQL injection, and secrets handling (see OWASP cheat sheets alongside framework docs).
Separate settings for development and production; never commit API keys. Use environment variables and a secrets manager in deployment.
from flask import Flask
app = Flask(__name__)
@app.get('/')
def home():
return {'service': 'ok'}
Flask versus Django: when each fits Flask provides routing, requests, responses, and extensions—you assemble ORM, auth, and forms.
Flask provides routing, requests, responses, and extensions—you assemble ORM, auth, and forms. Django bundles ORM, migrations, admin, forms, and conventions for larger products with many built-ins.
Neither is universally “better”; choose based on team familiarity, product shape, and operational constraints.
Prototype APIs or microservices often start fast in Flask or FastAPI; content-heavy sites with admin needs may favor Django. Both communities publish deployment guides—read them before production.
Evaluate async needs: ASGI stacks (Starlette, FastAPI, Django async views) suit websockets and high concurrency patterns; traditional WSGI remains common for many CRUD apps.
# Flask: minimal core
# Django: django-admin startproject mysite
What is a basic Flask application? Flask applications are WSGI callables created with Flask(__name__). Routes map URL rules to Python functions using decorators like @app.
Flask applications are WSGI callables created with Flask(__name__). Routes map URL rules to Python functions using decorators like @app.get("/path"). The development server (flask run) is for local work only.
Request and session objects expose headers, bodies, cookies, and query parameters; responses can be strings, tuples with status codes, or jsonify for JSON.
Run with flask --app module:app run --debug only in trusted environments—debug mode exposes a console. Production uses gunicorn, waitress, or uwsgi behind a reverse proxy.
Structure blueprints for larger apps; keep views thin and push logic into services you can unit test.
from flask import Flask, jsonify
app = Flask(__name__)
@app.get('/health')
def health():
return jsonify(status='ok')
What is routing in Flask? Routing maps HTTP methods and paths to Python callables. Variable rules like /user/ capture segments and pass them as arguments.
Routing maps HTTP methods and paths to Python callables. Variable rules like /user/
HTTP method routing splits read versus write operations cleanly and aligns with REST-ish APIs.
Register error handlers with @app.errorhandler for consistent JSON or HTML error pages. Use trailing slash rules consistently to avoid duplicate URLs.
For larger apps, Blueprint.route isolates feature areas under URL prefixes.
@app.post('/items')
def create_item():
return {'id': 1}, 201
What is Jinja2 templates in Flask? Jinja2 renders HTML (or text) from templates with {{ expressions }}, {% control %}, and filters.
Jinja2 renders HTML (or text) from templates with {{ expressions }}, {% control %}, and filters. Flask’s render_template loads files from the templates/ folder and auto-escapes HTML by default to reduce XSS risk.
Template inheritance (extends, block) shares layouts across pages without copy-paste.
Pass only the data templates need; avoid embedding business rules deeply in templates—compute in views or context processors when appropriate.
Keep user-generated content escaped; if you must render trusted HTML, understand Markup and the security implications (OWASP XSS prevention).
from flask import Flask, render_template
app = Flask(__name__)
@app.get('/hello/<name>')
def hello(name: str):
return render_template('hello.html', name=name)
What is an ORM in Python web apps? Object-relational mappers map tables to classes and rows to instances so you query with Python instead of raw SQL everywhere.
Object-relational mappers map tables to classes and rows to instances so you query with Python instead of raw SQL everywhere. Django ORM is built-in; SQLAlchemy is the dominant choice with Flask and many ASGI frameworks.
ORMs reduce boilerplate and help prevent some SQL injection classes when you parameterize queries through their APIs—but raw SQL still has a place for complex reporting.
Model relationships (foreign keys, many-to-many) should mirror your domain. Migrations (Alembic, Django migrations) evolve schema alongside code in version control.
Understand the N+1 query problem; use selectinload or prefetch_related patterns where your ORM documents them.
from sqlalchemy.orm import DeclarativeBase, Mapped, mapped_column
class Base(DeclarativeBase):
pass
class User(Base):
__tablename__ = 'users'
id: Mapped[int] = mapped_column(primary_key=True)
email: Mapped[str] = mapped_column(unique=True)
REST-style APIs in Python REST APIs expose resources over HTTP with nouns in paths, verbs in methods, and consistent status codes.
REST APIs expose resources over HTTP with nouns in paths, verbs in methods, and consistent status codes. Python frameworks return JSON via jsonify, Response models, or automatic serialization layers.
Security matters: authenticate callers, authorize per resource, validate input schemas, and rate-limit public endpoints. OWASP API Security Top 10 is the industry baseline.
Version your API (/v1), document with OpenAPI, and return problem+json or structured errors. Use pydantic or serializer classes to reject bad payloads early.
Test contracts with pytest and httpx client; add integration tests against a real database in CI when feasible.
from flask import Flask, request, jsonify
app = Flask(__name__)
@app.get('/users/<int:user_id>')
def get_user(user_id: int):
return jsonify(id=user_id, name='Ada')
What is WSGI and ASGI servers? WSGI is the synchronous Python web server interface used by Flask and classic Django.
WSGI is the synchronous Python web server interface used by Flask and classic Django. ASGI adds async capabilities and websocket support; Starlette and FastAPI are ASGI-native, and Django supports ASGI deployment.
Your framework produces an application callable; a server like Gunicorn or Uvicorn accepts TCP connections, parses HTTP, and invokes that callable.
Choose ASGI when you rely on async I/O end-to-end; mixing blocking code in async workers can stall the event loop. Profile under load before committing.
Terminate TLS at a reverse proxy (nginx, Caddy) or load balancer; run app workers behind it for simpler certificate rotation.
# WSGI: gunicorn myapp:app
# ASGI: uvicorn myapp:app --host 0.0.0.0 --port 8000
Data science with Python: arrays, tables, and models Python’s data stack centers on NumPy for dense arrays, pandas for labeled tables, Matplotlib for plotting, and scikit-learn for
Python’s data stack centers on NumPy for dense arrays, pandas for labeled tables, Matplotlib for plotting, and scikit-learn for classical machine learning. Together they cover exploration, visualization, and baseline modeling before deep learning frameworks.
This section emphasizes correct dtypes, vectorized operations, and reproducible notebooks or scripts—not memorizing every function name.
Learn array broadcasting and pandas indexes before stacking libraries. Keep random seeds fixed when demonstrating models; separate train/validation/test honestly.
For production ML, graduate from notebook-only workflows to packaged code, tests, and monitoring—but master the fundamentals here first.
import numpy as np
import pandas as pd
df = pd.DataFrame({'x': np.arange(5), 'y': np.linspace(0, 1, 5)})
print(df.head())
What is NumPy? NumPy provides ndarray, a contiguous n-dimensional array with homogeneous dtype and vectorized C/Fortran-backed operations.
NumPy provides ndarray, a contiguous n-dimensional array with homogeneous dtype and vectorized C/Fortran-backed operations. It is the foundation for pandas, SciPy, and many ML libraries.
Broadcasting rules let you combine arrays of compatible shapes without explicit loops, which is fast and readable when used intentionally.
Prefer ufuncs and slicing over Python loops for numeric hot paths. Watch copies versus views—some slices share memory; .copy() when you need independence.
Linear algebra lives in numpy.linalg; random number generation uses Generator in modern code instead of legacy global RNG patterns.
import numpy as np
a = np.array([1, 2, 3])
print(a * 2, a.mean())
What is pandas? pandas models tabular data as DataFrame (columns of Series) with indexes for alignment.
pandas models tabular data as DataFrame (columns of Series) with indexes for alignment. It shines at cleaning, grouping, merging, time series, and exporting to CSV/Parquet/SQL.
The index is part of the data model; many bugs come from accidental reindexing or duplicate labels.
Use vectorized methods and avoid iterrows in performance-sensitive code. describe(), value_counts(), and groupby answer exploratory questions quickly.
Handle missing data explicitly with isna, fillna, or dropna; understand nullable dtypes for integers and booleans in recent pandas.
import pandas as pd
df = pd.read_csv('sales.csv')
print(df.groupby('region')['amount'].sum())
What is Matplotlib?
Matplotlib is Python’s foundational plotting library: pyplot offers MATLAB-like state; the object-oriented API (figures and axes) scales to publication-quality charts.
It integrates with pandas (.plot) and Jupyter for quick exploration, yet remains suitable for scripted reporting.
Choose the OO API for reusable functions: fig, ax = plt.subplots(); ax.plot(...). Label axes, titles, and legends for clarity.
For interactive web dashboards, teams often layer Plotly or Altair; Matplotlib remains the default for static scientific figures.
import matplotlib.pyplot as plt
fig, ax = plt.subplots()
ax.plot([0, 1, 2], [0, 1, 0])
ax.set_title('Demo')
fig.savefig('demo.png')
What is scikit-learn? scikit-learn implements classical ML algorithms with a consistent estimator API: fit, predict, transform, and score.
scikit-learn implements classical ML algorithms with a consistent estimator API: fit, predict, transform, and score. Pipelines bundle preprocessing and models to prevent data leakage between train and test.
It focuses on CPU-friendly models and excellent documentation for learning; deep learning belongs in other frameworks.
Always split data before any fit; use cross-validation for hyperparameters. Inspect metrics appropriate to the problem (ROC-AUC, F1, MAE).
Read the user guide sections on preprocessing (StandardScaler, OneHotEncoder) and model selection—misapplied scaling is a common beginner mistake.
import numpy as np
from sklearn.linear_model import LogisticRegression
X = np.array([[0], [1], [2], [3]])
y = np.array([0, 0, 1, 1])
model = LogisticRegression(max_iter=200).fit(X, y)
print(model.predict([[1.5]]))
Async IO in Python: asyncio, concurrency, and parallelism Async programming lets one thread interleave many I/O-bound tasks by awaiting points where the event loop can switch work.
Async programming lets one thread interleave many I/O-bound tasks by awaiting points where the event loop can switch work. asyncio is the standard library framework for coroutines, tasks, and event loops in modern Python.
This section separates async concurrency from threads and processes: asyncio suits network and disk I/O; threads help blocking C extensions and some mixed workloads; multiprocessing sidesteps the GIL for CPU-bound parallelism.
Write async def coroutines, await other awaitables, and run them with asyncio.run(main()) at program entry. Use create_task for fire-and-forget work you still want to track.
Avoid calling blocking APIs inside async functions without executors—you will stall the loop. Profile with realistic concurrency to choose the right tool.
import asyncio
async def main() -> None:
await asyncio.sleep(0)
print('done')
asyncio.run(main())
What is the asyncio library?
asyncio provides the event loop, Task and Future types, transports and protocols for streams, synchronization primitives (Lock, Queue), and helpers like gather and wait.
It is the standard way to build async network clients and servers alongside frameworks like aiohttp or FastAPI.
Prefer high-level asyncio APIs in application code; drop to transports only when you need fine control. Use timeout wrappers (wait_for or 3.11+ timeouts) so hung I/O does not wedge your service.
Shutdown cleanly: cancel outstanding tasks on SIGTERM in servers and await them to release sockets and files.
import asyncio
async def tick() -> None:
for i in range(3):
print('tick', i)
await asyncio.sleep(0.1)
asyncio.run(tick())
What is async and await? async def defines a coroutine function; await suspends until the awaited awaitable completes, yielding control to the event loop.
async def defines a coroutine function; await suspends until the awaited awaitable completes, yielding control to the event loop. Only awaitables may follow await: coroutines, Tasks, Futures, and certain objects implementing __await__.
Calling a coroutine function returns a coroutine object—you must schedule it (asyncio.run, create_task, or await) or it never runs.
Async context managers (async with) and iterators (async for) integrate resources like streams and pagination with the same suspension model.
Generators use yield; native coroutines use await—do not confuse the two syntaxes when reading older stackoverflow answers.
async def fetch_label() -> str:
return 'Softaims'
async def main() -> None:
label = await fetch_label()
print(label)
What is the threading module? threading runs multiple OS threads in one interpreter, sharing memory.
threading runs multiple OS threads in one interpreter, sharing memory. The GIL limits parallel execution of Python bytecode, but threads still help when work releases the GIL (many I/O operations, some C extensions) or when integrating blocking libraries.
Synchronize shared mutable state with locks, queues, or concurrent.futures.ThreadPoolExecutor for bounded worker pools.
Race conditions are subtle: prefer immutable messages between threads or queue.Queue for producer-consumer patterns.
daemon threads exit abruptly when the main thread ends—understand shutdown ordering for logging and flush requirements.
import threading
def work(n: int) -> None:
print('from thread', n)
t = threading.Thread(target=work, args=(1,))
t.start()
t.join()
What is the multiprocessing module? multiprocessing spawns separate interpreter processes with independent memory, bypassing the GIL for CPU-bound Python code.
multiprocessing spawns separate interpreter processes with independent memory, bypassing the GIL for CPU-bound Python code. Pools map work across workers; Queues and Pipes move pickled data between processes.
Startup cost and IPC overhead mean processes win for heavy computation, not tiny tasks.
On Unix, fork semantics differ from Windows spawn—guard entry points with if __name__ == "__main__": to avoid recursive launches.
Prefer concurrent.futures.ProcessPoolExecutor for many batch jobs; it wraps multiprocessing with a familiar executor API.
from concurrent.futures import ProcessPoolExecutor
def square(x: int) -> int:
return x * x
with ProcessPoolExecutor() as pool:
print(list(pool.map(square, [1, 2, 3])))
Databases in Python: drivers, connections, and safe queries Python talks to databases through DB-API 2 drivers (sqlite3, psycopg, mysqlclient) and higher-level ORMs.
Python talks to databases through DB-API 2 drivers (sqlite3, psycopg, mysqlclient) and higher-level ORMs. You manage connections, transactions, and parameterized SQL to avoid injection and handle failures predictably.
This section maps common stores—SQLite for embedded workloads, PostgreSQL and MySQL for servers, MongoDB for document models—and contrasts ORMs with raw SQL.
Always parameterize queries; never interpolate user strings into SQL. Use context managers or pool APIs so connections return to the pool on success and failure.
Index and explain analyze slow queries at the database; ORMs cannot fix a missing index. OWASP SQL injection prevention applies even in Python.
import sqlite3
with sqlite3.connect(':memory:') as conn:
conn.execute('CREATE TABLE t (id INTEGER PRIMARY KEY, name TEXT)')
conn.execute('INSERT INTO t (name) VALUES (?)', ('ada',))
What is SQLite with Python? sqlite3 ships with CPython and embeds a full SQL engine in-process—ideal for tools, tests, and small deployments.
sqlite3 ships with CPython and embeds a full SQL engine in-process—ideal for tools, tests, and small deployments. Connections support transactions via commit/rollback or context managers.
SQLite has specific concurrency characteristics; for high write contention, server databases fit better.
Use bound parameters for all external input. Enable foreign keys with PRAGMA foreign_keys = ON when you rely on referential integrity.
For ORM users, SQLAlchemy’s SQLite dialect is mature; Alembic still applies for migrations.
import sqlite3
conn = sqlite3.connect('app.db')
try:
conn.execute('SELECT 1')
finally:
conn.close()
What is SQLAlchemy? SQLAlchemy provides a SQL expression language and ORM layered over DB-API drivers. Version 2.
SQLAlchemy provides a SQL expression language and ORM layered over DB-API drivers. Version 2.0 style encourages Mapped annotations and select() constructs for typed, composable queries.
You can use Core without ORM when reporting SQL needs full control but you still want connection pooling.
Engine, Session, and metadata are the structural pieces; migrations typically use Alembic. Avoid string-concatenated SQL; use bound parameters from the expression API.
Read pooling and timeout settings before production—defaults differ by deployment.
from sqlalchemy import create_engine, text
engine = create_engine('sqlite:///:memory:')
with engine.connect() as conn:
print(conn.execute(text('SELECT 1')).scalar_one())
What is psycopg (PostgreSQL driver)? psycopg is the widely used PostgreSQL adapter for Python (psycopg2 legacy numbering, psycopg3 current).
psycopg is the widely used PostgreSQL adapter for Python (psycopg2 legacy numbering, psycopg3 current). It implements DB-API patterns: connections, cursors, executemany, and server-side features like LISTEN/NOTIFY when you need them.
Pair it with SQLAlchemy or Django’s PostgreSQL backend for full-stack apps.
Use parameterized queries (%s placeholders with psycopg2) and set isolation levels explicitly for sensitive transactions.
Connection pooling: use SQLAlchemy’s pool or psycopg’s pool in services; do not open a new TCP connection per HTTP request at scale.
# pip install psycopg[binary]
# conn = psycopg.connect('dbname=test user=postgres')
What is MySQL connectors for Python? MySQL is accessed via mysqlclient, PyMySQL, or Oracle’s mysql-connector-python—each wraps the wire protocol with DB-API compatibility.
MySQL is accessed via mysqlclient, PyMySQL, or Oracle’s mysql-connector-python—each wraps the wire protocol with DB-API compatibility. Django and SQLAlchemy abstract most differences if you configure the right dialect.
Choose a driver your team will support in production images; some are pure Python, others use C extensions for speed.
Match server SQL modes and charset (utf8mb4) at connection time. Parameterize queries the same as PostgreSQL.
Test failover and read-replica lag if you split reads and writes.
# pip install mysql-connector-python
# import mysql.connector
What is PyMongo and MongoDB? PyMongo is the official Python driver for MongoDB.
PyMongo is the official Python driver for MongoDB. Documents are dict-like BSON values; you insert, query with filters, update with operators, and create indexes for performance.
Schema flexibility shifts validation to application code or MongoDB schema validation rules.
Understand eventual consistency and replica set behavior for your read concern. Sanitize query documents—NoSQL injection is real (OWASP).
For async services, Motor wraps PyMongo for asyncio.
from pymongo import MongoClient
client = MongoClient('mongodb://localhost:27017')
db = client['demo']
db.users.insert_one({'name': 'Ada', 'role': 'engineer'})
What is database connections and pooling? A connection is an expensive, stateful link to a database server.
A connection is an expensive, stateful link to a database server. Pools reuse connections across requests to amortize TCP and auth costs while bounding maximum open handles.
Transactions scope atomic units of work; commit on success, rollback on error, and never leave connections checked out longer than needed.
Configure pool size, overflow, recycle timeouts, and pre-ping options (SQLAlchemy) to survive idle disconnects behind load balancers.
In serverless, pooling strategies differ—consider external poolers like PgBouncer when many ephemeral workers connect.
# SQLAlchemy engine manages pooling by default
# engine = create_engine(url, pool_size=10, max_overflow=5)
ORM versus raw SQL: trade-offs ORMs map rows to objects, automate migrations, and reduce repetitive CRUD.
ORMs map rows to objects, automate migrations, and reduce repetitive CRUD. Raw SQL (or Core expressions) offers maximal control for analytics, bulk loads, and database-specific features.
Healthy codebases mix both: ORM for domain logic, targeted SQL for hot paths or reports—document where and why.
Watch for implicit lazy loads (N+1) in ORMs; use joins or eager loading. For raw SQL, centralize string templates and always bind parameters.
Security and performance reviews should include generated SQL from ORMs, not only hand-written queries.
# ORM-style (SQLAlchemy 2.0 sketch)
# stmt = select(User).where(User.active.is_(True))
# Raw with bound params
# cursor.execute('SELECT * FROM users WHERE email = %s', (email,))
Testing Python code: units, integration, and confidence Automated tests document behavior, catch regressions, and enable refactors.
Automated tests document behavior, catch regressions, and enable refactors. Python’s unittest module provides xUnit-style classes; pytest is the dominant third-party runner with fixtures and rich plugins.
Mocking isolates units from I/O, clocks, and external services so tests stay fast and deterministic.
Start by testing pure functions and small classes, then add integration tests around database and HTTP boundaries. Run tests in CI on every push.
Measure coverage as a guide, not a goal—focus on critical paths and invariants rather than chasing 100% on trivial getters.
def add(a: int, b: int) -> int:
return a + b
assert add(2, 3) == 5
What is unit testing in Python? Unit tests verify the smallest meaningful pieces—functions, methods, or classes—in isolation.
Unit tests verify the smallest meaningful pieces—functions, methods, or classes—in isolation. Fast feedback lets you change code confidently; slow, flaky suites get ignored.
Arrange inputs, act on the system under test, and assert expected outputs or exceptions.
Name tests after behavior, not implementation details. One logical assertion per test (or closely related group) keeps failures diagnosable.
Keep tests beside code (tests/ mirror) and import the package the way users do.
import unittest
class TestMath(unittest.TestCase):
def test_add(self) -> None:
self.assertEqual(1 + 1, 2)
if __name__ == '__main__':
unittest.main()
What is the unittest module? unittest follows xUnit patterns: test classes subclass unittest.
unittest follows xUnit patterns: test classes subclass unittest.TestCase, methods named test_* run automatically, and assertions like assertEqual raise failures with messages.
Fixtures use setUp/tearDown; test discovery via python -m unittest discover finds packages of tests.
unittest.mock.patch replaces attributes temporarily for isolation. For async code, IsolatedAsyncioTestCase exists in recent Python versions.
Many teams still prefer pytest for ergonomics, but unittest ships with the standard library and integrates everywhere.
import unittest
class Demo(unittest.TestCase):
def test_truth(self) -> None:
self.assertTrue(True)
What is pytest? pytest runs plain functions named test_* and uses plain assert with introspection on failure.
pytest runs plain functions named test_* and uses plain assert with introspection on failure. Fixtures inject dependencies (tmp_path, databases, clients) with scopes for function, module, or session.
Parametrize generates matrix tests; plugins extend coverage, asyncio, and Django integration.
conftest.py shares fixtures across directories. Keep fixtures focused; avoid hidden global state.
Use markers to skip slow tests locally but run them nightly in CI.
def test_upper():
assert 'softaims'.upper() == 'SOFTAIMS'
What is mocking in tests? Mocks stand in for real collaborators: network clients, databases, or time. unittest.mock provides Mock, MagicMock, patch, and AsyncMock for coroutines.
Mocks stand in for real collaborators: network clients, databases, or time. unittest.mock provides Mock, MagicMock, patch, and AsyncMock for coroutines.
Over-mocking couples tests to implementation; mock boundaries, not every private helper.
patch targets where a name is looked up (usually the module under test), not where it is defined—getting this wrong is a common confusion.
Assert call counts and arguments with mock.assert_called_once_with; use spec= to catch typos on methods.
from unittest.mock import Mock
api = Mock()
api.fetch.return_value = {'ok': True}
assert api.fetch('/health')['ok']
Deploying Python services: processes, servers, and containers Deployment packages your app with dependencies, runs it under a production WSGI/ASGI server, often behind nginx for TL
Deployment packages your app with dependencies, runs it under a production WSGI/ASGI server, often behind nginx for TLS and static files, and ships in containers for reproducible environments. CI/CD automates test, build, and rollout.
Operational excellence means health checks, structured logs, metrics, and rollback plans—not only getting code onto a VM.
Pin dependencies, use multi-stage Docker builds, run as non-root, and inject configuration via environment variables. Terminate gracefully on SIGTERM so in-flight requests finish or timeout cleanly.
Pair this reading with your cloud provider’s Python runtime docs and OWASP deployment hardening guidance.
FROM python:3.12-slim
WORKDIR /app
COPY requirements.txt .
RUN pip install --no-cache-dir -r requirements.txt
COPY . .
CMD ["gunicorn", "myapp:app", "-b", "0.0.0.0:8000"]
What is virtual environments in deployment? Production images usually create a clean environment (venv or base image plus pip install) so only declared dependencies exist.
Production images usually create a clean environment (venv or base image plus pip install) so only declared dependencies exist. Avoid copying a developer laptop’s site-packages.
requirements.txt or lockfiles (pip-tools, poetry export) make builds repeatable; scan for vulnerabilities in CI.
Separate build and runtime stages: compile wheels where needed, then copy artifacts into a slim final image.
Document the Python minor version and platform wheels you rely on (manylinux, aarch64).
python -m venv /opt/venv
/opt/venv/bin/pip install -r requirements.txt
What is Gunicorn? Gunicorn is a pre-fork WSGI HTTP server for UNIX. A master process manages worker processes that serve your Flask or Django app callable.
Gunicorn is a pre-fork WSGI HTTP server for UNIX. A master process manages worker processes that serve your Flask or Django app callable. Tune worker count from CPU cores and workload (sync vs gevent workers if you accept their trade-offs).
It terminates workers that miss heartbeat thresholds and reloads on signals for zero-downtime patterns when paired with process managers.
Bind to localhost behind nginx or a cloud load balancer; let the edge terminate TLS. Set timeouts compatible with upstream proxies.
For ASGI apps, Uvicorn or Hypercorn replaces Gunicorn’s default worker model—pick the stack your framework documents.
gunicorn mypackage.wsgi:application --bind 0.0.0.0:8000 --workers 4
What is nginx as a reverse proxy? nginx terminates TLS, serves static assets, enforces rate limits, and reverse-proxies to Gunicorn/Uvicorn.
nginx terminates TLS, serves static assets, enforces rate limits, and reverse-proxies to Gunicorn/Uvicorn. It buffers slow clients so your Python workers stay free for application logic.
Configuration splits server blocks, upstream pools, and location paths for APIs versus static sites.
Set proxy headers (X-Forwarded-For, X-Forwarded-Proto) so apps build correct external URLs and audit logs. Tune client_max_body_size for uploads.
Health checks at the proxy can remove bad upstreams before users see errors.
location / {
proxy_pass http://127.0.0.1:8000;
proxy_set_header Host $host;
proxy_set_header X-Forwarded-For $proxy_add_x_forwarded_for;
}
What is Docker for Python apps? Containers package your interpreter, dependencies, and code into an image runnable anywhere with the Docker engine or compatible runtimes.
Containers package your interpreter, dependencies, and code into an image runnable anywhere with the Docker engine or compatible runtimes. They improve parity between dev and prod when used thoughtfully.
Images are layered; order Dockerfile instructions so dependency layers cache when code changes frequently.
Use .dockerignore to omit .git, venv, and secrets. Run migrations as an init job or entrypoint step with idempotence, not only at image build time.
Scan images for CVEs; update base images regularly.
docker build -t myapp:1.0 .
docker run --rm -p 8000:8000 myapp:1.0
CI/CD pipelines for Python projects Continuous integration runs tests, linters, and type checks on every change.
Continuous integration runs tests, linters, and type checks on every change. Continuous delivery extends that to build artifacts and deploy to staging or production with approvals and rollbacks.
Caches for pip and prebuilt wheels speed pipelines; matrix jobs test multiple Python versions.
Typical stages: install deps, ruff/mypy/pytest, build container, push to registry, deploy with infrastructure as code. Store secrets in the CI vault, not repository variables checked into git.
Fail builds on coverage drops only when the metric is meaningful; prefer smoke tests post-deploy.
# GitHub Actions example (conceptual)
# - uses: actions/setup-python@v5
# with:
# python-version: '3.12'
# - run: pip install -r requirements.txt && pytest
Web scraping in Python: HTTP, HTML, and ethics Web scraping programmatically fetches pages and extracts structured data from HTML, APIs, or rendered DOM.
Web scraping programmatically fetches pages and extracts structured data from HTML, APIs, or rendered DOM. Python’s requests library handles HTTP; parsers like Beautiful Soup and frameworks like Scrapy scale crawling; Selenium drives browsers when JavaScript or anti-bot measures require a real DOM.
Respect robots.txt where applicable, site terms of service, rate limits, and privacy law. OWASP guidance on secure consumption of third-party HTML still applies when you render scraped content.
Start with simple GETs and stable CSS selectors or XPath; add retries, backoff, and caching. Log failures and snapshot HTML when parsers break on layout changes.
Prefer official APIs when they exist. For dynamic sites, measure whether network APIs exposed to the front end are cleaner than headless browsers.
import requests
resp = requests.get('https://example.com', timeout=10)
resp.raise_for_status()
print(resp.status_code, len(resp.text))
What is the requests library? requests wraps urllib3 with an ergonomic API for HTTP verbs, headers, query parameters, JSON bodies, timeouts, and session cookies.
requests wraps urllib3 with an ergonomic API for HTTP verbs, headers, query parameters, JSON bodies, timeouts, and session cookies. It is the de facto choice for synchronous scraping and API clients.
Always set timeouts; default infinite waits hang workers in production.
Use resp.raise_for_status() after calls to fail fast on 4xx/5xx. Stream large downloads with stream=True to avoid loading entire bodies into memory.
Reuse requests.Session for connection pooling and shared headers across many URLs.
import requests
s = requests.Session()
s.headers.update({'User-Agent': 'SoftaimsBot/1.0'})
r = s.get('https://httpbin.org/get', timeout=5)
print(r.json()['url'])
What is Beautiful Soup? Beautiful Soup parses HTML and XML into navigable trees. It pairs with html.parser, lxml, or html5lib backends.
Beautiful Soup parses HTML and XML into navigable trees. It pairs with html.parser, lxml, or html5lib backends. find, find_all, CSS selectors (soupsieve), and .text extraction turn messy markup into data.
It is forgiving with broken HTML, which mirrors the real web better than strict parsers.
Prefer stable attributes (data-*, ids agreed with frontend) over brittle positional selectors. When sites change often, centralize parsing functions and add regression tests with saved fixtures.
Watch encoding declarations; Beautiful Soup can guess encodings but explicit resp.content + parser beats surprises.
from bs4 import BeautifulSoup
html = '<p class="title">Softaims</p>'
soup = BeautifulSoup(html, 'html.parser')
print(soup.select_one('.title').get_text(strip=True))
What is Scrapy? Scrapy is an asynchronous crawling framework: spiders yield requests, pipelines clean and store items, middlewares handle retries and proxies.
Scrapy is an asynchronous crawling framework: spiders yield requests, pipelines clean and store items, middlewares handle retries and proxies. It schedules fetches efficiently and scales beyond ad hoc scripts.
Projects include settings for concurrency, obeying robots.txt, and exporting to JSON, CSV, or databases.
Learn Item, Spider, and Pipeline concepts before customizing downloader middleware. Use CrawlSpider rules for structured site graphs.
Run scrapy bench and tune AUTOTHROTTLE for polite crawling; log dropped responses to detect bans.
# scrapy startproject demo
# define parse() in spiders/site.py
What is Selenium with Python?
Selenium WebDriver automates real browsers—useful when content loads via JavaScript, infinite scroll complicates static HTML, or you need screenshots and interaction. The Python bindings drive Chrome, Firefox, or remote grids.
Headless modes reduce resource use but still execute full browser code; they are not a free pass on ethics or rate limits.
Prefer explicit waits (WebDriverWait) over time.sleep to avoid flaky tests. Scope selectors tightly and close drivers in finally blocks.
For large crawl jobs, consider Playwright as an alternative with strong automation APIs—compare licensing and browser support for your stack.
from selenium import webdriver
from selenium.webdriver.common.by import By
driver = webdriver.Chrome()
try:
driver.get('https://example.com')
print(driver.find_element(By.TAG_NAME, 'h1').text)
finally:
driver.quit()
What is parsing scraped data? Parsing converts raw HTML, JSON, or text into typed fields: dates, numbers, currencies, and URLs.
Parsing converts raw HTML, JSON, or text into typed fields: dates, numbers, currencies, and URLs. Normalization handles whitespace, encodings, and locale-specific formats before storage.
Schema validation (pydantic, marshmallow) catches bad rows early instead of corrupting downstream analytics.
Strip and canonicalize URLs; resolve relative links with urllib.parse.urljoin. Use datetime.fromisoformat or dateutil for messy date strings when appropriate.
Log parse failures with a snippet of source markup (redacted) to debug selector drift.
from urllib.parse import urljoin
base = 'https://example.com/a/'
href = '/b/c'
print(urljoin(base, href))
What is HTTP sessions and cookies? Many sites require cookies, CSRF tokens, or login flows spanning multiple requests. requests.
Many sites require cookies, CSRF tokens, or login flows spanning multiple requests. requests.Session persists cookies and headers across calls; you replay login POSTs then scrape authenticated pages.
Rotate User-Agent strings responsibly; fingerprinting and bot detection are increasingly sophisticated.
Store secrets outside code; load credentials from environment variables. Handle token refresh if APIs use short-lived JWTs.
When scraping authenticated areas, verify you have legal authorization and document data retention.
import requests
s = requests.Session()
payload = {'user': 'ada', 'token': 'from-env-not-hardcoded'}
s.post('https://example.com/login', data=payload, timeout=10)
r = s.get('https://example.com/dashboard', timeout=10)
print(r.ok)
Regular expressions in Python for text and validation Regular expressions describe patterns in text: digits, emails-like tokens, log lines, or markup fragments.
Regular expressions describe patterns in text: digits, emails-like tokens, log lines, or markup fragments. Python implements them in the re module with both procedural functions and compiled pattern objects for reuse.
They excel at line-oriented parsing and quick extractions; for HTML, prefer a parser unless the task is trivial and stable.
Learn a core subset first: character classes, quantifiers, anchors, groups, and non-greedy *?. Read the official HOWTO, then practice on realistic strings.
Mind catastrophic backtracking on nested quantifiers; simplify patterns or use atomic groups/possessive quantifiers where supported alternatives exist.
import re
m = re.search(r'\b\d{3}-\d{4}\b', 'call 555-0199')
print(m.group(0) if m else 'no match')
What is the re module? re.compile builds a pattern object for repeated searches. Functions like search, match, fullmatch, findall, finditer, and sub cover common workflows.
re.compile builds a pattern object for repeated searches. Functions like search, match, fullmatch, findall, finditer, and sub cover common workflows. Flags such as IGNORECASE or MULTILINE change matching rules.
Raw strings (r"...") avoid excessive backslash escaping in patterns.
match anchors at the start; fullmatch requires the whole string. Prefer search when the pattern can appear anywhere.
Use named groups (?P
import re
pat = re.compile(r'(?P<word>[A-Za-z]+)')
print(pat.findall('Softaims builds teams'))
What is pattern matching with regex? Patterns combine literals with metacharacters: . any char (with DOTALL exceptions), [] sets, ^ $ anchors, | alternation, () grouping.
Patterns combine literals with metacharacters: . any char (with DOTALL exceptions), [] sets, ^ $ anchors, | alternation, () grouping. Quantifiers ?, *, +, {m,n} control repetition.
Greedy quantifiers consume as much as possible; add ? after them for minimal matches in dense text.
Test patterns against edge cases: empty strings, unicode, and newlines. Unicode-aware classes exist (\w may include more than ASCII depending on flags).
When readability suffers, split into multiple passes or use a parser.
import re
text = '<tag>one</tag><tag>two</tag>'
print(re.findall(r'<tag>(.*?)</tag>', text))
What is search and replace with re.sub? re.sub(pattern, repl, string, count=0) substitutes matches.
re.sub(pattern, repl, string, count=0) substitutes matches. repl can be a string with backreferences (\1, \g
For simple literal replacements, str.replace avoids regex overhead and surprises.
Anchor replacements to whole words with \b when needed to avoid partial matches inside longer tokens.
Chain sub calls or build one pattern with alternation when refactoring structured logs.
import re
slug = re.sub(r'\s+', '-', ' soft aims ').lower().strip('-')
print(slug)
What is special characters in regex? Metacharacters like . * + ? [ ] ( ) { } ^ $ | \ need escaping with \ when you want literals, or place them in character classes with care.
Metacharacters like . * + ? [ ] ( ) { } ^ $ | \ need escaping with \ when you want literals, or place them in character classes with care. Raw strings keep Python-level backslashes sane.
Character classes support ranges, negation ^ at the start, and POSIX-like shorthands \d \s \w with UNICODE flags.
Non-capturing groups (?:...) group without creating a backreference—useful for alternation efficiency.
Comments (?#...) and verbose mode re.VERBOSE improve maintainability for long patterns.
import re
print(re.findall(r'[\w.-]+@[\w.-]+', 'contact [email protected] today'))
Decorators and generators: extending and pausing functions Decorators are callables that wrap functions or classes to add behavior—logging, caching, access checks, or registration—
Decorators are callables that wrap functions or classes to add behavior—logging, caching, access checks, or registration—without duplicating boilerplate. Generators pause execution with yield, producing lazy sequences and enabling cooperative multitasking patterns.
Together they underpin contextlib.contextmanager, many framework hooks, and memory-efficient iteration.
Write decorators with functools.wraps to preserve metadata. Understand closure timing when decorators capture arguments.
Practice small examples: a timing decorator, a simple count-up generator, then itertools consumption patterns.
def shout(fn):
def wrapper(*args, **kwargs):
return fn(*args, **kwargs).upper()
return wrapper
@shout
def greet(name: str) -> str:
return f'hello {name}'
print(greet('team'))
What is a Python decorator? A decorator is syntactic sugar: @decorator above def class replaces the defined object with decorator(original).
A decorator is syntactic sugar: @decorator above def class replaces the defined object with decorator(original). Decorators may return a new function, wrap with a class implementing __call__, or register the object in a framework.
They centralize cross-cutting concerns if used sparingly; deep stacks of decorators hurt debuggability.
Parameterizing decorators uses an outer function returning the real decorator. functools.wraps copies __name__ and __doc__ to the wrapper for introspection.
Class decorators modify or replace class objects similarly to function decorators.
def tag(label):
def deco(fn):
def wrapper(*args, **kwargs):
return f'[{label}] ' + fn(*args, **kwargs)
return wrapper
return deco
@tag('api')
def ping():
return 'pong'
print(ping())
What is function decorators? Function decorators wrap callables.
Function decorators wrap callables. Examples include staticmethod, classmethod, property builtins, and third-party tools like retry libraries or Flask’s @app.route (which registers endpoints).
Order matters when stacking decorators: the bottom decorator is applied first syntactically but wraps outward—read stacks top-down as “outer(inner(fn))”.
Type checkers understand @overload and ParamSpec for decorators that preserve signatures; use typing.Protocol when structural typing helps.
Test decorated functions still expose the right typing and docstrings for API consumers.
import functools
import time
def timed(fn):
@functools.wraps(fn)
def wrapper(*args, **kwargs):
start = time.perf_counter()
try:
return fn(*args, **kwargs)
finally:
print(f'{fn.__name__} took {time.perf_counter() - start:.4f}s')
return wrapper
What is class decorators? Class decorators receive a class object and return the same or a replacement class.
Class decorators receive a class object and return the same or a replacement class. Frameworks use them for dataclass transforms, ORM registration, or API schema generation.
They differ from metaclasses: decorators run after the class body executes; metaclasses customize class creation itself.
Keep class decorators pure when possible—mutating global registries has import-order side effects.
Combine with __slots__ or dataclasses carefully; understand attribute creation order.
def add_repr(cls):
cls.__repr__ = lambda self: f'{cls.__name__}()'
return cls
@add_repr
class Box:
pass
print(Box())
What is generators and iterators? Iterators implement __iter__ and __next__; generators are functions using yield that automatically create iterator objects.
Iterators implement __iter__ and __next__; generators are functions using yield that automatically create iterator objects. They compute values lazily, which saves memory for large streams.
The iterator protocol powers for loops, comprehensions, and many stdlib tools.
Generator expressions resemble list comprehensions but with parentheses; they pipeline nicely into functions like sum or any.
yield from delegates iteration to another iterable, simplifying recursive generators.
def count_up(n: int):
for i in range(n):
yield i
print(list(count_up(3)))
What is the yield keyword? yield pauses a generator function, returning a value to the consumer. On the next __next__ call, execution resumes after yield with local state intact.
yield pauses a generator function, returning a value to the consumer. On the next __next__ call, execution resumes after yield with local state intact. Two-way communication uses yield expressions with .send on generator objects.
return in a generator raises StopIteration with a value (used in asyncio internals and advanced patterns).
Do not mix return values for public APIs unless consumers understand PEP 380 semantics.
Close generators explicitly in long-lived pipelines to release resources tied to finally blocks.
def reader():
yield 'line-a'
yield 'line-b'
for line in reader():
print(line.upper())
Concurrency and parallelism in Python Concurrency interleaves tasks; parallelism runs them truly at the same time on multiple cores.
Concurrency interleaves tasks; parallelism runs them truly at the same time on multiple cores. Python offers asyncio for cooperative concurrency, threads for blocking I/O overlap, and multiprocessing (or native extensions) for CPU parallelism despite the GIL.
Choosing the right model depends on workload: I/O wait versus CPU saturation, library support, and operational complexity.
Profile with realistic data sizes. Avoid blocking calls inside asyncio; offload CPU work to executors or processes.
Read the official documentation on the GIL to set expectations for multithreaded numeric code in pure Python.
import asyncio
import time
async def main():
await asyncio.gather(
asyncio.sleep(0.1),
asyncio.sleep(0.1),
)
start = time.perf_counter()
asyncio.run(main())
print('elapsed', time.perf_counter() - start)
What is concurrency versus parallelism? Concurrent code may still run on one core if tasks await I/O; parallel code uses multiple cores simultaneously.
Concurrent code may still run on one core if tasks await I/O; parallel code uses multiple cores simultaneously. A web server can be concurrent with async workers while a video encode job needs parallelism.
Confusing the two leads to wrong tools: threads will not speed pure Python number crunching; processes will not fix poorly structured async code.
Diagram your critical path: where you wait on network, disk, locks, or CPU. Measure queue depths and saturation.
Combine models carefully—asyncio.to_thread bridges blocking libraries into async apps.
# asyncio for many waits; ProcessPoolExecutor for CPU-bound Python
What is asyncio for concurrency? asyncio schedules many coroutines on one thread, switching at await points.
asyncio schedules many coroutines on one thread, switching at await points. It excels when operations release the loop—network I/O, subprocess communication, certain file APIs on modern OS integrations.
Fairness and latency depend on not starving the loop with CPU work.
Use locks and semaphores to cap concurrency to external systems. Shield critical tasks from cancellation when needed.
On Python 3.11+, TaskGroup structures concurrent tasks; asyncio.gather remains the portable choice for many fan-out patterns.
import asyncio
async def worker(n: int) -> None:
await asyncio.sleep(0.05)
print('done', n)
async def main() -> None:
await asyncio.gather(*(worker(i) for i in range(3)))
asyncio.run(main())
What is threading for concurrency?
Threads let Python overlap blocking calls that release the GIL—socket reads, some database drivers, NumPy operations that drop to C without holding the GIL for long stretches.
Shared-memory concurrency requires discipline: locks around mutable structures, thread-safe queues for messaging.
ThreadPoolExecutor maps callables to a bounded pool—simpler than manual Thread lifecycle for many batch jobs.
Global Interpreter Lock details matter when you expect parallel Python bytecode execution—measure, do not assume.
from concurrent.futures import ThreadPoolExecutor, as_completed
def fetch(url: str) -> str:
return url
with ThreadPoolExecutor(max_workers=4) as pool:
futs = [pool.submit(fetch, u) for u in ['a', 'b', 'c']]
for fut in as_completed(futs):
print(fut.result())
What is multiprocessing for parallelism? Separate processes have independent Python interpreters and memory, enabling true parallel execution of Python code on multiple cores.
Separate processes have independent Python interpreters and memory, enabling true parallel execution of Python code on multiple cores. IPC uses pipes, queues, or shared memory for ndarray-heavy workflows.
Pickling limits which objects cross process boundaries—keep payloads simple or use shared memory for large arrays.
Pool.map is convenient for embarrassingly parallel numeric tasks; watch memory when inputs are huge.
Combine with numpy/scipy releases of the GIL only when you have profiled that C-level work dominates.
from multiprocessing import Pool
def work(x: int) -> int:
return x * x
if __name__ == '__main__':
with Pool(2) as pool:
print(pool.map(work, range(4)))
What is the Global Interpreter Lock (GIL)? The CPython GIL is a mutex allowing one thread to execute Python bytecode at a time in a process.
The CPython GIL is a mutex allowing one thread to execute Python bytecode at a time in a process. It simplifies memory management for core objects but limits parallel speedup for CPU-bound pure Python in threads.
I/O and many C extensions release the GIL during blocking or compute-heavy native code.
For CPU-bound Python, prefer multiprocessing, native extensions, or alternative runtimes if your constraints allow.
Free-threading and no-GIL experiments exist in the ecosystem—track release notes for your Python version.
# Profile first: threads can still help I/O-heavy apps under the GIL
Design patterns in Python: practical structure, not ceremony Design patterns name recurring solutions: creational patterns manage object construction, structural patterns compose o
Design patterns name recurring solutions: creational patterns manage object construction, structural patterns compose objects, behavioral patterns organize collaboration. Python’s first-class functions, duck typing, and dataclasses often replace verbose pattern implementations from other languages.
Use patterns when they clarify intent—not when they obscure a simpler function or module.
Study patterns alongside refactoring: extract functions before inventing hierarchies. Read the Gang of Four for vocabulary, then filter through Pythonic idioms.
Tests and types help you evolve patterns safely as requirements change.
from typing import Protocol
class Notifier(Protocol):
def send(self, msg: str) -> None: ...
def alert(n: Notifier, msg: str) -> None:
n.send(msg)
What is the Singleton pattern? Singleton restricts a class to one instance—useful for shared registries or expensive external clients.
Singleton restricts a class to one instance—useful for shared registries or expensive external clients. In Python, module-level objects are natural singletons because import caches modules.
Overuse creates hidden global state; prefer explicit dependency injection for testability.
If you need lazy initialization, combine a function returning a cached instance with threading.Lock in multithreaded apps.
Consider dependency injection frameworks only when complexity warrants them.
_cache: dict[str, object] = {}
def get_client(name: str) -> object:
if name not in _cache:
_cache[name] = object()
return _cache[name]
What is the Factory pattern? Factories encapsulate object construction behind functions or callable classes so callers depend on abstractions, not concrete constructors.
Factories encapsulate object construction behind functions or callable classes so callers depend on abstractions, not concrete constructors. They help when subclasses vary by configuration or runtime discovery.
Simple functions often suffice before introducing abstract Factory hierarchies.
Pair factories with registries (dict mapping keys to constructors) for plugin-style extensibility.
Use typing.Protocol or ABCs to document the returned interface.
def build_parser(kind: str):
if kind == 'json':
import json
return json.loads
raise ValueError(kind)
What is the Observer pattern? Observers subscribe to subject changes; the subject notifies them when state updates. GUI toolkits and reactive systems use this constantly.
Observers subscribe to subject changes; the subject notifies them when state updates. GUI toolkits and reactive systems use this constantly.
In Python, callbacks, lists of listeners, or asyncio.Queue fan-out can implement observers without heavy boilerplate.
Avoid notification cycles; document threading expectations if listeners run off the main thread.
For evented systems at scale, evaluate message brokers rather than in-process lists.
from typing import Callable
class Subject:
def __init__(self) -> None:
self._listeners: list[Callable[[str], None]] = []
def subscribe(self, fn: Callable[[str], None]) -> None:
self._listeners.append(fn)
def notify(self, payload: str) -> None:
for fn in list(self._listeners):
fn(payload)
What is dependency injection? Dependency injection passes collaborators (database clients, clocks, HTTP sessions) into objects or functions instead of constructing them internally.
Dependency injection passes collaborators (database clients, clocks, HTTP sessions) into objects or functions instead of constructing them internally. It improves testing because you can substitute fakes and controls time.
Python rarely needs heavyweight DI containers; constructor parameters and factory functions cover most cases.
Avoid service locators that hide dependencies; explicit parameters read better in large codebases.
Frameworks like FastAPI inject dependencies via Depends—study their scoping rules for request versus app lifetime.
class Greeter:
def __init__(self, clock):
self._clock = clock
def hello(self, name: str) -> str:
return f'{self._clock.now().isoformat()} hi {name}'
What is the Strategy pattern? Strategy factors interchangeable algorithms behind a common interface: sorting, pricing rules, or payment providers.
Strategy factors interchangeable algorithms behind a common interface: sorting, pricing rules, or payment providers. Callers depend on the protocol, not the concrete strategy class.
Functions are first-class strategies in Python—classes add state when needed.
typing.Protocol defines structural interfaces without inheritance. Register strategies in a dict for simple dispatch tables.
Keep strategies small and composable; avoid deep inheritance where a chain of functions reads clearer.
from typing import Protocol
class Discount(Protocol):
def apply(self, total: float) -> float: ...
def checkout(total: float, policy: Discount) -> float:
return policy.apply(total)
