Python context managers define setup and exit behavior around a with block. They are commonly used for files, locks, transactions, and other resources whose cleanup should occur even when the block raises an exception. Their value comes from making ownership explicit, not from making every operation inside the block automatically safe.
A reliable design specifies what was acquired, who releases it, and what an error means. This guide explains the context protocol, generator-based helpers, composed resources, and tests that reveal incomplete cleanup or accidentally suppressed failures.
Identify the resource and its owner
Start with a concrete resource: an open file, a connection, a temporary directory, or a lock. Define the beginning and end of its useful lifetime. A context manager should make that boundary easier to see rather than hide unrelated state changes.
Distinguish owning a resource from borrowing it. A helper that receives a caller’s connection should not necessarily close it. Conversely, one that opens its own connection must provide a clear cleanup path. Ambiguous ownership can cause both leaks and premature closure.
Document what callers may retain after the block exits. Returning a handle that becomes unusable at exit is reasonable when the contract is clear. Letting it escape without explanation can create failures far from the original acquisition site.
Understand entry and exit behavior
The protocol uses entry and exit methods, with the exit path receiving information about an exception from the body. The return behavior of the exit method affects whether that exception is propagated or suppressed. Do not suppress errors merely to make cleanup look successful.
An entry failure is a special case. If acquisition partly succeeds before entry raises, ordinary exit behavior may not clean up everything automatically. Structure acquisition so partial resources are released, or use a supported composition mechanism for incremental ownership.
Review cleanup failures separately from body failures. An exception raised during cleanup can obscure the original problem. Preserve useful context and follow the library’s supported behavior rather than replacing every exception with a generic message.
Use generator-based managers carefully
Contextlib’s contextmanager decorator can express a simple acquire, yield, and cleanup pattern. Put cleanup in a finally block when it must run regardless of body outcome. The generator should yield exactly once according to the documented protocol.
from contextlib import contextmanager
@contextmanager
def owned_resource(acquire, release):
resource = acquire()
try:
yield resource
finally:
release(resource)This demonstrates ownership structure for trusted callbacks, not a universal implementation for every resource. If acquisition itself can create partial state, that state needs its own protection. If release can fail, define how that error is reported.
Do not catch an exception around yield solely to log it and then forget to re-raise. That can make the with block appear successful after its operation failed. Logging and propagation are separate decisions.
Compose variable resources with ExitStack
ExitStack supports cleanup composition when the number or sequence of resources is dynamic. It can register exit behavior and unwind acquired resources if later acquisition fails. This is often clearer than deeply nested manual try/finally blocks.
Register ownership as soon as a resource is successfully acquired. Avoid a gap where a subsequent operation can fail before the cleanup is associated with the stack. Read the distinction between entering a context and registering a callback so you use the appropriate interface.
Cleanup order matters. Resources acquired later may depend on earlier ones and should often be released first. Verify the intended ordering rather than assuming independently written cleanup functions can run in any sequence.
Separate transactions from connections
A database object’s context behavior may manage a transaction without closing the connection. Another library might implement different semantics. Read the actual interface instead of assuming every with connection block releases every associated resource.
Keep commit, rollback, and connection disposal explicit in the design. An application can roll back correctly while leaking the connection, or close the connection while mishandling the business transaction. Both boundaries need tests through the real library.
Do not include external side effects inside a database block and assume rollback reverses them. A sent email or remote API call has a separate lifecycle. Coordinate those effects through an appropriate workflow rather than expecting a context manager to provide distributed atomicity.
Review asynchronous ownership separately
Async context managers use the asynchronous protocol and can await entry and exit work. Use the interface supported by the resource, especially for network clients or asynchronous connections. A synchronous wrapper does not automatically perform asynchronous cleanup correctly.
Cancellation can occur while work or cleanup is running. Define the resource’s supported behavior and test interrupted operations. Do not leave tasks running after the context exits unless that transfer of ownership is intentional and documented.
Composition with asynchronous resources has its own supported tools. Choose the appropriate contextlib mechanism and avoid mixing sync and async interfaces through ad hoc event-loop manipulation.
Test normal and exceptional exits
Use a test resource that records acquisition and release without affecting real infrastructure. Verify cleanup after success, body failure, and later acquisition failure. Assert the intended exception propagation, not only that a release function was called.
Include cleanup failure and repeated-use cases according to the contract. Some context-manager instances are not reusable or reentrant. A helper that works once should not be advertised as safe for nested or repeated use without evidence.
A practical scenario is an export process that opens several input files and a temporary output. Use ExitStack to own the resources as they are acquired, and publish the completed output only after successful work. Failure should release handles and preserve a clear distinction between an incomplete file and an accepted artifact.
Keep cleanup observable without leaking secrets
Record resource categories, failure outcomes, and relevant nonsecret identifiers when needed. Avoid dumping file contents, connection strings, or private configuration during cleanup diagnostics. Errors often occur precisely when verbose troubleshooting is tempting.
Review long-lived resources separately from per-operation contexts. A shared client may belong to application startup and shutdown, while a temporary response belongs to one request. The right lifetime reduces overhead and makes leaks easier to reason about.
Frequently asked questions
Does with always close a database connection?
No. Context behavior is defined by the object. It may manage transaction state without closing the connection.
Should exit methods suppress every error?
No. Suppression is a deliberate semantic choice. Preserve failures unless the manager has a documented reason to handle them completely.
Where can I learn the composition tools?
Read the Python contextlib documentation. For interrupted asynchronous workflows, see our asyncio cancellation guide.
