Python's hashlib Module: Hashing Strings and Files

Hashing a string: encode first, then hash

hashlib hash objects operate on bytes, not text, so a plain Python string must be encoded first — nearly always with .encode("utf-8"):

import hashlib

data = "hello world"
digest = hashlib.sha256(data.encode("utf-8")).hexdigest()
print(digest)
# b94d27b9934d3e08a52e52d7da7dabfac484efe37a5380ee9088f7ace2efcde

# Same input always produces the same output -- hashing is deterministic
print(hashlib.sha256(b"hello world").hexdigest() == digest)   # True

.hexdigest() returns the hash as a readable hex string; .digest() returns the raw bytes, useful when you need to store or compare the hash in binary form rather than as text.

Hashing files without loading them entirely into memory

Hashing a multi-gigabyte file by reading it whole into a Python string or bytes object would use gigabytes of memory unnecessarily. Hash objects support incremental updates precisely so you can feed them a file in fixed-size chunks:

import hashlib

def file_sha256(path, chunk_size=65536):
    h = hashlib.sha256()
    with open(path, "rb") as f:
        while chunk := f.read(chunk_size):
            h.update(chunk)
    return h.hexdigest()

print(file_sha256("large_video.mp4"))

Feeding data through update() in pieces produces exactly the same final digest as hashing the whole thing at once — hashlib is designed for this streaming pattern, which is why downloading a file and verifying its checksum can happen chunk-by-chunk as the data arrives, rather than after the whole download completes.

Choosing an algorithm: sha256 vs md5 vs sha1

hashlib exposes several algorithms — md5(), sha1(), sha256(), sha512(), and the newer sha3_256() family among them — and it's worth knowing why the answer is almost always sha256 or better for anything new:

import hashlib

print(hashlib.algorithms_guaranteed)   # every algorithm available on this platform

# MD5 and SHA-1 still work, and are still fine for non-security uses like a quick
# content fingerprint, but must not be used anywhere collision resistance matters
checksum = hashlib.md5(b"some data").hexdigest()    # fine for a cache key, not for security
secure_hash = hashlib.sha256(b"some data").hexdigest()   # the safe default choice

MD5 has practical, demonstrated collision attacks — two different inputs producing the same hash — and SHA-1 does too, which is why major browsers and certificate authorities stopped trusting SHA-1 certificates years ago. Neither should anchor anything where a malicious party might benefit from crafting a colliding input, such as file integrity verification for untrusted uploads or digital signatures. For a quick internal deduplication key on data you trust, MD5's speed is sometimes still a reasonable tradeoff; for anything facing untrusted input, sha256 is the safe default.

hmac: hashing with a secret key

A plain hash proves data wasn't altered but doesn't prove who produced it, since anyone can compute a SHA-256 hash of anything. hmac combines a hash algorithm with a secret key to produce a keyed digest that only someone holding the same key could have generated — the standard building block for verifying webhook payloads and signed API requests:

import hmac, hashlib

secret_key = b"shared-secret-between-both-sides"
payload = b'{"event": "payment.completed", "amount": 4200}'

signature = hmac.new(secret_key, payload, hashlib.sha256).hexdigest()

# On the receiving side, verifying an incoming signature
def verify(payload, received_signature, secret_key):
    expected = hmac.new(secret_key, payload, hashlib.sha256).hexdigest()
    return hmac.compare_digest(expected, received_signature)   # constant-time compare

hmac.compare_digest() instead of == matters here: a plain string comparison returns as soon as it finds the first mismatched character, and the tiny timing difference this creates is, in principle, exploitable to guess a correct signature byte by byte. compare_digest() always takes the same amount of time regardless of where a mismatch occurs.

Why hashlib alone is the wrong tool for passwords

SHA-256 is intentionally fast — hashing gigabytes per second is the point for file integrity checks. That exact property makes it a poor choice for password storage: an attacker with a stolen table of SHA-256 password hashes can try billions of guesses per second on ordinary hardware. Purpose-built password hashing functions like bcrypt, scrypt, and Argon2 are deliberately slow and memory-hard, making large-scale guessing prohibitively expensive even with specialised hardware. Python's standard library ships hashlib.pbkdf2_hmac() as a built-in key-derivation option, but for new applications a maintained library implementing bcrypt or Argon2 is the more defensible choice. The full list of guaranteed algorithms and their properties is in the hashlib documentation.