Magic bytes and file carving

5 min read

A file's extension is a hint; its magic bytes are the truth. The first few bytes of a file identify its real type regardless of what it is named — PK\x03\x04 is a ZIP (and so is every Office document and JAR), \x89PNG is a PNG, \xFF\xD8\xFF is a JPEG, %PDF is a PDF. A file called photo.jpg whose first bytes are PK is a ZIP someone renamed.

Carving is finding files inside other files by scanning for those signatures. A disk image, a memory dump, or an innocent-looking picture can have a second file embedded in it — after the first file's end marker, or in a slack space. binwalk automates the scan; by hand you look for a magic sequence where the outer file should have ended. Many "extract the hidden file" challenges are exactly this: a valid image with a ZIP appended after its trailer, which image viewers ignore and archive tools happily open.