Skip to content

Partitioning and Filesystem

Theory

On your operating system, the raw data on your disks can be accessed by files in /dev/ with appropriate previleges.

This lab assumes that your disk is formatted with GPT.

All examples in this lab are consistent as they came from the same disk.

Determining the name of your disk

To find out which file names are your disk, use lsblk.

$ lsblk
NAME         MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINTS
nvme0n1      259:0    0 476.9G  0 disk
├─nvme0n1p1  259:1    0     2G  0 part  /boot/efi
├─nvme0n1p2  259:2    0     6G  0 part  /boot
└─nvme0n1p3  259:3    0 468.9G  0 part  /

In this example, the system has one single disk, accessible at /dev/nvme0n1. This disk has a few partitions, each accessible at /dev/nvme0n1p1, /dev/nvme0n1p2, etc.

Your system may have a different type of disk and use completely different names, like sda or mmcblk0. You need to use your own names consistently.

Listing partitions

First, let’s use the fdisk tool to list the partitions on your disk.

sudo fdisk /dev/nvme0n1 -l

Here is an example output.

Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: ...
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A80B1F45-EEA2-4649-826A-A65635984BC5

Device             Start         End    Sectors    Size Type
/dev/nvme0n1p1      2048     4196351    4194304      2G EFI System
/dev/nvme0n1p2   4196352    16779263   12582912      6G Linux filesystem
/dev/nvme0n1p3  16779264  1000215215  983435952  468.9G Linux filesystem

Taking a look at the raw data of GPT

The following command reads the first 4 KiB of the disk /dev/nvme0n1 and displays the data in hexadecimal form.

sudo head -c 4KiB /dev/nvme0n1 | xxd

Here is an example output. Data starting from 0x00000600 is omitted.

00000000: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000160: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001c0: 0200 eeff ffff 0100 0000 af12 9e3b 0000  .............;..
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 55aa  ..............U.
00000200: 4546 4920 5041 5254 0000 0100 5c00 0000  EFI PART....\...
00000210: ffd5 9941 0000 0000 0100 0000 0000 0000  ...A............
00000220: af12 9e3b 0000 0000 0008 0000 0000 0000  ...;............
00000230: 8e12 9e3b 0000 0000 451f 0ba8 a2ee 4946  ...;....E.....IF
00000240: 826a a656 3598 4bc5 e007 0000 0000 0000  .j.V5.K.........
00000250: 8000 0000 8000 0000 607d 64a3 0000 0000  ........`}d.....
00000260: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000270: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000280: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000290: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000002f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000300: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000310: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000320: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000330: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000340: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000350: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000360: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000370: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000380: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000390: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000400: 2873 2ac1 1ff8 d211 ba4b 00a0 c93e c93b  (s*......K...>.;
00000410: 5ec1 c86e e48d 464f 9b69 5d02 4719 b5f2  ^..n..FO.i].G...
00000420: 0008 0000 0000 0000 ff07 4000 0000 0000  ..........@.....
00000430: 0000 0000 0000 0000 4500 4600 4900 2000  ........E.F.I. .
00000440: 5300 7900 7300 7400 6500 6d00 2000 5000  S.y.s.t.e.m. .P.
00000450: 6100 7200 7400 6900 7400 6900 6f00 6e00  a.r.t.i.t.i.o.n.
00000460: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000470: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000480: af3d c60f 8384 7247 8e79 3d69 d847 7de4  .=....rG.y=i.G}.
00000490: 6cfd 0ebb 729a 4ceb a35f 37b6 782e 6f3f  l...r.L.._7.x.o?
000004a0: 0008 4000 0000 0000 ff07 0001 0000 0000  ..@.............
000004b0: 0000 0000 0000 0000 4200 6f00 6f00 7400  ........B.o.o.t.
000004c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000004f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000500: af3d c60f 8384 7247 8e79 3d69 d847 7de4  .=....rG.y=i.G}.
00000510: 6ac7 0793 10d9 5649 90e7 a4f2 38e6 deb5  j.....VI....8...
00000520: 0008 0001 0000 0000 af12 9e3b 0000 0000  ................
00000530: 0000 0000 0000 0000 4c00 6900 6e00 7500  ........L.i.n.u.
00000540: 7800 2000 6600 6900 6c00 6500 7300 7900  x. .f.i.l.e.s.y.
00000550: 7300 7400 6500 6d00 0000 0000 0000 0000  s.t.e.m.........
00000560: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000570: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000580: 0000 0000 0000 0000 0000 0000 0000 0000  ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005b0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005c0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005d0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005e0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
000005f0: 0000 0000 0000 0000 0000 0000 0000 0000  ................
...

We are looking at the GPT on this disk. If you have the GPT specification at hand, you can actually interpret this output. Here is a brief description.

  • The GPT header (0x00000000 to 0x000003ff (LBAs 0, 1)). This part contains information to identify the GPT structure on this disk, including a protective MBR (0x00000000 to 0x000001ff) for backward compatibility. The disk information from the previous fdisk output is a human-readable representation of this part.

  • GPT partition entries (0x00000400 (LBA 2) onward). Each entry takes exactly 0x80 bytes containing a partition type GUID, a partition GUID, start and end LBA, attribute flags, and partition name. The partition list from the previous fdisk output is a human-readable representation of these entries.

If you look closer, you will find a clear structure in this data. You may further realize that when storing data in binary, it just makes (engineering) sense to align structures, because one can only read or overwrite on binary data. This is what we mean by “binary data has a fixed size and does not become longer or shorter” in the Binary Data chapter.

Partitions made accessible

Your operating system makes the partitions specified in the GPT appear as separate pieces of data, namely the parts you previously obtained with lsblk.

Using the same command as above, but with the OS-provided file /dev/nvme0n1p1, you can display the first 1 KiB of a partition. Alternatively, you can still use the raw disk /dev/nvme0n1 and manually skip to the partition. Both will display the same part of the disk. In this example, the byte we should skip to is given by (512 bytes/LBA) × (2048 LBAs) = 1048576 bytes = 1 MiB.

sudo head -c 1024 /dev/nvme0n1p1 | xxd
sudo tail -c +1048576 /dev/nvme0n1 | head -c 1024 | xxd

Filesystems made accessible

Normally, there is one filesystem established in each partition. Mounting is one layer of abstraction that makes a piece of raw filesystem data available as its logical content. The mount program manages what filesystems are made available by the operating system. To list currently available filesystems, use a single mount.

$ mount
/dev/nvme0n1p3 on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache,subvolid=257,subvol=/root)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime,seclabel)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
...

Behavior of filesystems

Deletion

As previously noted, filesystems delete files by marking the corresponding space as unused. Then, it is only a matter of time and chance whether or not that space will be overwritten by new or growing files.

Try creating a new file with some unique content, deleting it, and searching the raw filesystem data for that content. First, you need determine which filesystem you want to test. This example uses /dev/nvme0n1p3, which is mounted at /. Then, a file flag is created somewhere inside that filesystem (~ is inside /).

cd ~
FLAG="ja98n65c7n5t38zr"
echo $FLAG > flag
rm flag
sudo grep $FLAG /dev/nvme0n1p3

Note: Do not exit the shell during this experiment, or your shell may write the command history to disk.

Overwriting

Filesystems have different overwriting behavior. Some will overwrite at the original position, while others use an unpredictable position. To test this, we use shred. It simply does a file overwrite in a mounted filesystem, so the raw operation is still determined by the filesystem.

Use the following procedure to test. You need to test multiple times to get a reliable result.

cd ~
FLAG="sm0f8hc862f8ywvb"
echo $FLAG > flag
shred flag
sudo grep $FLAG /dev/nvme0n1p3