Partitioning and Filesystem
On your operating system, the raw data on your disks can be accessed by files in /dev/
with appropriate previleges.
This lab assumes that your disk is formatted with GPT.
All examples in this lab are consistent as they came from the same disk.
Determining the name of your disk
To find out which file names are your disk, use lsblk
.
$ lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINTS
nvme0n1 259:0 0 476.9G 0 disk
├─nvme0n1p1 259:1 0 2G 0 part /boot/efi
├─nvme0n1p2 259:2 0 6G 0 part /boot
└─nvme0n1p3 259:3 0 468.9G 0 part /
In this example, the system has one single disk, accessible at /dev/nvme0n1
. This disk has a few partitions, each accessible at /dev/nvme0n1p1
, /dev/nvme0n1p2
, etc.
Your system may have a different type of disk and use completely different names, like sda
or mmcblk0
. You need to use your own names consistently.
Listing partitions
First, let’s use the fdisk
tool to list the partitions on your disk.
sudo fdisk /dev/nvme0n1 -l
Here is an example output.
Disk /dev/nvme0n1: 476.94 GiB, 512110190592 bytes, 1000215216 sectors
Disk model: ...
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: gpt
Disk identifier: A80B1F45-EEA2-4649-826A-A65635984BC5
Device Start End Sectors Size Type
/dev/nvme0n1p1 2048 4196351 4194304 2G EFI System
/dev/nvme0n1p2 4196352 16779263 12582912 6G Linux filesystem
/dev/nvme0n1p3 16779264 1000215215 983435952 468.9G Linux filesystem
Taking a look at the raw data of GPT
The following command reads the first 4 KiB of the disk /dev/nvme0n1
and displays the data in hexadecimal form.
sudo head -c 4KiB /dev/nvme0n1 | xxd
Here is an example output. Data starting from 0x00000600
is omitted.
00000000: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000010: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000020: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000030: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000040: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000050: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000060: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000070: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000080: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000090: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000000f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000100: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000110: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000120: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000130: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000140: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000150: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000160: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000170: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000180: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000190: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001c0: 0200 eeff ffff 0100 0000 af12 9e3b 0000 .............;..
000001d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000001f0: 0000 0000 0000 0000 0000 0000 0000 55aa ..............U.
00000200: 4546 4920 5041 5254 0000 0100 5c00 0000 EFI PART....\...
00000210: ffd5 9941 0000 0000 0100 0000 0000 0000 ...A............
00000220: af12 9e3b 0000 0000 0008 0000 0000 0000 ...;............
00000230: 8e12 9e3b 0000 0000 451f 0ba8 a2ee 4946 ...;....E.....IF
00000240: 826a a656 3598 4bc5 e007 0000 0000 0000 .j.V5.K.........
00000250: 8000 0000 8000 0000 607d 64a3 0000 0000 ........`}d.....
00000260: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000270: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000280: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000290: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000002f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000300: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000310: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000320: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000330: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000340: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000350: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000360: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000370: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000380: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000390: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000003f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000400: 2873 2ac1 1ff8 d211 ba4b 00a0 c93e c93b (s*......K...>.;
00000410: 5ec1 c86e e48d 464f 9b69 5d02 4719 b5f2 ^..n..FO.i].G...
00000420: 0008 0000 0000 0000 ff07 4000 0000 0000 ..........@.....
00000430: 0000 0000 0000 0000 4500 4600 4900 2000 ........E.F.I. .
00000440: 5300 7900 7300 7400 6500 6d00 2000 5000 S.y.s.t.e.m. .P.
00000450: 6100 7200 7400 6900 7400 6900 6f00 6e00 a.r.t.i.t.i.o.n.
00000460: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000470: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000480: af3d c60f 8384 7247 8e79 3d69 d847 7de4 .=....rG.y=i.G}.
00000490: 6cfd 0ebb 729a 4ceb a35f 37b6 782e 6f3f l...r.L.._7.x.o?
000004a0: 0008 4000 0000 0000 ff07 0001 0000 0000 ..@.............
000004b0: 0000 0000 0000 0000 4200 6f00 6f00 7400 ........B.o.o.t.
000004c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000004d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000004e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000004f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000500: af3d c60f 8384 7247 8e79 3d69 d847 7de4 .=....rG.y=i.G}.
00000510: 6ac7 0793 10d9 5649 90e7 a4f2 38e6 deb5 j.....VI....8...
00000520: 0008 0001 0000 0000 af12 9e3b 0000 0000 ................
00000530: 0000 0000 0000 0000 4c00 6900 6e00 7500 ........L.i.n.u.
00000540: 7800 2000 6600 6900 6c00 6500 7300 7900 x. .f.i.l.e.s.y.
00000550: 7300 7400 6500 6d00 0000 0000 0000 0000 s.t.e.m.........
00000560: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000570: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000580: 0000 0000 0000 0000 0000 0000 0000 0000 ................
00000590: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005a0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005b0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005c0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005d0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005e0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
000005f0: 0000 0000 0000 0000 0000 0000 0000 0000 ................
...
We are looking at the GPT on this disk. If you have the GPT specification at hand, you can actually interpret this output. Here is a brief description.
-
The GPT header (
0x00000000
to0x000003ff
(LBAs 0, 1)). This part contains information to identify the GPT structure on this disk, including a protective MBR (0x00000000
to0x000001ff
) for backward compatibility. The disk information from the previousfdisk
output is a human-readable representation of this part. -
GPT partition entries (
0x00000400
(LBA 2) onward). Each entry takes exactly0x80
bytes containing a partition type GUID, a partition GUID, start and end LBA, attribute flags, and partition name. The partition list from the previousfdisk
output is a human-readable representation of these entries.
If you look closer, you will find a clear structure in this data. You may further realize that when storing data in binary, it just makes (engineering) sense to align structures, because one can only read or overwrite on binary data. This is what we mean by “binary data has a fixed size and does not become longer or shorter” in the Binary Data chapter.
Partitions made accessible
Your operating system makes the partitions specified in the GPT appear as separate pieces of data, namely the part
s you previously obtained with lsblk
.
Using the same command as above, but with the OS-provided file /dev/nvme0n1p1
, you can display the first 1 KiB of a partition. Alternatively, you can still use the raw disk /dev/nvme0n1
and manually skip to the partition. Both will display the same part of the disk. In this example, the byte we should skip to is given by (512 bytes/LBA) × (2048 LBAs) = 1048576 bytes = 1 MiB.
sudo head -c 1024 /dev/nvme0n1p1 | xxd
sudo tail -c +1048576 /dev/nvme0n1 | head -c 1024 | xxd
Filesystems made accessible
Normally, there is one filesystem established in each partition. Mounting is one layer of abstraction that makes a piece of raw filesystem data available as its logical content. The mount
program manages what filesystems are made available by the operating system. To list currently available filesystems, use a single mount
.
$ mount
/dev/nvme0n1p3 on / type btrfs (rw,relatime,seclabel,compress=zstd:1,ssd,space_cache,subvolid=257,subvol=/root)
/dev/nvme0n1p2 on /boot type ext4 (rw,relatime,seclabel)
/dev/nvme0n1p1 on /boot/efi type vfat (rw,relatime,fmask=0077,dmask=0077,codepage=437,iocharset=ascii,shortname=winnt,errors=remount-ro)
...
Behavior of filesystems
Deletion
As previously noted, filesystems delete files by marking the corresponding space as unused. Then, it is only a matter of time and chance whether or not that space will be overwritten by new or growing files.
Try creating a new file with some unique content, deleting it, and searching the raw filesystem data for that content. First, you need determine which filesystem you want to test. This example uses /dev/nvme0n1p3
, which is mounted at /
. Then, a file flag
is created somewhere inside that filesystem (~
is inside /
).
cd ~
FLAG="ja98n65c7n5t38zr"
echo $FLAG > flag
rm flag
sudo grep $FLAG /dev/nvme0n1p3
Note: Do not exit the shell during this experiment, or your shell may write the command history to disk.
Overwriting
Filesystems have different overwriting behavior. Some will overwrite at the original position, while others use an unpredictable position. To test this, we use shred
. It simply does a file overwrite in a mounted filesystem, so the raw operation is still determined by the filesystem.
Use the following procedure to test. You need to test multiple times to get a reliable result.
cd ~
FLAG="sm0f8hc862f8ywvb"
echo $FLAG > flag
shred flag
sudo grep $FLAG /dev/nvme0n1p3