Osquerybeat: Add elastic_ntfs_volumes and elastic_ntfs_partitions tables to osquery extension

[brian-mckinney]

Proposed commit message

Adds two new Windows-only tables to the Elastic osquery extension:

elastic_ntfs_volumes — Enumerates all mounted volumes on the system using the Windows Volume Management APIs. Provides the underlying physical device path (\.\PhysicalDriveN), device type (DISK vs CD_ROM), drive letter, volume label, and filesystem name. Fills a gap in logical_drives, which provides size/free-space but omits device path,
device type, and volume label.

elastic_ntfs_partitions — Enumerates all GPT/MBR partition table entries per physical disk using IOCTL_DISK_GET_DRIVE_LAYOUT_EX. Reports partition number, style (GPT/MBR/RAW), type name (System/Reserved/Basic/Recovery), GUID, starting offset, byte length, drive letter assignment, and GPT attributes flags (RequiredPartition, NoDriveLetter).

Implementation

Both tables share a single implementation package (pkg/ntfs) backed by an LRU cache to avoid redundant device enumeration within a single query. The package is structured around three layers:

• volume.go — enumerates volumes via FindFirstVolumeW/FindNextVolumeW + GetVolumeInformationW
• partition.go — enumerates partitions per disk via IOCTL_DISK_GET_DRIVE_LAYOUT_EX
• volume_reader.go — correlates volumes to physical drives via IOCTL_VOLUME_GET_VOLUME_DISK_EXTENTS
• ntfs_session.go — caches results per query context to avoid re-enumerating within a single osquery call
• lru.go — bounded LRU cache for session reuse

Testing

Accuracy verified live against a Windows 11 host with two virtual NVMe disks (5 volumes, 8 partitions total). See comment below with a claude built testing report

Checklist

• My code follows the style guidelines of this project
• I have commented my code, particularly in hard-to-understand areas
• I have made corresponding changes to the documentation
• I have made corresponding change to the default configuration files
• I have added tests that prove my fix is effective or that my feature works. Where relevant, I have used the stresstest.sh script to run them under stress conditions and race detector to verify their stability.
• I have added an entry in ./changelog/fragments using the changelog tool.

Disruptive User Impact

How to test this PR locally

Related issues

Use cases

Screenshots

Logs

[botelastic]

This pull request doesn't have a Team:<team> label.

[github-actions]

🤖 GitHub comments

Just comment with:

• run docs-build : Re-trigger the docs validation. (use unformatted text in the comment!)

[mergify]

This pull request does not have a backport label.
If this is a bug or security fix, could you label this PR @brian-mckinney? 🙏.
For such, you'll need to label your PR with:

• The upcoming major version of the Elastic Stack
• The upcoming minor version of the Elastic Stack (if you're not pushing a breaking change)

To fixup this pull request, you need to add the backport labels for the needed
branches, such as:

• backport-8./d is the label to automatically backport to the 8./d branch. /d is the digit
• backport-active-all is the label that automatically backports to all active branches.
• backport-active-8 is the label that automatically backports to all active minor branches for the 8 major.
• backport-active-9 is the label that automatically backports to all active minor branches for the 9 major.

[brian-mckinney]

I gave claude access to run osquery queries against our extension, and asked it to validate our table results for accuracy.

---

Accuracy Test: elastic_ntfs_volumes & elastic_ntfs_partitions

Date: 2026-04-14
Host: Windows 11 Pro (VMware, 2 NVMe virtual disks)
Branch: osquerybeat_extension_partitions_table

---

Test Environment

Item	Value
Disk 0	\\.\PHYSICALDRIVE0 — VMware Virtual NVMe, 171.8 GB, serial B3A1_0A13...
Disk 1	\\.\PHYSICALDRIVE1 — VMware Virtual NVMe, 4.3 GB, serial C1CF_078B...
Drive letters	C: (NTFS), D: (UDF/CD-ROM), E: (FAT32), F: (FAT16), G: (NTFS)

Comparison sources used:

Table	Origin	Notes
logical_drives	osquery built-in	Drive letter, size, free space, filesystem
disk_info	osquery built-in	Disk-level model, serial, partition count (WMI-backed)
physical_disk_performance	osquery built-in	Confirms disk-to-drive-letter mapping
device_partitions	Trail of Bits extension	TSK (Sleuth Kit) partition view; requires WHERE device = '\.\PhysicalDriveN'

Note: ntfs_part_data was not found in the loaded extensions on this host — the table does not exist in this osquery instance.

---

elastic_ntfs_volumes

Raw output

device	device_type	drive_letter	file_system_name	volume_label
\\.\PhysicalDrive0	DISK	C	NTFS
\\.\D:	CD_ROM	D	UDF	CCCOMA_X64FRE_EN-US_DV9
\\.\PhysicalDrive1	DISK	E	FAT32	NEW VOLUME
\\.\PhysicalDrive1	DISK	F	FAT	FATTY
\\.\PhysicalDrive1	DISK	G	NTFS	Whoa

Comparison against logical_drives

Volume enumeration: PASS — all 5 volumes (C:–G:) are found by both tables.

Drive	logical_drives file_system	elastic_ntfs_volumes file_system_name	Match?
C:	NTFS	NTFS	✓
D:	UDF	UDF	✓
E:	FAT32	FAT32	✓
F:	FAT	FAT	✓
G:	NTFS	NTFS	✓

physical_disk_performance names its entries "0 C:" and "1 E: F: G:", confirming the device-to-volume attribution in our table is correct.

Observation — FAT subtype: device_partitions (TSK) identifies drive F: as fat16 while both logical_drives and elastic_ntfs_volumes report it as FAT. The Windows filesystem API does not distinguish FAT12/FAT16 — it returns the generic string "FAT". TSK reads the superblock directly and is more precise. This is a Windows API limitation, not a bug in our table.

Unique value-add vs logical_drives:

• Physical device path (\\.\PhysicalDrive0) instead of just a drive letter
• device_type (DISK vs CD_ROM)
• volume_label

Gap vs logical_drives: No size or free-space data. A JOIN with logical_drives is needed for those fields.

---

elastic_ntfs_partitions

Raw output

PhysicalDrive0 (4 partitions):

#	style	type	drive_letter	id	starting_offset	length	attributes
1	GPT	System	—	87F914E5-9DB1-438A-BFAE-E7AE73B9B315	1,048,576	209,715,200	NoDriveLetter
2	GPT	Reserved	—	9B7EEA74-DE0B-4C71-AAED-FF36DC203BA5	210,763,776	16,777,216	NoDriveLetter
3	GPT	Basic	C	98CC5FAA-17C0-4151-8D2A-5B3C5EAE2162	227,540,992	170,778,427,392	None
4	GPT	Recovery	—	4F0D120F-9922-4787-904F-7D7E8B611519	171,005,968,384	790,626,304	RequiredPartition,NoDriveLetter

PhysicalDrive1 (4 partitions):

#	style	type	drive_letter	id	starting_offset	length	attributes
1	GPT	Reserved	—	E856A570-C9AF-4317-B02A-E83CFC804256	17,408	16,759,808	None
2	GPT	Basic	E	E616519B-839B-4328-9C4A-49C30E2A5FB1	16,777,216	1,048,576,000	None
3	GPT	Basic	F	37A6402A-B92A-4092-8F17-DA0150860D1C	1,065,353,216	1,048,576,000	None
4	GPT	Basic	G	84879964-152B-4534-B95F-898BBB9DBB3F	2,113,929,216	2,178,940,928	None

Comparison against device_partitions (Trail of Bits / TSK)

device_partitions exposes a richer TSK view that includes meta-entries (GPT header, safety table) and unallocated regions alongside the actual partitions. Filtering to type NOT IN ('meta','unallocated') gives the apples-to-apples comparison.

Offset alignment — PASS for all partitions

Every TSK offset matches the corresponding starting_offset in our table exactly:

Partition	TSK offset	elastic starting_offset	Match?
Drive0 EFI	1,048,576	1,048,576	✓
Drive0 Reserved	210,763,776	210,763,776	✓
Drive0 C:	227,540,992	227,540,992	✓
Drive0 Recovery	171,005,968,384	171,005,968,384	✓
Drive1 Reserved	17,408	17,408	✓
Drive1 E:	16,777,216	16,777,216	✓
Drive1 F:	1,065,353,216	1,065,353,216	✓
Drive1 G:	2,113,929,216	2,113,929,216	✓

Partition size comparison

TSK computed_bytes = blocks × blocks_size. For FAT and non-data partitions the match is exact; for NTFS partitions there is a consistent 4,096-byte (one cluster) discrepancy:

Partition	TSK computed bytes	elastic length	Diff	Status
Drive0 EFI (fat32)	209,715,200	209,715,200	0	✓ exact
Drive0 Reserved	16,777,216	16,777,216	0	✓ exact
Drive0 C: (ntfs)	170,778,423,296	170,778,427,392	−4,096	✓ expected
Drive0 Recovery (ntfs)	790,622,208	790,626,304	−4,096	✓ expected
Drive1 Reserved	16,759,808	16,759,808	0	✓ exact
Drive1 E: (fat32)	1,048,576,000	1,048,576,000	0	✓ exact
Drive1 F: (fat16)	1,048,576,000	1,048,576,000	0	✓ exact
Drive1 G: (ntfs)	2,178,936,832	2,178,940,928	−4,096	✓ expected

The NTFS discrepancy is structural, not an error: TSK counts filesystem clusters (the NTFS volume reports one fewer cluster than the raw partition size because the last cluster is reserved as the boot-sector backup). The Windows API used by elastic_ntfs_partitions reports the raw partition byte length inclusive of that reserved cluster. In all three NTFS cases the partition length in bytes is exactly divisible by the cluster size and TSK's block count is exactly (length / cluster_size) − 1.

FSType naming

Partition	device_partitions type	elastic_ntfs_partitions type	Note
Drive0 EFI	fat32	System	TSK sees the filesystem; Windows sees the GPT type GUID
Drive0 Reserved	normal	Reserved	TSK generic; Windows uses GPT type name
Drive0 C:	ntfs	Basic	✓ both correct in their model
Drive0 Recovery	ntfs	Recovery	TSK sees NTFS filesystem; Windows sees GPT recovery GUID
Drive1 E:	fat32	Basic	✓
Drive1 F:	fat16	Basic	TSK more specific than Windows API for FAT subtype
Drive1 G:	ntfs	Basic	✓

Partition count vs disk_info

disk_info (WMI) reports partitions = 3 for both disks. elastic_ntfs_partitions and device_partitions both return 4. WMI counts only data partitions; both TSK and the Windows IOCTL_DISK_GET_DRIVE_LAYOUT_EX API count all GPT entries including EFI, Reserved, and Recovery partitions. Our count is correct and more complete.

Unique value-add vs device_partitions

elastic_ntfs_partitions provides data not available from TSK:

• GPT partition GUID (id)
• GPT partition name string (e.g. "Basic data partition", "Microsoft reserved partition")
• attributes / attributes_mask flags (NoDriveLetter, RequiredPartition)
• Drive letter assignment
• GPT partition style (GPT) — relevant when MBR disks are also present

device_partitions provides data not available from our table:

• Inode count (filesystem-level, from TSK)
• Unallocated regions between partitions
• GPT header and table meta-entries
• More specific FAT subtype (FAT16 vs the generic "FAT" from Windows)

---

Summary

Check	Result
All volumes enumerated	PASS — 5/5 volumes found
File system types correct	PASS — NTFS, UDF, FAT32, FAT all match (Windows API limitation prevents FAT16 specificity)
Volume-to-disk attribution	PASS — matches physical_disk_performance
Partition count per disk	PASS — 4 reported vs WMI's 3; extra entries are EFI/Reserved/Recovery (correct)
Partition starting offsets	PASS — byte-perfect match against TSK for all 8 partitions
Partition sizes (FAT/non-data)	PASS — exact match against TSK
Partition sizes (NTFS)	PASS — consistently 4,096 bytes larger than TSK; expected (Windows reports raw length, TSK reports filesystem cluster count)
GPT partition GUIDs populated	PASS — all 8 partitions have valid GUIDs
GPT attributes flags	PASS — EFI and Recovery partitions correctly flagged NoDriveLetter/RequiredPartition
ntfs_part_data comparison	N/A — table not present in this osquery instance

[brian-mckinney]

closing in favor of #50140