doc: include interaction of dirty page tracking with hugepages

Extend the limitation section of the huge pages docs to include a note
about how dirty page tracking negates the benefits of huge pages.

Also add a cross-reference to the diff snapshot docs.

Signed-off-by: Patrick Roy <[email protected]>

Author: roypat

docs/hugepages.md

@@ -24,9 +24,7 @@ pool, please refer to the [Linux Documentation][hugetlbfs_docs].
 Restoring a Firecracker snapshot of a microVM backed by huge pages will also use
 huge pages to back the restored guest. There is no option to flip between
 regular, 4K, pages and huge pages at restore time. Furthermore, snapshots of
-microVMs backed with huge pages can only be restored via UFFD. Lastly, note that
-even for guests backed by huge pages, differential snapshots will always track
-write accesses to guest memory at 4K granularity.
+microVMs backed with huge pages can only be restored via UFFD.
 
 When restoring snapshots via UFFD, Firecracker will send the configured page
 size (in KiB) for each memory region as part of the initial handshake, as
@@ -40,6 +38,11 @@ Firecracker features:
 
 - Memory Ballooning via the [Balloon Device](./ballooning.md)
 
+Furthermore, enabling dirty page tracking for hugepage memory negates the
+performance benefits of using huge pages. This is because KVM will
+unconditionally establish guest page tables at 4K granularity if dirty page
+tracking is enabled, even if the host users huge mappings.
+
 ## FAQ
 
 ### Why does Firecracker not offer a transparent huge pages (THP) setting?

docs/snapshotting/snapshot-support.md

@@ -318,6 +318,9 @@ in the snapshot by instead considering all pages that were _accessed_ during the
 VM's lifetime. This potentially results in bigger memory files (although they
 are still sparse), but avoids the runtime overhead of dirty page logging.
 
+*Note*: Dirty page tracking negates most of the benefits of
+[huge pages](../hugepages.md#known-limitations).
+
 Without dirty page tracking enabled, Firecracker uses the
 [`mincore(2)`][man mincore] syscall to determine which pages to include in the
 snapshot. As such, this mode of snapshot taking will only work _if swap is