docs: snapshots: block devices not explicitly flushed

Signed-off-by: Adrian Catangiu <[email protected]>

Author: acatangiu

docs/snapshotting/snapshot-support.md

@@ -18,7 +18,14 @@ except ARM which is not supported.
 ### Overview
 A Firecracker microVM snapshot can be used for loading it later in a different
 Firecracker process, and the original guest workload is being simply resumed.
-It is not guaranteed though that the state of the network connections survives
+
+The original guest which the snapshot is created from, should see no side
+effects from this process (other than the latency introduced by the snapshot
+creation process).
+
+Both network and vsock packet loss can be expected on guests that are resumed
+from snapshots in another Firecracker process.
+It is also not guaranteed that the state of the network connections survives
 the process.
 
 In order to make restoring possible, Firecracker snapshots save the full state
@@ -72,7 +79,7 @@ deal with cryptographic secrets. Please see [Snapshot security and uniqueness](#
   is not enabled, results in the full contents of guest memory being written to the
   snapshot.
 - The _memory file_ and _microVM state file_ are generated by Firecracker on snapshot
-  creation, and the disk contents are flushed to their backing files.  
+  creation. The disk contents are _not_ explicitely flushed to their backing files.
 - The API calls exposing the snapshotting functionality have clear **Prerequisites**
   that describe the requirements on when/how they should be used.
 
@@ -152,9 +159,12 @@ Details about the required and optional fields can be found in the
   - The file indicated by `snapshot_path` (e.g. `/path/to/snapshot_file`) contains the
     devices' model state and emulation state. The one indicated by `mem_file_path`
     (e.g. `/path/to/mem_file`) contains a full copy of the guest memory.
-  - The block devices contents are flushed to their backing files. At this point, these
-    should be backed up externally by the user. The generated snapshot files are
-    immediately available to be used (current process releases ownership).
+  - The generated snapshot files are immediately available to be used (current process
+    releases ownership). At this point, the block devices backing files should be
+    backed up externally by the user.
+    Please note that block device contents are only guaranteed to be committed/flushed
+    to the host FS, but not necessarily to the underlying persistent storage
+    (could still live in host FS cache).
   - If diff snapshots were enabled, the snapshot creation resets then the dirtied page
     bitmap and marks all pages clean (from a diff snapshot point of view).
 
@@ -334,3 +344,20 @@ properties are preserved and guaranteed before resuming the workload.
 
 We've started a discussion on how the Linux operating system might securely
 handle being snapshotted [here](https://lkml.org/lkml/2020/10/16/629).
+
+## Known Issues
+
+### Vsock must be inactive during snapshot
+
+Vsock device can break if snapshotted while having active connections.
+Firecracker snapshots do not capture any inflight network or vsock (through the
+linux unix domain socket backend) traffic that has left or not yet entered
+Firecracker.
+
+The above, coupled with the fact that Vsock control protocol is not resilient
+to vsock packet loss leads to Vsock device breakage when doing a snapshot while
+there are active Vsock connections.
+
+#### Workaround
+
+Close all active Vsock connections prior to snapshotting the VM.