Skip to content

Pre linux.pagecache.recoverfs support #1561

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
ikelos merged 14 commits into from
Jan 19, 2025
Merged

Pre linux.pagecache.recoverfs support #1561

Changes from all commits
Commits
Show all changes
14 commits
Select commit Hold shift + click to select a range
08830fe
use architectures.LINUX_ARCHS
Abyss-W4tcher Jan 19, 2025
d325e1c
add inode_size and format_symlink to Inode* dataclasses
Abyss-W4tcher Jan 19, 2025
1d01593
switch to InodeUser.format_symlink
Abyss-W4tcher Jan 19, 2025
48a8f39
add and leverage follow_symlinks parameter
Abyss-W4tcher Jan 19, 2025
6f2ff4f
add InodeSize column to Files
Abyss-W4tcher Jan 19, 2025
8594106
1.0.1 -> 1.2.0 Files bump
Abyss-W4tcher Jan 19, 2025
e8b44ef
add and leverage write_inode_content_to_stream
Abyss-W4tcher Jan 19, 2025
818ddb7
2.0.0 -> 2.1.0 InodePages bump
Abyss-W4tcher Jan 19, 2025
da06867
typo
Abyss-W4tcher Jan 19, 2025
452d6e7
1.0.1 -> 1.1.0 Files bump
Abyss-W4tcher Jan 19, 2025
eddba98
type hint format_symlink
Abyss-W4tcher Jan 19, 2025
5970304
switch calling convention to context and layer name
Abyss-W4tcher Jan 19, 2025
a02243b
2.1.0 -> 3.0.0 InodePages bump
Abyss-W4tcher Jan 19, 2025
d42ffc0
typo
Abyss-W4tcher Jan 19, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
97 changes: 65 additions & 32 deletions volatility3/framework/plugins/linux/pagecache.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -6,8 +6,9 @@
import logging
import datetime
from dataclasses import dataclass, astuple
from typing import List, Set, Type, Iterable
from typing import List, Set, Type, Iterable, IO

from volatility3.framework.constants import architectures
from volatility3.framework import renderers, interfaces
from volatility3.framework.renderers import format_hints
from volatility3.framework.interfaces import plugins
Expand Down Expand Up @@ -37,6 +38,11 @@ class InodeUser:
modification_time: str
change_time: str
path: str
inode_size: int

@staticmethod
def format_symlink(symlink_source: str, symlink_dest: str) -> str:
return f"{symlink_source} -> {symlink_dest}"
Copy link
Contributor

@gcmoreira gcmoreira Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Not too fan of this format_symlink staticmethod. I'm not sure about the benefit of including a method in the inode data class that simply formats two external strings to 's1 -> s2'?
I think it would make more sense, and it's my mistake, but _follow_symlink should ideally be a public method in the inode object extension

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

A future implementation will require to format symlinks given an inode as well. Unifying it to a function allows to prevent code duplication, even if it might seem futile for one line indeed.

I put it under InodeUser as it is in fact only needed to make it more user readable after rendering 👍.



@dataclass
Expand Down Expand Up @@ -80,6 +86,7 @@ def to_user(
access_time_dt = self.inode.get_access_time()
modification_time_dt = self.inode.get_modification_time()
change_time_dt = self.inode.get_change_time()
inode_size = int(self.inode.i_size)
Copy link
Member

@ikelos ikelos Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Why is this being cast to an int? Is it not already an int? Please check, I'm really try to avoid people including unnecessary casts because they don't expect something to be an int, because then other people see it and they think they need to cast to an int, then some people do it out of caution, and before you know it the whole codebase is bursting full of pointless casts...

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This was explained by gcmoreira, in the comment right before the added line:

Ensure all types are atomic immutable. Otherwise, astuple() will take a long time doing a deepcopy of the Volatility objects.

I was also able to verify that not casting it to int, like the other properties, results in nested deepcopies.


inode_user = InodeUser(
superblock_addr=superblock_addr,
Expand All @@ -95,6 +102,7 @@ def to_user(
modification_time=modification_time_dt,
change_time=change_time_dt,
path=self.path,
inode_size=inode_size,
)
return inode_user

Expand All @@ -104,15 +112,15 @@ class Files(plugins.PluginInterface, timeliner.TimeLinerInterface):

_required_framework_version = (2, 0, 0)

_version = (1, 0, 1)
_version = (1, 1, 0)

@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(
name="kernel",
description="Linux kernel",
architectures=["Intel32", "Intel64"],
architectures=architectures.LINUX_ARCHS,
),
requirements.PluginRequirement(
name="mountinfo", plugin=mountinfo.MountInfo, version=(1, 2, 0)
Expand Down Expand Up @@ -148,10 +156,10 @@ def _follow_symlink(
"""
# i_link (fast symlinks) were introduced in 4.2
if inode and inode.is_link and inode.has_member("i_link") and inode.i_link:
i_link_str = inode.i_link.dereference().cast(
symlink_dest = inode.i_link.dereference().cast(
"string", max_length=255, encoding="utf-8", errors="replace"
)
symlink_path = f"{symlink_path} -> {i_link_str}"
symlink_path = InodeUser.format_symlink(symlink_path, symlink_dest)

return symlink_path

Expand Down Expand Up @@ -212,12 +220,14 @@ def get_inodes(
cls,
context: interfaces.context.ContextInterface,
vmlinux_module_name: str,
follow_symlinks: bool = True,
) -> Iterable[InodeInternal]:
"""Retrieves the inodes from the superblocks

Args:
context: The context that the plugin will operate within
vmlinux_module_name: The name of the kernel module on which to operate
follow_symlinks: Whether to follow symlinks or not

Yields:
An InodeInternal object
Expand Down Expand Up @@ -289,7 +299,9 @@ def get_inodes(
continue
seen_inodes.add(file_inode_ptr)

file_path = cls._follow_symlink(file_inode_ptr, file_path)
if follow_symlinks:
file_path = cls._follow_symlink(file_inode_ptr, file_path)

inode_in = InodeInternal(
superblock=superblock,
mountpoint=mountpoint,
Expand Down Expand Up @@ -377,6 +389,7 @@ def run(self):
("ModificationTime", datetime.datetime),
("ChangeTime", datetime.datetime),
("FilePath", str),
("InodeSize", int),
]

return renderers.TreeGrid(
Expand All @@ -389,15 +402,15 @@ class InodePages(plugins.PluginInterface):

_required_framework_version = (2, 0, 0)

_version = (2, 0, 0)
_version = (3, 0, 0)

@classmethod
def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]:
return [
requirements.ModuleRequirement(
name="kernel",
description="Linux kernel",
architectures=["Intel32", "Intel64"],
architectures=architectures.LINUX_ARCHS,
),
requirements.PluginRequirement(
name="files", plugin=Files, version=(1, 0, 0)
Expand All @@ -422,49 +435,69 @@ def get_requirements(cls) -> List[interfaces.configuration.RequirementInterface]

@staticmethod
def write_inode_content_to_file(
context: interfaces.context.ContextInterface,
layer_name: str,
inode: interfaces.objects.ObjectInterface,
filename: str,
open_method: Type[interfaces.plugins.FileHandlerInterface],
vmlinux_layer: interfaces.layers.TranslationLayerInterface,
) -> None:
"""Extracts the inode's contents from the page cache and saves them to a file

Args:
context: The context on which to operate
layer_name: The name of the layer on which to operate
inode: The inode to dump
filename: Filename for writing the inode content
open_method: class for constructing output files
vmlinux_layer: The kernel layer to obtain the page size
"""
if not inode.is_reg:
vollog.error("The inode is not a regular file")
return None

# By using truncate/seek, provided the filesystem supports it, a sparse file will be
# created, saving both disk space and I/O time.
# Additionally, using the page index will guarantee that each page is written at the
# appropriate file position.
try:
with open_method(filename) as f:
inode_size = inode.i_size
f.truncate(inode_size)

for page_idx, page_content in inode.get_contents():
current_fp = page_idx * vmlinux_layer.page_size
max_length = inode_size - current_fp
page_bytes = page_content[:max_length]
if current_fp + len(page_bytes) > inode_size:
vollog.error(
"Page out of file bounds: inode 0x%x, inode size %d, page index %d",
inode.vol.offset,
inode_size,
page_idx,
)
f.seek(current_fp)
f.write(page_bytes)

InodePages.write_inode_content_to_stream(context, layer_name, inode, f)
except OSError as e:
vollog.error("Unable to write to file (%s): %s", filename, e)

@staticmethod
def write_inode_content_to_stream(
context: interfaces.context.ContextInterface,
layer_name: str,
inode: interfaces.objects.ObjectInterface,
Copy link
Member

@ikelos ikelos Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

This is starting to make use of more complex data types (not too bad) and break away from the standard format for methods (first param context). I'd really prefer this take a context and a layer_name rather than just a layer.

I don't have very good reason for sticking with convention, originally it was because everything you'd do needed a context (and if you wanted to extend this in the future, you'll likely want the context there to do it). It was also the thought in the back of my mind of one day serializing such calls for some reason, although I no longer remember what (perhaps parallelization). Regardless, I think it'd be a good convention to maintain unless you have strong objections...

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 19, 2025 •
edited
Loading

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I basically followed the write_inode_content_to_file design, as it was the accepted way of doing it.

I also prefer to pass a context and a layer name, happy to revert it that way !

Copy link
Member

@ikelos ikelos Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yes please, I try not to be too nitpicky but it's hard keeping everything in order as it comes in so quickly. So it's be MINOR bump if we're just adding write_inode_content_to_stream and MAJOR bump if we're changing the signature of write_inode_content_to_file.

Copy link
Contributor Author

@Abyss-W4tcher Abyss-W4tcher Jan 19, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I proceeded to update write_inode_content_to_file as well.

stream: IO,
) -> None:
"""Extracts the inode's contents from the page cache and saves them to a stream

Args:
context: The context on which to operate
layer_name: The name of the layer on which to operate
inode: The inode to dump
stream: An IO stream to write to, typically FileHandlerInterface or BytesIO
"""
layer = context.layers[layer_name]
# By using truncate/seek, provided the filesystem supports it, and the
# stream is a File interface, a sparse file will be
# created, saving both disk space and I/O time.
# Additionally, using the page index will guarantee that each page is written at the
# appropriate file position.
inode_size = inode.i_size
stream.truncate(inode_size)

for page_idx, page_content in inode.get_contents():
current_fp = page_idx * layer.page_size
max_length = inode_size - current_fp
page_bytes = page_content[:max_length]
if current_fp + len(page_bytes) > inode_size:
vollog.error(
"Page out of file bounds: inode 0x%x, inode size %d, page index %d",
inode.vol.offset,
inode_size,
page_idx,
)
stream.seek(current_fp)
stream.write(page_bytes)

def _generator(self):
vmlinux_module_name = self.config["kernel"]
vmlinux = self.context.modules[vmlinux_module_name]
Expand Down Expand Up @@ -528,7 +561,7 @@ def _generator(self):
filename = open_method.sanitize_filename(f"inode_0x{inode_address:x}.dmp")
vollog.info("[*] Writing inode at 0x%x to '%s'", inode_address, filename)
self.write_inode_content_to_file(
inode, filename, open_method, vmlinux_layer
self.context, vmlinux_layer.name, inode, filename, open_method
)

def run(self):
Expand Down
Loading