Using Volatility 3 as a Library

This portion of the documentation discusses how to access the Volatility 3 framework from an external application.

The general process of using volatility as a library is to as follows:

  1. Creating a context

  2. (Optional) Determine what plugins are available

  3. (Optional) Determine what configuration options a plugin requires

  4. Set the configuration in the context

  5. (Optional) Using automagic to complete the configuration

  6. Run the plugin

  7. Render the TreeGrid

Creating a context

First we make sure the volatility framework works the way we expect it (and is the version we expect). The versioning used is semantic versioning, meaning any version with the same major number and a higher or equal minor number will satisfy the requirement. An example is below since the CLI doesn’t need any of the features from versions 1.1 or 1.2:

volatility3.framework.require_interface_version(1, 0, 0)

Contexts can be spun up quite easily, just construct one. It’s not a singleton, so multiple contexts can be constructed and operate independently, but be aware of which context you’re handing where and make sure to use the correct one. Typically once a context has been handed to a plugin, all objects will be created with a reference to that context.

ctx = contexts.Context()  # Construct a blank context

Determine what plugins are available

You can also interrogate the framework to see which plugins are available. First we have to try to load all available plugins. The import_files() method will automatically use the module paths for the provided module (in this case, volatility3.plugins) and walk the directory (or directories) loading up all python files. Any import failures will be provided in the failures return value, unless the second parameter is False in which case the call will raise any exceptions encountered. Any additional directories containing plugins should be added to the __path__ attribute for the volatility3.plugins module. The standard paths should generally also be included, which can be found in volatility3.constants.PLUGINS_PATH.

volatility3.plugins.__path__ = <new_plugin_path> + constants.PLUGINS_PATH
failures = framework.import_files(volatility3.plugins, True)

Note

Volatility uses the volatility3.plugins namespace for all plugins (including those in volatility3.framework.plugins). Please ensure you only use volatility3.plugins and only ever import plugins from this namespace. This ensures the ability of users to override core plugins without needing write access to the framework directory.

Once the plugins have been imported, we can interrogate which plugins are available. The list_plugins() call will return a dictionary of plugin names and the plugin classes.

plugin_list = framework.list_plugins()

Determine what configuration options a plugin requires

For each plugin class, we can call the classmethod get_requirements() on it, which will return a list of objects that adhere to the RequirementInterface method. The various types of Requirement are split roughly in two, SimpleTypeRequirement (such as integers, booleans, floats and strings) and more complex requirements (such as lists, choices, multiple requirements, translation layer requirements or symbol table requirements). A requirement just specifies a type of data and a name, and must be combined with a configuration hierarchy to have meaning.

List requirements are a list of simple types (integers, booleans, floats and strings), choices must match the available options, multiple requirements needs all their subrequirements fulfilled and the other types require the names of valid translation layers or symbol tables within the context, respectively. Luckily, each of these requirements can tell you whether they’ve been fulfilled or not later in the process. For now, they can be used to ask the user to fill in any parameters they made need to. Some requirements are optional, others are not.

The plugin is essentially a multiple requirement. It should also be noted that automagic classes can have requirements (as can translation layers).

Set the configuration in the context

Once you know what requirements the plugin will need, you can populate them within the context.config. The configuration is essentially a hierarchical tree of values, much like the windows registry. Each plugin is instantiated at a particular branch within the hierarchy and will look for its configuration options under that hierarchy (if it holds any configurable items, it will likely instantiate those at a point underneaths its own branch). To set the hierarchy, you’ll need to know where the configurables will be constructed.

For this example, we’ll assume plugins’ base_config_path is set as plugins, and that automagics are configured under the automagic tree. We’ll see later how to ensure this matches up with the plugins and automagic when they’re constructed. Joining configuration options should always be carried out using path_join() in case the separator value gets changed in the future. Configuration items can then be set as follows:

config_path = path_join(base_config_path, plugin.__class__.__name__, <plugin_parameter>)
context.config['plugins.<plugin_class_name>.<plugin_parameter>'] = value

Using automagic to complete the configuration

Many of the options will require a lot of construction (layers on layers on layers). The automagic functionality is there to help take some of that burden away. There are automagics designed to stack layers (such as compression and file formats, as well as architectures) and automagics for determining critical information from windows, linux and mac layers about the operating system. The list of available automagics can be found using:

available_automagics = automagic.available(ctx)

This again, will require that all automagic modules have been loaded but this should happen simply as part of importing the automagic module. The available list will be pre-instantiated copies of the automagic with their configuration path and context provided (based on constants.AUTOMAGIC_CONFIG_PATH and the automagic class name).

A suitable list of automagics for a particular plugin (based on operating system) can be found using:

automagics = automagic.choose_automagic(available_automagics, plugin)

This will take the plugin module, extract the operating system (first level of the hierarchy) and then return just the automagics which apply to the operating system. Each automagic can exclude itself from being used for specific operating systems, so that an automagic designed for linux is not used for windows or mac plugins.

These automagics can then be run by providing the list, the context, the plugin to be run, the hierarchy name that the plugin will be constructed on (‘plugins’ by default) and a progress_callback. This is a callable which takes a percentage of completion and a description string and will be called throughout the process to indicate to the user how much progress has been made.

errors = automagic.run(automagics, context, plugin, base_config_path, progress_callback = progress_callback)

Any exceptions that occur during the execution of the automagic will be returned as a list of exceptions.

Run the plugin

Firstly, we should check whether the plugin will be able to run (ie, whether the configuration options it needs have been successfully set). We do this as follow (where plugin_config_path is the base_config_path (which defaults to plugins and then the name of the class itself):

unsatisfied = plugin.unsatisfied(context, plugin_config_path)

If unsatisfied is an empty list, then the plugin has been given everything it requires. If not, it will be a Dictionary of the hierarchy paths and their associated requirements that weren’t satisfied.

The plugin can then be instantiated with the context (containing the plugin’s configuration) and the path that the plugin can find its configuration at. This configuration path only needs to be a unique value to identify where the configuration details can be found, similar to a registry key in Windows.

A progress_callback can also be provided to give users feedback whilst the plugin is running. A progress callback is a function (callable) that takes a percentage and a descriptive string. User interfaces implementing these can therefore provide progress feedback to a user, as the framework will call these every so often during intensive actions, to update the user as to how much has been completed so far.

Also, should the plugin produce files, an open_method can be set on the plugin, which will be called whenever a plugin produces an auxiliary file.

constructed = plugin(context, plugin_config_path, progress_callback = progress_callback)
constructed.set_open_method(file_handler)

The file_handler must adhere to the FileHandlerInterface, which represents an IO[bytes] object but also contains a preferred_filename attribute as a hint indicating what the file being produced should be called. When a plugin produces a new file, rather than opening it with the python open method, it will use the FileHandlerInterface and construct it with a descriptive filename, and then write bytes to it using the write method, just like other python file-like objects. This allows web user interfaces to offer the files for download, whilst CLIs to write them to disk and other UIs to handle files however they need.

All of this functionality has been condensed into a framework method called construct_plugin which will take and run the automagics, and instantiate the plugin on the provided base_config_path. It also accepts an optional progress_callback and an optional file_consumer.

constructed = plugins.construct_plugin(ctx, automagics, plugin, base_config_path, progress_callback, file_consumer)

Finally the plugin can be run, and will return a TreeGrid.

treegrid = constructed.run()

Render the TreeGrid

The results are now in a structure of rows, with a hierarchy (allowing a row to be a child of another row).

The TreeGrid can tell you what columns it contains, and the types of each column, but does not contain any data yet. It must first be populated. This actually iterates through the results of the plugin, which may have been provided as a generator, meaning this step may take the actual processing time, whilst the plugin does the actual work. This can return an exception if one occurs during the running of the plugin.

The results can be accessed either as the results are being processed, or by visiting the nodes in the tree once it is fully populated. In either case, a visitor method will be required. The visitor method should accept a TreeNode and an accumulator. It will return an updated accumulator.

When provided a TreeNode, it can be accessed as a dictionary based on the column names that the treegrid contains. It should be noted that each column can contain only the type specified in the column.type field (which can be a simple type like string, integer, float, bytes or a more complex type, like a DateTime, a Disassembly or a descendant of BaseAbsentValue). The various fields may also be wrapped in format_hints designed to tell the user interface how to render the data. These hints can be things like Bin, Hex or HexBytes, so that fields like offsets are displayed in hex form or so that bytes are displayed in their hex form rather than their raw form. Descendants of BaseAbsentValue can currently be one of UnreadableValue, UnparsableValue, NotApplicableValue or NotAvailableValue. These indicate that data could not be read from the memory for some reason, could not be parsed properly, was not applicable or was not available.

A simple text renderer (that returns output immediately) would appear as follows. This doesn’t use the accumulator, but instead uses print to directly produce the output. This is not recommended:

for column in grid.columns:
    print(column.name)

def visitor(node, _accumulator):
    # Nodes always have a path value, giving them a path_depth of at least 1, we use max just in case
    print("*" * max(0, node.path_depth - 1), end = " ")
    for column_index in range(len(grid.columns)):
        column = grid.columns[column_index]
        print(repr(node.values[column_index]), end = '\t')

    print('')
    return None

grid.populate(visitor, None)

More complex examples of renderers can be found in the default CLI implementation, such as the QuickTextRenderer or the PrettyTextRenderer.