Skip to content

Commit

Permalink
Merge pull request #445 from volatilityfoundation/release/v1.0.0
Browse files Browse the repository at this point in the history
Release/v1.0.0
  • Loading branch information
ikelos authored Feb 1, 2021
2 parents 93c6a62 + 2797ac5 commit 0e372b3
Show file tree
Hide file tree
Showing 259 changed files with 1,124 additions and 1,068 deletions.
42 changes: 42 additions & 0 deletions .github/workflows/build-pypi.yml
Original file line number Diff line number Diff line change
@@ -0,0 +1,42 @@
name: build-pypi

on:
push:
branches:
- stable
- develop
- 'release/**'

pull_request:
branches:
- stable
- 'release/**'

jobs:

build:
runs-on: ubuntu-latest
steps:
- uses: actions/checkout@v2

- name: Set up Python 3.x
uses: actions/setup-python@v2
with:
python-version: '3.x'

- name: Install dependencies
run: |
python -m pip install --upgrade pip
pip install setuptools wheel
- name: Build PyPi packages
run: |
python setup.py sdist --formats=gztar,zip
python setup.py bdist_wheel
- name: Archive dist
uses: actions/upload-artifact@v2
with:
name: volatility3-pypi
path: |
dist/
10 changes: 5 additions & 5 deletions .gitignore
Original file line number Diff line number Diff line change
Expand Up @@ -4,7 +4,7 @@ __pycache__

# Sphinx generated files
doc/build
doc/source/volatility.*
doc/source/volatility3.*

# Pycharm/PyDev development files
workspace.xml
Expand All @@ -14,14 +14,14 @@ workspace.xml
# Manually generated files
.mypy_cache
stubs
volatility/symbols/linux*
volatility/symbols/windows*
volatility/symbols/mac*
volatility3/symbols/linux*
volatility3/symbols/windows*
volatility3/symbols/mac*

# Mac metadata files
.DS_Store

# Volatility's config
# Volatility's config
config*.json

# Pyinstaller files
Expand Down
4 changes: 2 additions & 2 deletions MANIFEST.in
Original file line number Diff line number Diff line change
Expand Up @@ -2,5 +2,5 @@ prune development
include * .*
include doc/make.bat doc/Makefile
recursive-include doc/source *
recursive-include volatility *.json
recursive-exclude doc/source volatility*.rst
recursive-include volatility3 *.json
recursive-exclude doc/source volatility3.*.rst
2 changes: 1 addition & 1 deletion README.md
Original file line number Diff line number Diff line change
Expand Up @@ -77,7 +77,7 @@ The hashes to verify whether any of the symbol pack files have downloaded succes
<https://downloads.volatilityfoundation.org/volatility3/symbols/SHA1SUMS>
<https://downloads.volatilityfoundation.org/volatility3/symbols/MD5SUMS>

Symbol tables zip files must be placed, as named, into the `volatility/symbols` directory (or just the symbols directory next to the executable file).
Symbol tables zip files must be placed, as named, into the `volatility3/symbols` directory (or just the symbols directory next to the executable file).

Windows symbols that cannot be found will be queried, downloaded, generated and cached. Mac and Linux symbol tables must be manually produced by a tool such as [dwarf2json](https://github.com/volatilityfoundation/dwarf2json).

Expand Down
2 changes: 1 addition & 1 deletion development/schema_validate.py
Original file line number Diff line number Diff line change
Expand Up @@ -17,7 +17,7 @@
logger.addHandler(console)
logger.setLevel(logging.DEBUG)

from volatility import schemas
from volatility3 import schemas

if __name__ == '__main__':
parser = argparse.ArgumentParser("Validates ")
Expand Down
2 changes: 1 addition & 1 deletion doc/Makefile
Original file line number Diff line number Diff line change
Expand Up @@ -15,7 +15,7 @@ help:

# Ensure we clean the autodoc files when we clean
clean:
-rm source/volatility.*.rst
-rm source/volatility3.*.rst
@$(SPHINXBUILD) -M clean "$(SOURCEDIR)" "$(BUILDDIR)" $(SPHINXOPTS) $(O)

# Catch-all target: route all unknown targets to Sphinx using the new
Expand Down
56 changes: 28 additions & 28 deletions doc/source/basics.rst
Original file line number Diff line number Diff line change
Expand Up @@ -7,7 +7,7 @@ Volatility splits memory analysis down to several components:
* Templates and Objects
* Symbol Tables

Volatility 3 stores all of these within a :py:class:`Context <volatility.framework.interfaces.context.ContextInterface>`,
Volatility 3 stores all of these within a :py:class:`Context <volatility3.framework.interfaces.context.ContextInterface>`,
which acts as a container for all the various layers and tables necessary to conduct memory analysis.

Memory layers
Expand All @@ -21,8 +21,8 @@ two other sources. These are typically handled by programs that process file fo
processor, but these are all translations (either in the geometric or linguistic sense) of the original data.

In Volatility 3 this is represented by a directed graph, whose end nodes are
:py:class:`DataLayers <volatility.framework.interfaces.layers.DataLayerInterface>` and whose internal nodes are
specifically called a :py:class:`TranslationLayer <volatility.framework.interfaces.layers.TranslationLayerInterface>`.
:py:class:`DataLayers <volatility3.framework.interfaces.layers.DataLayerInterface>` and whose internal nodes are
specifically called a :py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`.
In this way, a raw memory image in the LiME file format and a page file can be
combined to form a single Intel virtual memory layer. When requesting addresses from the Intel layer, it will use the
Intel memory mapping algorithm, along with the address of the directory table base or page table map, to translate that
Expand All @@ -39,17 +39,17 @@ Templates and Objects

Once we can address contiguous chunks of memory with a means to translate a virtual address (as seen by the programs)
into the actual data used by the processor, we can start pulling out
:py:class:`Objects <volatility.framework.interfaces.objects.ObjectInterface>` by taking a
:py:class:`~volatility.framework.interfaces.objects.Template` and constructing
it on the memory layer at a specific offset. A :py:class:`~volatility.framework.interfaces.objects.Template` contains
:py:class:`Objects <volatility3.framework.interfaces.objects.ObjectInterface>` by taking a
:py:class:`~volatility3.framework.interfaces.objects.Template` and constructing
it on the memory layer at a specific offset. A :py:class:`~volatility3.framework.interfaces.objects.Template` contains
all the information you can know about the structure of the object without actually being populated by any data.
As such a :py:class:`~volatility.framework.interfaces.objects.Template` can tell you the size of a structure and its
As such a :py:class:`~volatility3.framework.interfaces.objects.Template` can tell you the size of a structure and its
members, how far into the structure a particular member lives and potentially what various values in that field would
mean, but not what resides in a particular member.

Using a :py:class:`~volatility.framework.interfaces.objects.Template` on a memory layer at a particular offset, an
:py:class:`Object <volatility.framework.interfaces.objects.ObjectInterface>` can be constructed. In Volatility 3, once an
:py:class:`Object <volatility.framework.interfaces.objects.ObjectInterface>` has been created, the data has been read from the
Using a :py:class:`~volatility3.framework.interfaces.objects.Template` on a memory layer at a particular offset, an
:py:class:`Object <volatility3.framework.interfaces.objects.ObjectInterface>` can be constructed. In Volatility 3, once an
:py:class:`Object <volatility3.framework.interfaces.objects.ObjectInterface>` has been created, the data has been read from the
layer and is not read again. An object allows its members to be interrogated and in particular allows pointers to be
followed, providing easy access to the data contained in the object.

Expand All @@ -62,23 +62,23 @@ Symbol Tables
-------------

Most compiled programs know of their own templates, and define the structure (and location within the program) of these
templates as a :py:class:`Symbol <volatility.framework.interfaces.symbols.SymbolInterface>`. A
:py:class:`Symbol <volatility.framework.interfaces.symbols.SymbolInterface>` is often an address and a template and can
templates as a :py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>`. A
:py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>` is often an address and a template and can
be used to refer to either independently. Lookup tables of these symbols are often produced as debugging information
alongside the compilation of the program. Volatility 3 provides access to these through a
:py:class:`SymbolTable <volatility.framework.interfaces.symbols.SymbolTableInterface>`, many of which can be collected
within a :py:class:`~volatility.framework.contexts.Context` as a :py:class:`SymbolSpace <volatility.framework.interfaces.symbols.SymbolSpaceInterface>`.
A :py:class:`~volatility.framework.contexts.Context` can store only one :py:class:`~volatility.framework.symbols.SymbolSpace`
at a time, although a :py:class:`~volatility.framework.symbols.SymbolSpace` can store as
many :py:class:`~volatility.framework.symbols.SymbolTable` items as necessary.
:py:class:`SymbolTable <volatility3.framework.interfaces.symbols.SymbolTableInterface>`, many of which can be collected
within a :py:class:`~volatility3.framework.contexts.Context` as a :py:class:`SymbolSpace <volatility.framework.interfaces.symbols.SymbolSpaceInterface>`.
A :py:class:`~volatility3.framework.contexts.Context` can store only one :py:class:`~volatility.framework.symbols.SymbolSpace`
at a time, although a :py:class:`~volatility3.framework.symbols.SymbolSpace` can store as
many :py:class:`~volatility3.framework.symbols.SymbolTable` items as necessary.

Volatility 3 uses the de facto naming convention for symbols of `module!symbol` to refer to them. It reads them from its
own JSON formatted file, which acts as a common intermediary between Windows PDB files, Linux DWARF files, other symbol
formats and the internal Python format that Volatility 3 uses to represent
a :py:class:`~volatility.framework.interfaces.objects.Template` or
a :py:class:`Symbol <volatility.framework.interfaces.symbols.SymbolInterface>`.
a :py:class:`~volatility3.framework.interfaces.objects.Template` or
a :py:class:`Symbol <volatility3.framework.interfaces.symbols.SymbolInterface>`.

.. note:: Volatility 2's name for a :py:class:`~volatility.framework.symbols.SymbolSpace` was a profile, but it could
.. note:: Volatility 2's name for a :py:class:`~volatility3.framework.symbols.SymbolSpace` was a profile, but it could
not differentiate between symbols from different modules and required special handling for 32-bit programs that
used Wow64 on Windows. This meant that all symbols lived in a single namespace with the possibility of symbol name
collisions. It read the symbols using a format called *vtypes*, written in Python code directly.
Expand All @@ -88,18 +88,18 @@ Plugins
-------

A plugin acts as a means of requesting data from the user interface (and so the user) and then using it to carry out a
specific form of analysis on the :py:class:`Context <volatility.framework.interfaces.context.ContextInterface>`
specific form of analysis on the :py:class:`Context <volatility3.framework.interfaces.context.ContextInterface>`
(containing whatever symbol tables and memory layers it may). The means of communication between the user interface and
the library is the configuration tree, which is used by components within the :py:class:`~volatility.framework.contexts.Context`
the library is the configuration tree, which is used by components within the :py:class:`~volatility3.framework.contexts.Context`
to store configurable data. After the plugin has been run, it then returns the results in a specific format known as a
:py:class:`~volatility.framework.interfaces.renderers.TreeGrid`. This ensures that the data can be handled by consumers of
:py:class:`~volatility3.framework.interfaces.renderers.TreeGrid`. This ensures that the data can be handled by consumers of
the library, without knowing exactly what the data is or how it's formatted.

Output Renderers
----------------

User interfaces can choose how best to present the output of the results to their users. The library always responds from
every plugin with a :py:class:`~volatility.framework.renderers.TreeGrid`, and the user interface can then determine how
every plugin with a :py:class:`~volatility3.framework.renderers.TreeGrid`, and the user interface can then determine how
best to display it. For the Command Line Interface, that might be via text output as a table, or it might output to an
SQLite database or a CSV file. For a web interface, the best output is probably as JSON where it could be displayed as
a table, or inserted into a database like Elastic Search and trawled using an existing frontend such as Kibana.
Expand All @@ -111,9 +111,9 @@ Configuration Tree
------------------

The configuration tree acts as the interface between the calling program and Volatility 3 library. Elements of the
library (such as a :py:class:`Plugin <volatility.framework.interfaces.plugins.PluginInterface>`,
a :py:class:`TranslationLayer <volatility.framework.interfaces.layers.TranslationLayerInterface>`,
an :py:class:`Automagic <volatility.framework.interfaces.automagic.AutomagicInterface>`, etc.) can use the configuration
library (such as a :py:class:`Plugin <volatility3.framework.interfaces.plugins.PluginInterface>`,
a :py:class:`TranslationLayer <volatility3.framework.interfaces.layers.TranslationLayerInterface>`,
an :py:class:`Automagic <volatility3.framework.interfaces.automagic.AutomagicInterface>`, etc.) can use the configuration
tree to inform the calling program of the options they require and/or optionally support, and allows the calling program
to provide that information when the library is then called.

Expand All @@ -122,7 +122,7 @@ Automagic

There are certain setup tasks that establish the context in a way favorable to a plugin before it runs, removing
several tasks that are repetitive and also easy to get wrong. These are called
:py:class:`Automagic <volatility.framework.interfaces.automagic.AutomagicInterface>`, since they do things like magically
:py:class:`Automagic <volatility3.framework.interfaces.automagic.AutomagicInterface>`, since they do things like magically
taking a raw memory image and automatically providing the plugin with an appropriate Intel translation layer and an
accurate symbol table without either the plugin or the calling program having to specify all the necessary details.

Expand Down
Loading

0 comments on commit 0e372b3

Please sign in to comment.