Sandboxed Execution Environment¶
Sandboxed Execution Environment provides a secure framework for test automation of software which behaviour is unknown and must be investigated. Aim of SEE is to empower its users allowing them to quickly build automated test cases in order to keep up with the progressing speed of software evolution.
SEE is very well suited for automating malware analysis, vulnerability discovery and software behavioural analysis in general.
Rationale¶
Analysing the execution of unknown software has become a quite complicated matter. The amount of Operating Systems and libraries to support is growing at an incredible speed. Developing and operating automated behavioural analysis platforms requires several different areas of expertise. The co-operation of such fields is critical for the effectiveness of the end product.
F-Secure developed its first prototype for an automated behavioural analysis platform in 2005. The expertise acquired over the years led to the need for a better approach in building such technologies. Rather than a single and monolithic platform trying to cover all the possible scenarios, a family of specifically tailored ones seemed a more reasonable approach for the analysis of unknown software.
Sandboxed Execution Environment has been built to enable F-Secure malware experts to quickly prototype and develop behavioural analysis engines.
The technology consists of few well known design patterns enclosed in a small framework. With SEE is possible to quickly deploy a Sandbox and attach different plugins to control it. The overall design allows to build highly flexible, robust and relatively safe platforms for test automation.
Tutorial¶
This tutorial will give an introduction on how to setup an example analysis environment using SEE.
The environment will comprise a Windows 7 Operating System virtualized using QEMU/KVM on a Debian derived distribution (Debian Stretch).
Installation¶
Please refer to the installation documentation if problems are encountered.
Requirements¶
The following base Debian packages are required.
- python3
- python3-pip
The following Debian packages provide the core virtualization technologies.
- qemu
- qemu-tools
- dnsmasq
- virt-viewer
- virtinst
- python3-libvirt
These Debian packages are required in the Tutorial’s last chapters.
- tshark
- volatility
- virt-manager
- python3-guestfs
- python3-requests
This Python package is required in the Tutorial’s last chapters.
Set up¶
SEE can be installed using pip.
# pip3 install python-see
For correct operation, the user which will run the analysis platform will need to be part of the following groups.
- kvm
- libvirt
- libvirt-qemu
The following command allows to add a user to a group.
# adduser <username> <group>
Make sure hardware acceleration is supported.
To verify that KVM is available it is enough to run:
# modprobe kvm
# modprobe kvm_intel
for Intel processors or:
# modprobe kvm
# modprobe kvm_amd
for AMD ones.
Environment setup¶
Preparing the Guest Image¶
In order to perform our tests, we need to provide a copy of a Windows Operating System to be executed within a Virtual Machine.
Microsoft made available for downloading ready-made virtual machine disk images containing Windows 7.
These images are for testing purposes only and they do not allow samples analysis at scale. Yet they are a good starting point for our analysis environment.
Unfortunately, there are no QCOW2 copies available. Even though QEMU allows to execute other disk formats, it is highly recommended to convert the downloaded disk image in QCOW2 format. The VMWare platform is the one used in the Tutorial.
If the downloaded ZIP archive contains an OVA file, the following command will successfully unpack it.
$ tar -xvf "IE8 - Win7.ova"
IE8 - Win7.ovf
IE8 - Win7-disk1.vmdk
The OVF file can be automatically converted and imported using virt-convert tool.
$ mkdir /home/username/images
$ virt-convert --disk-format qcow2 --destination /home/username/images --connect qemu:///system "./IE8 - Win7.ovf"
If successful, the User will be prompted to a VNC connection where the Operating System will be booting. Make sure the drivers get automatically installed (it might require to reboot the OS).
In case of issues, a suggested solution is replacing the video adapter of the virtual machine.
$ virsh -c qemu:///system edit IE8_-_Win7.ovf
Replace the xml attribute type of the field model in the video section from qxl to cirrus.
Once done, ensure basic things such as Internet connection are working.
Proceed with the desired customisation. As an additional action, install Python3 within the OS. This is not strictly necessary but will help later in the Tutorial.
Make sure the virtualized OS is correctly shut down once done.
The Virt Manager tool can help to configure and set up the virtual machine.
Configuring SEE¶
SEE requires few configuration files to correcly operate.
The libvirt XML configuration related to the imported Windows image must be stored somewhere.
$ virsh --connect qemu:///system list --all
Id Name State
- IE8_-_Win7.ovf shut off
$ virsh --connect qemu:///system dumpxml IE8_-_Win7.ovf >> /home/username/windows7.xml
Then, a JSON file will be used to describe the Context configuration.
/home/username/context.json
{
"domain":
{
"configuration": "/home/username/windows7.xml"
},
"disk":
{
"image":
{
"uri": "/home/username/images/IE8_-_Win7-disk1.qcow2",
"provider": "see.image_providers.DummyProvider"
},
"clone":
{
"storage_pool_path": "/home/username/instances",
"copy_on_write": true
}
}
}
The following configuration will instruct SEE to create a virtual machine (a domain) using the above mentioned libvirt configuration file.
The domain will use the disk image we just converted. The clone field, instructs SEE to make a clone of the given disk. In this way, the running tests will not change or affect the base disk image. The clone will be located in storage_pool_path and it will be few Mb in size as copy_on_write is enabled.
In case we want to run multiple sandboxes concurrently, might be a good idea to isolate their networks. The documentation illustrates how to do so.
Hello Sandbox¶
Now that the basics are ready, we can start building our automated test environment.
Core elements of SEE consist of the Environment, which takes care of resources management, and the Context which wraps the sandbox allowing to control its lifecycle.
The following example shows the basic workflow. Once initialized the Context, we power on the sandbox, we let it run for one minute and then we power it off.
import time
from see import Environment
from see.context import QEMUContextFactory
def main():
context_factory = QEMUContextFactory('/home/username/context.json')
with Environment(context_factory, {}) as environment:
context = environment.context
context.poweron()
time.sleep(60)
context.poweroff()
if __name__ == '__main__':
main()
SEE takes care of the resources allocation/deallocation as well as for their isolation. This script can be executed multiple times resulting in multiple test environment running concurrently.
Once the execution ends, no trace will remain of the virtual machine.
Controlling the Sandbox¶
Every proper tutorial includes the “Hello World” example, we cannot be outdone.
To better control the execution flow, SEE provides an event-driven architecture allowing to trigger Events and subscribe handlers through the Context object.
In the next code snippet, we print the “Hello Sandbox” string after starting the sandbox. At the same time, we open a VNC connection to inspect its execution.
To complete the script, we turn it into a command line tool complete with parameters.
#!/usr/bin/env python3
import time
import argparse
import subprocess
from see import Environment
from see.context import QEMUContextFactory
def hello_sandbox_handler(event):
print("Hello Sandbox!")
def vnc_handler(event):
command = ('virt-viewer', '--connect', 'qemu:///system', event.identifier)
subprocess.call(command)
def main():
arguments = parse_arguments()
context_factory = QEMUContextFactory(arguments.context)
with Environment(context_factory, {}) as environment:
context = environment.context
context.subscribe('vm_started', hello_sandbox_handler)
# asynchronous handlers do not block the execution
# when triggering the Event
context.subscribe_async('vm_started', vnc_handler)
context.poweron()
# the Environment ID is appended to the event as extra information
context.trigger('vm_started', identifier=environment.identifier)
time.sleep(60)
context.poweroff()
def parse_arguments():
parser = argparse.ArgumentParser(description='Run a Sandbox.')
parser.add_argument('context', help='path to Context JSON configuration')
return parser.parse_args()
if __name__ == '__main__':
main()
The above logic subscribes through the Context the handlers hello_sandbox_handler and vnc_handler to the vm_started Event. The Event is then triggered right after powering on the Context.
Once launched the script, a VNC session will be automatically opened showing us the guest OS. On the log we should see the “Hello Sandbox!” text.
$ ./sandbox.py context.json
Hello Sandbox!
Plugins and Protocol¶
In the previous chapter we created a Sandbox with a minimal set of functionalities.
As the number of features grows, so does the complexity of the system. SEE has been designed to contain the development cost providing high flexibility and promoting code re-usability.
As shown previously, the Context allows to subscribe event handlers and trigger Events. This is commonly referred as the Observer pattern.
Rather than adding additional features, let’s try to encapsulate the ones we already provided in a re-usable module.
The Plugins¶
If the Context is an observable, the Hooks or plugins play the observer’s role. They can register their handlers to the desired Events using them as a mean of synchronisation.
The Environment class accepts an arbitrary amount of Hooks. Once initialized, each Hook receives a reference to the Context and its own specific configuration.
Here follows as an example the VNC Hook. To make it more generic, we allow the User to configure at which Event to open the VNC connection.
"""VNC Hook.
Opens a VNC connection with the Guest at the given event.
Configuration:
{
"start_vnc": "event_at_which_starting_vnc"
}
"""
import subprocess
from see import Hook
class VNCHook(Hook):
def __init__(self, parameters):
super().__init__(parameters)
if 'start_vnc' in self.configuration:
self.context.subscribe_async(self.configuration['start_vnc'], self.vnc_handler)
def vnc_handler(self, event):
self.logger.info("Event %s: starting VNC connection.", event)
command = ('virt-viewer', '--connect', 'qemu:///system', self.identifier)
subprocess.call(command)
The Protocol¶
In the natural sciences a protocol is a predefined written procedural method in the design and implementation of experiments. When writing test cases to analyse unknown samples, we define a protocol as a sequence of instructions affecting the Context and leading to several Events to be triggered.
With a well defined set of protocols and a collection of independent plugins, we can assemble a vaste amount of test cases without the need of writing any specific code.
The protocol will act on the Context which will be observed by the Hooks and the Events will be the transport mechanism for such protocol.
TIMEOUT = 60
RUNTIME = 60
def protocol(context, sample_path, execution_command):
context.poweron()
wait_for_ip_address(context, TIMEOUT)
context.trigger('run_sample', sample=sample_path, command=execution_command)
time.sleep(RUNTIME)
context.pause()
context.trigger('snapshots_capture')
context.resume()
context.shutdown()
context.trigger('start_analysis')
context.trigger('wait_analysis')
def wait_for_ip_address(context, timeout):
timestamp = time.time()
while time.time() - timestamp < timeout:
if context.ip_address is not None:
context.trigger('ip_address', address=context.ip_address)
return
time.sleep(1)
raise TimeoutError("Waiting for IP address")
The above example is pretty simple to understand. After powering on the Sandbox, we wait for its IP address to be ready. We then inject the sample and let it run for a given amount of time. We notify the plugins that the VM is about to be shut down letting them capture any necessary information. Once powered off the VM, we proceed analysing the gathered information.
The Event sequence is the following.
Triggered by the Context.poweron method:
- pre_poweron
- post_poweron
Triggered by the wait_for_ip_address function once the IP address is available:
- ip_address
Triggered in order to start start the sample:
- run_sample
Triggered by the Context.pause method:
- pre_pause
- post_pause
Triggered in order to take snapshots of the virtual machine state:
- snapshots_capture
Triggered by the Context.resume methods:
- pre_resume
- post_resume
Triggered by the Context.shutdown method:
- pre_shutdown
- post_shutdown
Triggered in order to start analysis plugins:
- start_analysis
- wait_analysis
To conclude the chapter, we show the new script. The sample path, its execution command as well as the Hooks configuration path have been parametrised.
Refer to the Documetation to configure the Hooks.
#!/usr/bin/env python3
import time
import argparse
from see import Environment
from see.context import QEMUContextFactory
TIMEOUT = 60
RUNTIME = 60
def main():
arguments = parse_arguments()
context_factory = QEMUContextFactory(arguments.context)
with Environment(context_factory, arguments.hooks) as environment:
protocol(environment.context, arguments.sample, arguments.command)
def protocol(context, sample_path, execution_command):
context.poweron()
wait_for_ip_address(context, TIMEOUT)
context.trigger('run_sample', sample=sample_path, command=execution_command)
time.sleep(RUNTIME)
context.trigger('snapshots_capture')
context.poweroff()
context.trigger('start_analysis')
context.trigger('wait_analysis')
def wait_for_ip_address(context, timeout):
timestamp = time.time()
while time.time() - timestamp < timeout:
if context.ip4_address is not None:
context.trigger('ip_address', address=context.ip4_address)
return
raise TimeoutError("Waiting for IP address")
def parse_arguments():
parser = argparse.ArgumentParser(description='Execute a sample within a Sandbox.')
parser.add_argument('context', help='path to Context JSON configuration')
parser.add_argument('sample', help='path to Sample to execute')
parser.add_argument('-k', '--hooks', default={}, help='path to Hooks JSON configuration')
parser.add_argument('-c', '--command', default='start {sample}',
help="""command used to start the sample.
The string {sample} will be expanded to the actual file name within the guest.
Example: 'notepad.exe {sample}'""")
return parser.parse_args()
if __name__ == '__main__':
main()
Sample Execution¶
In order to study the behaviour of an unknown sample, we need to automate its injection and execution. To do so, we will use a Python HTTP server running as an agent within the Guest Operating System.
The Agent¶
As specified in the setup chapter, Python 3 is required in order to run the agent.
The agent code can be fount at this link.
The easiest way to ensure its execution at the Windows startup is to place a batch script like the following in the startup folder.
C:\path\to\the\Python\Installation\python.exe C:\path\to\the\agent\agent.py 0.0.0.0 8080
The startup folder is usually located at the following path:
C:\Microsoft\Windows\Start Menu\Programs\Startup
The script will listen to the port 8080, make sure the firewall will allow traffic through such port.
At reboot, a command prompt should show the agent log. In case of issues, add the —debug parameter to the agent to better follow the exchanged messages.
The Commands Hook¶
The Commands Hook has been designed to be used altogether with the agent.
The only configuration parameter is the agent port. The Hook will automatically detect the IP address of the Sandbox. To add it to the analysis environment, add the following snippet to the Hooks configuration.
{
"name": "plugins.commands.CommandsHook",
"configuration": {
"agent-port": 8080
}
}
We can now run the script of the previous chapter as follows.
$ ./sandbox.py context.json example.exe --hooks hooks.json --command "start {sample}"
Hello Malware¶
Now that all the pieces are in place, we can finally proceed with the analysis of our first malicious sample.
Please bear in mind that executing unknown software is dangerous. Sandboxing provides a mean of isolation which is all but effective against specifically crafted code. Other implied risks derive from User’s misconfigurations. Furthermore, the explicit handling and execution of malicious software might be restricted by the law.
Analysis Scenario¶
The sample we are about to execute is a simple ransomware. Ransomware are good analysis candidates because they show a quite complete set of malicious behaviour.
We will adopt the execution Protocol shown in the chapter Plugins and Protocol. In addition to that, we will use some of the provided example plugins.
It might be necessary to apply further changes within the Guest Operating System. Services such as the Windows Firewall and Windows Defender might prevent the samples from showing their full behaviour.
Behavioural analysis¶
The behavioural analysis is usually divided in two stages. The first stage takes care of acquiring the behavioural data while the Sandbox is executing. The second stage, which will take place at the end of the Sandbox execution, will analyse the collected data and produce a report.
Disk behaviour¶
The disk behaviour is gathered by taking two snapshots of the disk, before and after the sample execution. Once done, we compare the two snapshots extracting all the occurred changes.
The disk behaviour logic is contained within the disk plugin. The DiskCheckPointHook takes care of collecting the disk snapshots. We will configure it to take two snapshots at the ip_address and at the post_poweroff Events occurrence. This will allows a good coverage of the changes reducing a bit the noise. The DiskStateAnalyser will take care of analysing the snapshots. As the analysis is asynchronous, we need to specify when to start it and when to wait for its conclusion. The protocol’s start_analysis and wait_analysis Events suit our needs.
The report will be stored in the path given as results_folder with the file name filesystem.json. It will contain information about created, deleted and modified files and Windows registry keys.
Memory behaviour¶
The memory behaviour is gathered by taking a memory snapshot of the running virtual machine. The memory snapshot will be analysed using Volatility.
The memory behaviour logic is contained within the memory plugin. The MemoryHook takes care of collecting the memory snapshots. We will configure it to take a single snapshot at the snapshots_capture Event occurrence. The VolatilityHook will take care of analysing the snapshot. As for the Disk analysis hook, the analysis is asynchronous.
Furthermore, the VolatilityHook requires a profile to be specified and a set of scanning plugins to use. As we are running a Windows 7 32 bit, the Win7SP1x86 profile is the one to be used. For the plugins, we can select the mutantscan and the psscan ones.
The VolatilityHook will generate a report file per each Volatility plugin. The reports will be stored in the path given as results_folder.
Network behaviour¶
The network behaviour is gathered by tracing the network traffic of the running virtual machine. The network trace will be analysed using Tshark.
The network behaviour logic is contained within the network plugin. The NetworkTracerHook takes care of tracing the network traffic. We will configure it to start tracing at the ip_address Event occurrence. The NetworkAnalysisHook will take care of analysing the network trace. As for the Disk and Memory analysis hooks, the analysis is asynchronous.
The report will be stored in the path given as results_folder with the file name network.log. It will contain the list of exchanged packets.
Screenshot¶
We can take multiple screenshots during the execution of the Sandbox.
The screenshot plugin will take care of that.
Results¶
The sample is executed using the following command.
$ ./sandbox.py context.json sample.exe --hooks hooks.json
This particular variant takes longer than usual to execute. More than six minutes were required before getting the ransom request prompted.
Disk behaviour¶
When analyzing the created files, we can notice how the executable gets dropped in multiple locations.
...
{
"size": 708608,
"path": "C:\\Users\\Public\\Music\\quozilml.exe",
"type": "PE32 executable (GUI) Intel 80386, for MS Windows",
"sha1": "ec901b94061d27bb90e61360bffca2d409f83cca"
},
...
{
"size": 708608,
"path": "C:\\Users\\IEUser\\Documents\\rquyvgdk.exe",
"type": "PE32 executable (GUI) Intel 80386, for MS Windows",
"sha1": "ec901b94061d27bb90e61360bffca2d409f83cca"
},
...
As well, the sample creates multiple Windows Tasks to ensure the sample execution at startup.
...
{
"size": 274,
"path": "C:\\Windows\\Tasks\\pwknbvap.job",
"type": "VAX-order 68k Blit mpx/mux executable",
"sha1": "4591c704aa8d5c4a96dcdf869bba962ba380d603"
},
{
"size": 274,
"path": "C:\\Windows\\Tasks\\qlyecptc.job",
"type": "VAX-order 68k Blit mpx/mux executable",
"sha1": "7cbaf7b0474197eb7a116eb34eca7da01861a67a"
},
...
For the same purpose, multiple Windows registry keys are created.
...
"HKCU\\Software\\Microsoft\\Windows\\CurrentVersion\\Run": [
...
[
"pwknbvap",
"REG_SZ",
"C:\\Users\\IEUser\\Music\\hthdcemv.exe"
],
...
[
"qlyecptc",
"REG_SZ",
"C:\\Users\\Public\\Music\\quozilml.exe"
]
],
Decryption software and related instructions.
...
{
"size": 708608,
"path": "C:\\Users\\Public\\Documents\\!!!ForDecrypt!!!.exe",
"type": "PE32 executable (GUI) Intel 80386, for MS Windows",
"sha1": "ec901b94061d27bb90e61360bffca2d409f83cca"
},
{
"size": 2892,
"path": "C:\\Users\\IEUser\\Desktop\\ReadMeFilesDecrypt!!!.txt",
"type": "data",
"sha1": "d8816e1bf72194dd8fe3951a449bc4046a4d818b"
},
...
The User’s documents are modified in place, without changing their filename. Note the different SHA1 hashes and the data file type which indicates the content cannot be identified as it’s encrypted.
{
"size": 778032,
"path": "C:\\Users\\Public\\Pictures\\Sample Pictures\\Koala.jpg",
"original_sha1": "9c3dcb1f9185a314ea25d51aed3b5881b32f420c",
"sha1": "8d8e750d75d0ec4cb4de0ecabe21e2efd90fc662",
"type": "data"
},
Memory behaviour¶
Volatility’s psscan plugin reports the process as running.
Volatility Foundation Volatility Framework 2.6
Offset(P) Name PID PPID PDB Time created Time exited
------------------ ---------------- ------ ------ ---------- ------------------------------ ------------------------------
...
0x000000003c34d1a8 sample.exe 2152 776 0x3c3484c0 2017-01-15 21:57:30 UTC+0000
...
The same process (note the PID) owns a Mutex with a singular name. Mutexes are a very effective way to identify malware families.
Volatility Foundation Volatility Framework 2.6
Offset(P) #Ptr #Hnd Signal Thread CID Name
------------------ -------- -------- ------ ---------- --------- ----
...
0x000000003c35b4a0 2 1 0 0x85bb7d48 2152:2156 HelloWorldItsJokeFromMars
...
Network behaviour¶
Observing the network log, we can see how the malware relies on TOR protocol to communicate with the C&C server. Not only the TOR protocol encrypts the communication but also anonymizes both the C&C and the infected hosts making harder to identify the victims over the network.
...
13 1.047625677 192.168.3.66 → 192.168.3.1 DNS 87 Standard query 0x0dd2 A buxnfuoim27a3yvh.onion.link
16 1.130895451 192.168.3.1 → 192.168.3.66 DNS 103 Standard query response 0x0dd2 A buxnfuoim27a3yvh.onion.link A 103.198.0.2
...
22 1.264814158 192.168.3.66 → 103.198.0.2 HTTP 251 POST / HTTP/1.1 (application/x-www-form-urlencoded)
Conclusion¶
The automated analysis environment presented in the Tutorial is far from complete. Aim of the Tutorial was to show the potential of the SEE framework.
SEE aims to simplify the development of sandbox based behavioural analysis platforms focusing in providing multiple sandboxing technologies and a simple event driven architecture.
The effectiveness of the platform in identifying threats and analysing unknown software is in the plugins and the protocols provided by the developers. With a comprehensive collection of plugins, the User will be able to quickly assemble his or her own test cases without the need of writing any further logic.
A small set of reference plugins is available at this link.
Resources¶
Here follows the source code and configuration used in the examples.
sandbox.py
#!/usr/bin/env python3
import time
import argparse
from see import Environment
from see.context import QEMUContextFactory
TIMEOUT = 60
RUNTIME = 600
def main():
arguments = parse_arguments()
context_factory = QEMUContextFactory(arguments.context)
with Environment(context_factory, arguments.hooks) as environment:
protocol(environment.context, arguments.sample, arguments.command)
def protocol(context, sample_path, execution_command):
context.poweron()
wait_for_ip_address(context, TIMEOUT)
context.trigger('run_sample', sample=sample_path, command=execution_command)
time.sleep(RUNTIME)
context.pause()
context.trigger('snapshots_capture')
context.resume()
context.shutdown(timeout=TIMEOUT)
context.trigger('start_analysis')
context.trigger('wait_analysis')
def wait_for_ip_address(context, timeout):
timestamp = time.time()
while time.time() - timestamp < timeout:
if context.ip4_address is not None:
context.trigger('ip_address', address=context.ip4_address)
return
time.sleep(1)
raise TimeoutError("Waiting for IP address")
def parse_arguments():
parser = argparse.ArgumentParser(description='Execute a sample within a Sandbox.')
parser.add_argument('context', help='path to Context JSON configuration')
parser.add_argument('sample', help='path to Sample to execute')
parser.add_argument('-k', '--hooks', default={}, help='path to Hooks JSON configuration')
parser.add_argument('-c', '--command', default='start {sample}',
help="""command used to start the sample.
The string {sample} will be expanded to the actual file name within the guest.
Example: 'notepad.exe {sample}'""")
return parser.parse_args()
if __name__ == '__main__':
main()
vnc.py
"""VNC Hook.
Opens a VNC connection with the Guest at the given event.
Configuration:
{
"start_vnc": "event_at_which_starting_vnc"
}
"""
import subprocess
from see import Hook
class VNCHook(Hook):
def __init__(self, parameters):
super().__init__(parameters)
if 'start_vnc' in self.configuration:
self.context.subscribe_async(
self.configuration['start_vnc'], self.vnc_handler)
def vnc_handler(self, event):
self.logger.info("Event %s: starting VNC connection.", event)
command = ('virt-viewer', '--connect',
'qemu:///system', self.identifier)
subprocess.call(command)
hooks.json
{
"configuration":
{
"results_folder": "/home/username/results/"
},
"hooks":
[
{
"name": "vnc.VNCHook",
"configuration": {
"start_vnc": "post_poweron"
}
},
{
"name": "plugins.screen.ScreenHook",
"configuration": {
"screenshot_on_event": ["snapshots_capture"]
}
},
{
"name": "plugins.commands.CommandsHook",
"configuration": {
"agent-port": 8080
}
},
{
"name": "plugins.disk.DiskCheckPointHook",
"configuration": {
"checkpoint_on_event": ["ip_address", "post_shutdown"],
"delete_checkpoints": true
}
},
{
"name": "plugins.disk.DiskStateAnalyser",
"configuration": {
"identify_files": true,
"get_file_size": true,
"extract_files": false,
"use_concurrency": true,
"compare_registries": true,
"start_processing_on_event": "start_analysis",
"wait_processing_on_event": "wait_analysis"
}
},
{
"name": "plugins.memory.MemoryHook",
"configuration": {
"memory_snapshots_on_event": ["snapshots_capture"],
"delete_snapshots": true
}
},
{
"name": "plugins.memory.VolatilityHook",
"configuration": {
"start_processing_on_event": "start_analysis",
"wait_processing_on_event": "wait_analysis",
"profile": "Win7SP1x86",
"plugins": ["mutantscan", "psscan"]
}
},
{
"name": "plugins.network.NetworkTracerHook",
"configuration": {
"start_trace_on_event": "ip_address",
"stop_trace_on_event": "post_shutdown",
"delete_trace_file": true
}
},
{
"name": "plugins.network.NetworkAnalysisHook",
"configuration": {
"start_processing_on_event": "start_analysis",
"wait_processing_on_event": "wait_analysis",
"log_format": "text"
}
}
]
}
context.json
{
"hypervisor": "qemu:///system",
"domain":
{
"configuration": "/home/username/windows7.xml"
},
"disk":
{
"image":
{
"uri": "/home/username/images/IE8_-_Win7-disk1.qcow2",
"provider": "see.image_providers.DummyProvider"
},
"clone":
{
"storage_pool_path": "/home/username/instances",
"copy_on_write": true
}
},
"network":
{
"dynamic_address":
{
"ipv4": "192.168.0.0",
"prefix": 16,
"subnet_prefix": 24
}
}
}
Documentation¶
Set Up¶
Dependencies¶
- Required:
- python
- libvirt
- Recommended:
- qemu-kvm: KVM - Kernel-based virtual machine is a full virtualization solution for Linux.
- qemu-utils: QEMU utils package includes several useful utilities for handling virtual disk images.
- virtualbox: Virtualbox is a full-feature virtualization solution.
- dnsmasq: Dnsmasq is a lightweight DHCP server. It is required by libvirt networking stack.
Installation¶
Be sure to have libvirt’s development package installed.
In the root of the project run:
python setup.py install
SEE is available as Python package but it’s installation does not bring all the supported hypervisors.
SEE allows to control several different virtualization technologies (QEMU/KVM, Virtualbox, LXC), according to the chosen ones the set up may vary noticeably. For specific instructions on how to install and set up a given hypervisor, please refer to libvirt’s reference documentation.
Hardware acceleration for virtualization¶
KVM¶
To take advantages from hardware acceleration through KVM verify that the processor has VT (Intel) or SVM (AMD) capabilities and that they’re enabled in the BIOS configuration.
To verify that KVM is available it is enough to run:
# modporbe kvm_intel
for Intel processors or:
# modporbe kvm_amd
for AMD ones.
If the kernel is able to load the modules then KVM is fully available.
Virtualbox¶
To use Virtualbox ensure that its driver - vboxdrv - is properly loaded in the kernel. If not, just load it as follows:
# modprobe vboxdrv
Linux Containers (LXC)¶
Linux Containers require specific Control Groups to be enabled in order to operate:
- cpuacct
- memory
- devices
Additional recommended cgroups:
- cpu
- blkio
- freezer
To enable the required cgroups the User can rely on its init system. More details at:
Libvirt networking support¶
In order to activate the libvirt default network interface run the following command.
$ virsh net-start default
To ensure the default network interface to be active after a reboot, enable its autostart property.
$ virsh net-autostart default
Libvirt disk management¶
Libvirt manages the disk image files within storage pools. The storage pools are directories where libvirt looks up in order to find the correct disk image.
To create a storage pool, provide an XML file as the one in the example.
<pool type="dir">
<name>storage_pool_name</name>
<target>
<path>/var/lib/virt/images</path>
</target>
</pool>
Then load the pool within libvirt with the command.
$ virsh pool-create path_to_pool_file.xml
Start the newly created pool.
$ virsh pool-start storage_pool_name
Set the pool to be started at the machine startup.
$ virsh pool-autostart storage_pool_name
Finally, after each new disk image is moved into the pool directory, remember to refresh its list of storage volumes.
$ virsh pool-refresh storage_pool_name
More details about storage pools at:
User Manual¶
This section describes how to operate a Platform developed with SEE. The Environment interface and its documentation will be introduced to the reader. It will follow a description of the supported virtualization technologies and how to configure them.
Configuration¶
SEE configuration represents the interface the User can employ to describe a Test Case. It includes the chosen Sandbox technology, its specifications and the Hooks which will drive the test with their related options.
SEE requires a configuration for the Sandbox technology, usually referred as Resources configuration, and one for the Hooks. The Resources configuration is provided to a Context Factory callable which is responsible to spawn a new Sandbox. The Hooks configuration instead, will be passed to the Environment class constructor altogether with the Context Factory callable.
The configurations can be passed to the constructors both as a dictionary or as the path to the file in which they are stored, in the latter case the JSON format is the accepted one.
Hooks¶
The hooks section includes the list of required Hooks for the specific Test Case and their configuration. For each Hook, the User must provide its Python fully qualified name (PEP-3155). Furthermore, a set of configuration values can be specified either generic for all Hooks or Hook specific.
The example shows a configuration section including two Hooks - Hook1 and Hook2 - which will both receive the hook_generic configuration value while only Hook1 will have the hook_specific one.
{
"hooks":
{
"configuration":
{
"hooks_generic": "Hooks generic configuration value"
},
"hooks":
[
{
"name": "python.fully.qualified.name.Hook1",
"configuration":
{
"hook_specific": "Hook specific configuration value"
}
},
{
"name": "python.fully.qualified.name.Hook2",
"configuration": {}
}
]
}
}
Resources¶
SEE resources describes the layout of the sandboxes and their capabilities. Their configuration may vary according to the sandboxing technology which has been chosen. SEE includes a minimal support for QEMU/KVM, Virtualbox and Linux Containers but allows Developers to expand it through a simple interface.
The resources configuration syntax is virtualization provider specific and its details can be found within the modules implementation under:
see/context/resources/
The resources capabilities and configuration vary according to the sandbox provider chosen for the Test Case.
Linux Container (LXC)¶
Linux Container resources are provided by the module contained in:
see/context/resources/lxc.py
The hypervisor field is optional and allows the User to define the URL of the hypervisor driver, SEE can control Linux Containers located in remote machines as well. If not provided, the URL will point to the local host.
The domain field controls the sandbox instance properties, it must include the path to the Libvirt’s specific configuration XML file under the configuration field. Please refer to the libvirt Linux Container page for configuring a LXC domain.
The following tags are dynamically set, if missing, in the domain XML configuration:
- name: set as the Environment’s UUID.
- uuid: set as the Environment’s UUID.
Additionally, the User can specify one or more file system mounts to be exposed within the Linux Container. For each entry in the filesystem field, a source_path must be specified representing the mount location on the host side. In such location, a temporary subfolder, named as the Environment’s UUID, will be created avoiding collisions with other containers pointing at the same place. The target_path instead, contains the path which will be visible within the container. The mount points will be readable and writable from both the host and the guest sides.
The User can attach a Linux Container to a dynamically provisioned network relieving the User from their creation and management. See the Network section for more information.
The following JSON snippet shows an example of a LXC configuration.
{
"hypervisor": "lxc:///",
"domain":
{
"configuration": "/etc/myconfig/see/domain.xml",
"filesystem":
[
{
"source_path": "/srv/containers",
"target_path": "/"
},
{
"source_path": "/var/log/containers",
"target_path": "/var/log"
}
]
},
"network":
{ "See Network section" }
}
QEMU/KVM¶
QEMU resources are provided by the module contained in:
see/resources/qemu.py
The hypervisor field is optional and allows the User to define the URL of the hypervisor driver, SEE can control QEMU instances running in remote machines as well. If not provided, the URL will point to qemu:///system.
The domain field controls the sandbox instance properties, it must include the path to the Libvirt’s specific configuration XML file under the configuration field. Please refer to the libvirt QEMU page for configuring a QEMU/KVM domain.
The following tags are dynamically set, if missing, in the domain XML configuration:
- name: set as the Environment’s UUID.
- uuid: set as the Environment’s UUID.
The disk field must be provided with the image key containing the path to the disk image file intended to be used. Furthermore, the disk image must be contained in a Libvirt’s storage pool.
It is a common use case to start the virtual machine from a specific state - for example with the operative system installed and configured - preserving it for different tests. To fulfil this requirement, the original disk image can be cloned into a new one which will be employed to perform the test.
If the clone section it’s provided, a storage_pool_path must be present. A storage pool consists of a folder in which all the disk image files associated to a domain are contained. Within the given path, a new directory will be created with the Environment’s UUID as name and it will contain the clone of the original disk image.
The optional copy_on_write boolean flag dictates whether the whole disk image will be cloned or only the new files created during the test execution. This allows to save a considerable amount of disk space but the original disk image must be available during all the Environment’s lifecycle.
The User can attach a QEMU Virtual Machine to a dynamically provisioned network relieving the User from their creation and management. See the Network section for more information.
The following JSON snippet shows an example of a QEMU configuration.
{
"hypervisor": "qemu:///system",
"domain":
{
"configuration": "/etc/myconfig/see/domain.xml",
},
"disk":
{
"image":
{
"uri": "/home/username/images/IE8_-_Win7-disk1.qcow2",
"provider": "see.image_providers.DummyProvider"
},
"clone":
{
"storage_pool_path": "/var/data/pools",
"copy_on_write": true
}
},
"network":
{ "See Network section" }
}
Virtualbox¶
Unfortunately, due to the limited Virtualbox support offered by Libvirt, the amount of customisation is pretty poor. Nevertheless, the Virtualbox default resource provider can be expanded in order to increase its capabilities, please refer to the The Resources Class subchapter under the Developer Manual section to see how to customise the default resource providers.
Virtualbox resources are provided by the module contained in:
see/resources/vbox.py
The hypervisor field is optional and allows the User to define the URL of the hypervisor driver, SEE can control Virtualbox instances running in remote machines as well. If not provided, the URL will point to vbox:///session.
The domain field controls the sandbox instance properties, it must include the path to the Libvirt’s specific configuration XML file under the configuration field. Please refer to the libvirt Virtualbox page for configuring a Virtualbox domain.
The following tags are dynamically set, if missing, in the domain XML configuration:
- name: set as the Environment’s UUID.
- uuid: set as the Environment’s UUID.
The disk field must be provided with the image key containing the path to the disk image file intended to be used.
The following JSON snippet shows an example of a Virtualbox configuration.
{
"name": "see.resources.vbox.VBoxResources",
"hypervisor": "vbox:///session",
"domain":
{
"configuration": "/etc/myconfig/see/domain.xml",
},
"disk":
{
"image": "/var/mystoragepool/image.vdi",
}
}
Network¶
Network resources are provided by the module contained in:
see/resources/network.py
A typical scenario is the execution of a Sandbox connected to a subnetwork. For the simplest use cases, libvirt’s default network is enough. Yet there are different situations in which, for example, the User wants to execute multiple sandboxes on the same host ensuring their network isolation.
SEE can provision a subnetwork attaching to it the sandbox and taking care of its allocation and removal. This feature is controlled by the network field.
The network field specifies the virtual subnetwork in which the container will be placed. The User can optionally provide a configuration file. Please refer to the libvirt Networking page for configuring a virtual network. If no XML configuration file is specified, SEE will generate a network with NAT forward mode for the User based on the details provided by the dynamic_address.
The dynamic_address delegates to SEE the generation of a valid IPv4 address, the XML configuration must not contain an ip field if so. The User must specify the address and prefix of the network in which to create the subnetwork as well as the subnetwork prefix. SEE will generate a random subnetwork address according to the specifications avoiding collisions with other existing libvirt networks. A DHCP server will be provided within the subnetwork serving the sandbox guest Operating System.
The following JSON snippet shows an example of a network configuration with dynamic address generation.
{
"configuration": "/etc/myconfig/see/network.xml",
"dynamic_address":
{
"ipv4": "192.168.0.0",
"prefix": 16,
"subnet_prefix": 24
}
}
In the following example, SEE will generate a subnetwork within the network 192.168.0.0/16. The subnetwork will have the address 192.168.X.0/24 where X is a random number in the range 0-255. The DHCP server will assign addresses to the sandbox in the range 192.168.X.[0-255].
Image Providers¶
SEE image_providers define a simple common interface to interact with backend disk image providers. Their configuration may vary depending on the chosen provider and backend, however the image configuration section must hold a dictionary with the following structure, common to all providers:
{
"image":
{
"uri": "ImageURI",
"provider": "fully.qualified.class.name",
"provider_configuration": {}
}
}
The sections within the image dictionary have the following meanings:
- uri: A string representing an identifier or resource locator for the image within the provider’s backend.
- provider: The python fully qualified class name of the provider class to use.
- provider_configuration: A dictionary representing the image provider configuration. This is provider specific and varies depending on the image provider.
The image section can be a string for backwards compatibility, in which case no provider is used and an image file is expected to be found at the specified path. SEE makes no guarantee about this image being usable and it is up to the user to ensure that it is.
SEE includes a dummy image provider as well as providers for LibVirt Storage Pools and OpenStack Glance. Image providers are pluggable and can be extended through the ImageProvider interface.
The provider_configuration object is provider specific and its details can be found within the providers’ implementation under:
Note: Image providers do not perform any kind of image cleanup tasks. It is up to the client to make sure that any unused images available locally are cleaned up when no longer needed.
see/image_providers
Dummy Provider¶
The Dummy provider is provided by the module contained in:
see/image_providers/dummy.py
This provider has no internal logic and returns the image uri as is, it takes no provider configuration parameters and behaves in the same way as the deprecated image string.
The following JSON snippet shows an example of a DummyProvider configuration.
{
"image":
{
"uri": "/home/username/images/IE8_-_Win7-disk1.qcow2",
"provider": "see.image_providers.DummyProvider",
"provider_configuration": {}
}
}
LibVirt Pool Provider¶
The Libvirt pool provider is provided by the module contained in:
see/image_providers/libvirt_pool.py
This provider retrieves the absolute path to an image within a libvirt storage pool. The uri section represents a path to the image relative to the provider’s configured storage pool.
The Following JSON snippet shows an example of a LibvirtPoolProvider configuration.
{
"image":
{
"uri": "IE8_-_Win7-disk1.qcow2",
"provider": "see.image_providers.LibvirtPoolProvider",
"provider_configuration":
{
"hypervisor": "qemu:///system",
"storage_pool_path": "/home/username/images/"
}
}
}
A LibvirtPoolProvider with the previous configuration will ensure that the hypervisor has an active and fresh storage pool at storage_pool_path. The image file referenced by uri is expected to exist beforehand at storage_pool_path and the provider will fail otherwise.
Glance Provider¶
The Glance provider is provided by the module contained in:
see/image_providers/os_glance.py
This provider retrieves the requested image from a glance service and stores it locally at a configured location. The uri section refers to an image name or id within the glance service. The Glance provider will ensure that the image file at the returned path is at its newest available version, returning the freshest possible image from those matching the requested uri; if the local image is already fresh it will not be downloaded again.
The Following JSON snippet shows an example of a GlanceProvider configuration.
{
"image":
{
"uri": "IE8_-_Win7-disk1",
"provider": "see.image_providers.GlanceProvider"
"provider_configuration":
{
"target_path": "/home/username/images/",
"glance_url": "http://my.glance.service:9292"
"os_auth":
{
"auth_url": "http://my.keystone.service:5000/v2.0/",
"username": "user",
"password": "pass",
"project_name": "MyOpenStackTenant"
}
}
}
}
The provider_configuration sections are as follows:
target_path: A string representing the absolute local path to download the image to.
The path can be a directory or a file. If it is a directory the image will be downloaded and stored with its glance ID as filename. If the path is a file the image will be downloaded from Glance if the local file doesn’t exist or is older than the glance image.
glance_url: A string representing the URL of the backend glance service.
os_auth: A dictionary containing pertinent keys for OpenStack authentication parameters as understood by keystoneclient, for details on these parameters, see:
Developer Manual¶
This section describes how to further develop the SEE Platform. The core interfaces are introduced at first. It will follow an in depth description of the provided implementations.
The Context Interface¶
A Context is an Observable object. It allows multiple plugin objects - named Hooks - to operate over it and notifies them back through Events.
A Context must be followed by a Factory function which, when called, receives the Environment identifier and returns the fully functional Context object.
The Developer can extend the Context class to provide his/her own Sandbox implementation. The newly implemented class will need to provide a cleanup method which will be called during the Environment de-allocation phase which occurs at the end of the Test Case execution. All the Event related mechanisms will be provided by the parent class.
The Hook Interface¶
A Hook is an Observer object. It allows the plugin to control the given Context driving the Test Case lifecycle. Through the Context, the Hook can subscribe and unsubscribe its Handlers to the Events. The Handlers will be called at the triggering of the given Event and will receive the Event itself as parameter.
The Developer can extend the Hook class to provide his/her plugins for driving the Test Case execution. The Developer can override the Hook’s cleanup method to remove temporary resources during the Environment de-allocation phase which occurs at the end of the Test Case execution.
The parent class provides as well a logger object and the configuration values specified by the User in the Hooks configuration.
The Resources Interface¶
The Resources interface has been initially introduced as an adapter between the Context and libvirt API interface. It is a generic interface for the Sandbox resource provisioning and management.
The Developer can implement the Resource class to encapsulate his/her own Sandbox provider.
The Resources interfaces expose an allocation and de-allocation method and accepts an identifier and a configuration for its constructor parameters.
The SEE Context and its Factories employ the Resources interface to abstract the libvirt API.
The ImageProvider Interface¶
The ImageProvider interface facilitates a system of plugins to retrieve disk images from arbitrary sources.
The Developer can implement the ImageProvider class to encapsulate his/her own disk image provider.
The ImageProvider interface exposes a single image property. This property is expected to be a string representing an absolute path to the image file, locally available to the Context. The class implementing this interface is expected to hold the logic to retrieve the disk image and store it locally in the path returned by the image property.
The Environment Class¶
The User can spawn multiple isolated instances consisting in separate Test Cases through the Environment class. Each instance requires a Context Factory callable and its Hooks specific configuration.
The example code is a simple yet quite complete example of usage of the Environment class.
from time import time
from see import Environment
from see.resources import QemuFactory
context_factory = QemuFactory('/path/to/resources/configuration.json')
with Environment(context_factory, '/path/to/hooks/configuration.json') as environment:
context = environment.context
context.poweron()
ip_address = wait_for_ip_address(context)
context.trigger('got_ip_address', address=ip_address)
time.sleep(180)
context.poweroff()
def wait_for_ip_address(context):
while 1:
if context.ip4_address is not None:
return context.ip4_address
else:
time.sleep(1)
A QemuFactory class is employed to provide a Context Factory callable to the Environment. The Environment receives the Context Factory and the path to the Hooks configuration file. The Environment class implements the Python Context Manager protocol (PEP-343) allowing to encapsulate the allocate and deallocate methods in the with statement.
Once allocated the Environment, its Context is retrieved in order to power on the Sandbox. The poweron method triggers a pre_poweron and a post_poweron Events to the installed Hooks.
In order to detect when the guest Operating System has terminated its boot procedure, the Context ip4_address attribute is polled. If the network interface is using DHCP server for its address assignment, the Context ip4_address attribute will become available only once the IP address will be correctly assigned to the guest. Most of the Operating Systems request an IP address at the end of their boot routine.
The guest IP address is then forwarded to the Hooks as an appended attribute to the got_ip_address attribute.
Three minutes are then waited in order to allow the Hooks to perform their duties and then, the Sandbox is powered off triggering a pre_poweroff and a post_poweroff Events.
The Resources Class¶
The SEE Resources Classes implements the Sandbox provisioning machinery and expose its internals to the Hooks Developers.
The class constructor receives the Environment identifier and the Resources specific configuration. Once the object has been initialised, the Developer can call its allocate method to build the specific Sandbox.
The Resources interface contract can be found in the following file:
see/context/resources/resources.py
The Resources object is characterised by four attributes which are derived from libvirt’s specific terminology.
hypervisor¶
The Hypervisor connection represent the handler to the Sandbox provisioning controller. Resources are allocated and de-allocated through this object. Hooks Developers might use this handler to instruct the provisioning controller about some group specific property or to handle failures whenever the other resources become corrupted. In libvirt name space this object directly correlates with the virConnectPtr object.
domain¶
The Domain encapsulates the specific sandbox instance, allowing the Hooks to directly access to its state. The Context object employs this attribute to realise the state-change machinery and the Hooks Developers might access to the sandbox memory or CPU through it. In libvirt name space this object directly correlates with the virDomainPtr object.
storage_pool¶
The Storage Pool contains all the Disks associated to the sandbox instance. In libvirt name space this object directly correlates with the virStoragePool object.
network¶
The Network object represents the network to which the sandbox is connected. In libvirt name space this object directly correlates with the virNetwork object.
A deallocate method will be called at the end of the Environment’s lifecycle, it’s responsibility is to free the Sandbox specific resources.
The Context Class¶
The SEE Context class wraps the resources allocated within the Resources Class and takes care of providing thread safe access from the Hooks.
The following methods are exposed via the Context:
- poweron: Starts the virtual machine and triggers the pre_poweron and post_poweron events.
- pause: Suspends the virtual machine and triggers the pre_pause and post_pause events.
- resume: Resumes the suspended virtual machine and triggers the pre_resume and post_resume events.
- restart: Restarts the virtual machine and triggers the pre_reboot and post_reboot events.
- poweroff: Stops the virtual machine and triggers the pre_poweroff and post_poweroff events. This method is the equivalent of a power cut to a running machine.
- shutdown: Sends a shutdown request to the virtual machine and triggers the pre_shutdown and post_shutdown events. This method blocks until the Sandbox has not shut down or until the given timeout has expired. The method will block indefinitely if the guest Operating System does not handle correctly the shutdown request.
The Resources de-allocation is performed in the SEE Context cleanup method.
The Context Factory¶
The Context Factories are callable which receive the Environment identifier when called and are supposed to return a functional Context object.
The example in picture shows how the QEMU Context Factory is realised.
from see.context import SEEContext
from see.context.resources import QemuResources
class QemuFactory(object):
def __init__(self, configuration):
self.configuration = configuration
def __call__(self, identifier):
"""Called by the Environment allocate() method."""
resources = QemuResources(identifier, self.configuration)
try:
resources.allocate()
except Exception:
resources.deallocate()
raise
return SEEContext(identifier, resources)
The Hooks¶
A Hooks is an Observer class which receives a reference to the Context and uses it to drive the Test Case.
The code block shows a quite exhaustive example of a Hook.
from see import Hook
from time import time
from utils import delete_folder
class ExampleHook(Hook):
"""Example Hook"""
def __init__(self, *args):
super(ExampleHook, self).__init__(*args)
self.setup_handlers()
def setup_handlers(self):
self.context.subscribe_async('post_poweron', self.poweron_event_handler)
self.context.subscribe('custom_event', self.custom_event_handler)
self.context.subscribe('post_pause', self.pause_event_handler)
def poweron_event_handler(self, event):
"""This handler is run asynchronously. It does not block the Event flow"""
self.logger.info("%s event received, the Context is powered on", event)
time.sleep(60)
self.context.trigger('custom_event') # fire an Event to all Hooks
def custom_event_handler(self, event):
"""This Handler is run synchronously and powers off the Context."""
self.context.pause()
def pause_event_handler(self, event):
"""Event Handler for the last event (post_paused)."""
self.logger.info("%s event received, the Context is paused", event)
def cleanup(self):
"""
If defined, this method will be executed during the Environment de-allocation.
It allows the Developer to cleanup temporary resources.
"""
if 'temporary_folder' in self.configuration:
delete_folder(self.configuration['temporary_folder'])
Each Handler is run synchronously. This means that only a Handler at a time can be executed and triggering an Event will block the execution until all the subscribed Handlers have been consumed. In case this is not the desired behaviour, the Developer can subscribe asynchronous Handlers which will run concurrently without blocking the execution flow.
In the example an asynchronous Handler is subscribed to the post_poweron Event. Its Handler waits for a minute and then triggers a custom Event. The custom Event is handled by the custom_event_handler Handler which pauses the Context triggering a pre_pause and a post_pause Events. The Hook reacts to the post_pause through the pause_event_handler Handler and then waits for the Environment de-allocation in which cleans up the configured temporary_folder
The next example shows a possible implementation of a Hook which captures screenshots of the guest Operating System. The User can specify at which Events the screenshots should be taken through the screenshot_on_event configuration key.
from see import Hook
from see.context import RUNNING, PAUSED
from utils import create_folder, take_screenshot
class ScreenHook(Hook):
"""
Screenshot capturing hook.
On the given event, it captures the Context's screen on a PPM file in the given folder. The "screenshot_on_event" can be either a string representing the event or a list of multiple ones.
configuration::
{
"results_folder": "/folder/where/to/store/screenshots/",
"screenshot_on_event": ["post_poweron", "custom_event1", "custom_event2"]
}
"""
def __init__(self, parameters):
super(ScreenHook, self).__init__(parameters)
self.setup_handlers()
def setup_handlers(self):
if 'screenshot_on_event' in self.configuration:
configured_events = self.configuration['screenshot_on_event']
events = (isinstance(configured_events, basestring)
and [configured_events] or configured_events)
for event in events:
self.context.subscribe(event, self.capture_screenshot)
self.logger.debug("Screenshot registered at %s event", event)
def capture_screenshot(self, event):
folder_path = self.configuration['results_folder']
screenshot_path = os.path.join(folder_path, "%s_%s.ppm"
% (self.identifier, event))
self.logger.info("Event %s: capturing screenshot.", event)
create_folder(folder_path)
self.screenshot(screenshot_path)
self.logger.info("Screenshot %s captured.", screenshot_path)
def screenshot(self, screenshot_path):
self.assert_context_state()
with open(screenshot_path, 'wb') as screenshot_file:
screenshot_stream = take_screenshot(self.context)
screenshot_file.write(screenshot_stream)
def assert_context_state(self):
if self.context.domain.state()[0] not in (RUNNING, PAUSED):
raise RuntimeError("Cannot capture screenshots of a shutdown Contex")
The Events¶
The Events represent the communication interface between the Hooks and the Environment in which they are executed. Once an event is fired, all the Hooks which subscribed one or more of their Handlers will execute them.
Events are represented by a class, which extends a Python string, to which extra information is appended as attributes. To fire an Event, a Hook must use the Context trigger method which accepts as parameters either an Event instance or more simply a string and a set of keyword arguments. In the latter case, an Event object will be built from the given string and the keyword arguments will be appended to it as attributes.
The Hook handlers will receive the dispatched Event as argument with the attributes origin carrying the Python’s fully qualified name of the actor which generated it and timestamp representing the moment in which the Event has fired.
SEE is well suited for a model-view-controller type of design. The model is represented by the Context Class, the view are the Events flowing through the Hooks and the controller could be either the Hooks in charge of acting as decision-maker or the Environment user.
Context specific Events¶
The Context implementation provided by SEE triggers a set of default events every time its state transition methods are called. Each state transition operated through the Context triggers two Events: a pre_transition which precedes the transition itself and a post_transition one, triggered after the Context is in its new state.
These events allow the Hooks to synchronise with the Sandbox state in order to perform actions requiring certain states. Taking a snapshot of the Sandbox memory state, for example, requires the virtual machine to be paused in order to get coherent data.
The Developer must keep in mind that triggering a Context state change while reacting to another one is a dangerous approach. It is highly recommended as well, not to perform any long lasting operation during the Handling of these specific Events. If the latter case cannot be avoided, the Developer can rely on the asynchronous Handlers to prevent heavy routines to significantly slow down the Event processing.
Below are listed the Events generated by default by the Context.
pre_poweron¶
Triggered by the Context poweron method.
This Event is fired before powering on the Context, therefore the Context state is supposed to be shutoff.
post_poweron¶
Triggered by the Context poweron method.
This Event is fired after powering on the Context, therefore the Context state is supposed to be running.
pre_pause¶
Triggered by the Context pause method.
This Event is fired before suspending the Context, therefore the Context state is supposed to be running.
post_pause¶
Triggered by the Context pause method.
This Event is fired after suspending the Context, therefore the Context state is supposed to be paused.
pre_resume¶
Triggered by the Context resume method.
This Event is fired before resuming the Context from suspension, therefore the Context state is supposed to be paused.
post_resume¶
Triggered by the Context resume method.
This Event is fired after resuming the Context from suspension, therefore the Context state is supposed to be running.
pre_poweroff¶
Triggered by the Context poweroff method.
This Event is fired before forcing the Context to power off, therefore the Context state is supposed to be either running or paused.
post_poweroff¶
Triggered by the Context poweroff method.
This Event is fired after forcing the Context to power off, therefore the Context state is supposed to be shutoff.
pre_shutdown¶
Triggered by the Context shutdown method.
This Event is fired before requesting the guest Operating System to shut down , therefore the Context state is supposed to be running.
post_shutdown¶
Triggered by the Context shutdown method.
This Event is fired after requesting the guest Operating System to shut down , therefore the Context state is supposed to be shutdown.
pre_restart¶
Triggered by the Context restart method.
This Event is fired before requesting the guest Operating System to restart, therefore the Context state is supposed to be either running or crashed.
post_restart¶
Triggered by the Context restart method.
This Event is fired after requesting the guest Operating System to restart, therefore the Context state is supposed to be running.
Environment lifecycle¶
To ensure a Sandbox with a high level of security, for each Test Case all the needed Resources are created at the moment of the request and completely destroyed once the Environment is not necessary anymore.
An Environment has its own lifecycle starting from the moment in which its allocate method is called and ending with the deallocate method invocation. It is the User’s and the Developer’s responsibility to store and process the execution data as after the Environment de-allocation, all the Sandbox Resources and the Hooks won’t be accessible anymore.
An example of a typical Environment lifecycle could be:
- Generation of the Sandbox and the Hooks configurations.
- Initialisation of the Context Factory callable object.
- Initialisation of the Environment.
- Allocation of the Environment.
- Guest Operating System power on through the Context poweron method.
- Injection of the Sample via networking or other mean.
- Execution of the Test Case.
- Tracing of the Sample behaviour through the configured Hooks.
- Guest Operating System power off through the Context shutdown or poweroff methods.
- Data collection and analysis via the configured Hooks.
- Data storage according to configuration.
- Environment deletion and resources release.
Lifecycle traceability¶
To each Environment object is assigned a Universally Unique IDentifier (UUID), it is recommended to use the same identifier for all the involved parts as it helps to address the complete history of a Test Case instance.
The Resources provided by SEE are all sharing the same identifier simplifying the clean-up process.
