Welcome to oct’s documentation!¶
Oct stand for Open Charge Tester, the goal of this project is to give you the basics tools for write simple tests. The tests are simple python scripts that make calls to web page or web service, submit data, login, etc... OCT will give you basics tools for easily writing your test.
This documentation will provide you basics examples for writing tests, use oct-tools, lunch tests, get results or even customize the results to fit your needs
Note that the OCT project is in early development and is not suitable for productions tests actually.
If you want to contribute you’re welcome ! Check the git, fork the project, and submit your pull requests !
The OCT module steel need many features at this point, here somme examples :
- Full python3 support
- New lib for replace Mechanize (based on html5lib ?)
- Full celery integration for multi-processing
- More generic tests in core module
- More fancy templates
- etc...
Basics module information¶
OCT is based on multi-mechanize, a library for testing website. But this module is no longer under active development and the last commit was 3 years ago.
So instead of a fork, for building OCT module we include multi-mechanize as a module, and we update it. For the moment modifications are minors and the main job of OCT module is inside the core submodules, which contains a GenericTransaction class
which provide you useful methods for writing your tests scripts.
We already have done some update on the multi-mechanize modules like :
- update render of graphics
- update command for new projects
- more information in config file
- customisable templates
But other improvements are on the way ! So stay tune on github !
How to¶
For each functionality, we have tried to write a how to. In that way you should be able to do everything you need with this library, even customize it and add features to it !
See the examples project page
Installation¶
You’ll need some linux packages for the installation, To install the required development packages of these dependencies on Linux systems, use your distribution specific installation tool, e.g. apt-get on Debian/Ubuntu:
sudo apt-get install libxml2-dev libxslt-dev python-dev
You can install the module with :
python setup.py install
Or using pip :
pip install oct
NB : You may encounter build error with pip or easy_install, you
Contents¶
Indices and tables¶
examples¶
Creating a new project¶
For starting a new project you have access to this command :
oct-newproject <project_name>
This command will create a new project inside the current directory named with the <project_name> argument
The created directory must look like this :
.
├── config.cfg
├── templates
│ ├── css
│ │ └── style.css
│ ├── footer.html
│ ├── head.html
│ ├── img
│ └── scripts
└── test_scripts
└── v_user.py
This folders contains the basic for running an OCT project.
Configuration¶
For configuration explanation and examples see the Configuration page
Customizing your templates¶
You need an other render for the results ? the default template is ugly and you want to change it ? It’s ok, we have done some things for help you to do that.
If you have created your project with the oct-newproject command, you have a templates directory inside your project. This directory is used for writing the results, so each call to multimech-run command will read this files. With this you can easily update the template and customize it to fit your needs. It’s simple as that.
For the moment the templates can’t be fully modified, but you steel have plenty of options to change them.
Let’s take a look at the style.css file :
/* http://meyerweb.com/eric/tools/css/reset/
v2.0 | 20110126
License: none (public domain)
*/
html, body, div, span, applet, object, iframe,
h1, h2, h3, h4, h5, h6, p, blockquote, pre,
a, abbr, acronym, address, big, cite, code,
del, dfn, em, img, ins, kbd, q, s, samp,
small, strike, strong, sub, sup, tt, var,
b, u, i, center,
dl, dt, dd, ol, ul, li,
fieldset, form, label, legend,
table, caption, tbody, tfoot, thead, tr, th, td,
article, aside, canvas, details, embed,
figure, figcaption, footer, header, hgroup,
menu, nav, output, ruby, section, summary,
time, mark, audio, video {
margin: 0;
padding: 0;
border: 0;
font-size: 100%;
font: inherit;
vertical-align: baseline;
}
/* HTML5 display-role reset for older browsers */
article, aside, details, figcaption, figure,
footer, header, hgroup, menu, nav, section {
display: block;
}
body {
line-height: 1;
}
ol, ul {
list-style: none;
}
blockquote, q {
quotes: none;
}
blockquote:before, blockquote:after,
q:before, q:after {
content: '';
content: none;
}
table {
border-collapse: collapse;
border-spacing: 0;
}
body {
background-color: #f4f4f4;
font-family: "Helvetica Neue", Helvetica, Roboto, Arial, sans-serif;
}
h1 {
font-size: 4em;
background: #2b2b2b;
color: white;
font-weight: bold;
}
h2 {
font-size: 2em;
background: #f78930;
margin: 15px 0 15px 0;
}
h1, h2, h3, h4, h5, h6 {
padding: 15px;
}
h4 {
font-weight: bold;
font-size: 1.3em;
}
h3 {
font-size: 1.5em;
font-weight: bold;
}
.summary {
padding-left: 15px;
}
.summary > b {
font-weight: bold;
}
#main table {
margin-left: 15px;
border: 1px solid grey;
}
#main th {
font-weight: bold;
padding: 10px 0 10px 0;
border: 1px solid grey;
}
#main tr {
padding: 10px 0 10px 0;
text-align: center;
}
#main td {
min-width: 70px;
padding: 10px 5px 10px 5px;
border: 1px solid grey;
}
hr {
color: #f4f4f4;
background-color: #f4f4f4;
border: none;
}
As you can see, all style present on the result page is here, so feel free to update it. But you may need some other css files, like a css framework, or even javascript files ? why not after all ?
Well you can do that, you can include all the files you need for customize your results page.
How ? simply edit the `templates/head.html’ and include your files, you can even create your own header, add messages at the top of the page, etc...
A little explanation of how this work :
When you call the multimech-run command inside your project directory, the command will look for the templates directory and read the head.html and the footer.html files, and will create a new html page with them. At the same time the command will copy all files insides the img, scripts, and css directories. So everything added in this folders will be in the associated result directory. In that way you can add all the stuff you want to your results, and not reworking each result after each test
Writing your first script¶
It’s time to write our first script and test it, so first let’s take a look at the generated v_user.py file :
from oct.core.generic import GenericTransaction
import random
import time
import os
CONFIG_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../')
class Transaction(GenericTransaction):
def __init__(self):
GenericTransaction.__init__(self, True, CONFIG_PATH)
def run(self):
r = random.uniform(1, 2)
time.sleep(r)
self.custom_timers['Example_Timer'] = r
if __name__ == '__main__':
trans = Transaction()
trans.run()
print trans.custom_timers
So what does this script ? Since it’s an example script, actually it just sleep for 1 or 2 seconds.
Let’s update this script a little, but first don’t forget to update the configuration file to fit your configuration.
Okay so let’s write a simple script, just for accessing the index page of our web site and get the statics file of it
from oct.core.generic import GenericTransaction
import time
import os
CONFIG_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../')
class Transaction(GenericTransaction):
def __init__(self):
GenericTransaction.__init__(self, True, CONFIG_PATH)
def run(self):
test_time = time.time()
resp = self.open_url('/')
self.get_statics(resp, 'index_statics')
self.custom_timers['test_time'] = time.time() - test_time
if __name__ == '__main__':
trans = Transaction()
trans.run()
print trans.custom_timers
So that’s it, we just open the index url of the website (based on the base_url configuration variable), get the response object returned by the open_url method and pass it to the get_statics method.
So what does this test do ? well it accesses to the index page and retrieve all css, javascript and img files in it. Simple as this
Testing your script¶
So what’s next ? Now you got your basic script retrieving your index page and associated statics files. But does it works ?
Let’s figure it out. To test your script 1 time, just to make sure all code work, you actually call the script with your python interpreter like this :
python my_script.py
With the previous script, if everything is ok, you must see the timer on the standard output.
Everything work find ? Nice, let’s now run our tests with lot of users, so update your configuration file and then you just have to run :
multimech-run <myproject>
Or if you’re already inside the path of you’re project, simply run :
multimech-run .
You must see the progress bar appears, you now just have to wait till the tests end. This action will create a results directory inside your project folder, and a sub-directory containing the results in csv or html format.
Handle forms¶
You now know how to access url and retrieve statics files, but this still basics actions right ? Let’s handles some forms and submit some data.
So we gonna take our previous script and update it a bit :
from oct.core.generic import GenericTransaction
from oct.testing.content import must_contain
import time
import os
CONFIG_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../')
class Transaction(GenericTransaction):
def __init__(self):
GenericTransaction.__init__(self, True, CONFIG_PATH)
def run(self):
test_time = time.time()
# getting the url
resp = self.open_url('/')
# getting the form
self.get_form(form_id='searchForm')
# setting the data
self.fill_form({'q': 'test'})
# getting the response
resp = self.br.submit()
# checking response content
must_contain(resp, 'Results that must be found')
self.custom_timers['test_time'] = time.time() - test_time
if __name__ == '__main__':
trans = Transaction()
trans.run()
print trans.custom_timers
We removed statics management for tests. So what do we do now ? Well let’s resume that :
- access the index url
- getting the form with id attribute set to searchForm
- considering this form has 1 input named q, we set the data for this field to test
- submit the form
- checking the content of the returned page.
And that’s all, we handle a simple search form, and checking the results !
oct.core package¶
oct.core.exceptions module¶
- exception oct.core.exceptions.OctGenericException¶
Bases: exceptions.Exception
Provide generic exception for reports
oct.core.generic module¶
- class oct.core.generic.GenericTransaction(handle_robots, pathtoini, **kwargs)¶
Bases: object
Base class for Transaction, this class provides tools for simpler test writing
Parameters: - handle_robots (bool) – set if robots are handle or not
- pathtoini (str) – the path to the ini file
- threads (int) – number of threads for static files
- timeout – the timeout in second for static files requests
- use_cookies – default to True, set to False if you don’t want cookies with browser object
- auth(auth_url, data, use_form=True, **kwargs)¶
Authenticate yourself in the website with the provided data
Data must have this form :
- data {
- ‘login_field_name’: ‘login’, ‘password_filed_name’: ‘password’
}
Parameters: - auth_url (str) – the url of the page for authentication
- form_name (str) – the name attribute of the form
- form_id (str) – the id attribute of the form
- form_class (str) – the class attribute of the form
- nr (int) – the position of the form inside the page
Returns: the response object from the submission
- static csv_to_list(csv_file)¶
Take a csv file as parameter and read it. Return a list containing all lines
Parameters: csv_file (str) – the csv file to read Returns: A list containing the lines Return type: list
- fill_form(form_data)¶
Fill the form selected in self.br with form_data dict
Parameters: form_data (dict) – dict containing the data
- get_form(**kwargs)¶
This method help you for getting a form in a given response object The form will be set inside the br property of the class
Parameters: - form_name (str) – the name attribute of the form
- form_id (str) – the id attribute of the form
- form_class (str) – the class attribute of the form
- nr (int) – the position of the form inside the page
Returns: None
- static get_random_csv(csv_list)¶
Simply return a random element from csv_list param
Parameters: csv_list – a list Returns: random element from the csv_list
- get_statics(response, timer_name, include=None)¶
Get all static files for given response object. It will exclude all files in the exclude list
Parameters: - response (MechanizeResponse) – The response object from browser
- timer_name (str) – The timer name to increment
- include (tuple) – The list of statics to exclude
Returns: None
- multi_process_statics()¶
Multi threading static getter. This function will be call inside a Thread by the get_statics method
Returns: None
- open_url(url, data=None)¶
Open an url with the Browser object
Parameters: - url (str) – the url to open
- data (dict) – the data to pass to url
- run()¶
Run method will be call by multi-mechanize run function You must implement it
- run_generic_test(timer_name, url, test_func, *args)¶
Play the test_func param with *args parameters This function will call the browser on the url param for you You can pass existing or custom functions, but if you want to create custom test function, it must at least take a response object as first parameter
Parameters: - timer_name (str) – the name of the timer
- url (str) – the url to test
- test_func (function) – pointer on a testing function
- args – the parameters of the test function
Returns: The response object from Mechanize.Browser()
oct.core.browser module¶
- class oct.core.browser.Browser(session=None, base_url='')¶
Bases: object
This class represent a minimal browser. Build on top of lxml awesome library it let you write script for accessing or testing website with python scripts
Parameters: - session – The session object to use. If set to None will use requests.Session
- base_url – The base url for the website, will append it for every link without a full url
- back()¶
Go to the previous url in the history property
Returns: the Response object
- follow_link(selector, url_regex=None)¶
Will access the first link found with the selector
Parameters: - selector – a string representing a css selector
- url_regex – regex for finding the url, can represent the href attribute or the link content
Returns: Response object
- get_form(selector=None, nr=0)¶
Get the form selected by the selector and / or the nr param
Parameters: - selector – A css-like selector for finding the form
- nr – the index of the form, if selector is set to None, it will search on the hole page
Returns: None
- history¶
Return the actual history
Returns: the _history property Return type: list
- open_url(url, data=None, back=False, **kwargs)¶
Open the given url
Parameters: - url – The url to access
- data – Data to send. If data is set, the browser will make a POST request
- back – tell if we actually accessing a page of the history
Returns: The Response object from requests call
- submit_form()¶
Submit the form filled with form_data property dict
Returns: Response object after the submit
oct.multimechanize package¶
Subpackages¶
oct.multimechanize.utilities package¶
Submodules¶
oct.multimechanize.utilities.gridgui module¶
Multi-Mechanize Grid Controller sample gui application for controlling multi-mechanize instances via the remote management api
- class oct.multimechanize.utilities.gridgui.Application(root, hosts)¶
- check_servers()¶
- clear_window()¶
- get_configs()¶
- get_project_names()¶
- get_results()¶
- list_nodes()¶
- run_tests()¶
- update_configs()¶
- oct.multimechanize.utilities.gridgui.main()¶
oct.multimechanize.utilities.newproject module¶
- oct.multimechanize.utilities.newproject.create_project(project_name, config_name='config.cfg', script_name='v_user.py', scripts_dir='test_scripts', config_content='\n[global]\nrun_time = 30\nrampup = 0\nresults_ts_interval = 10\nprogress_bar = on\nconsole_logging = off\nxml_report = off\n\n\n[user_group-1]\nthreads = 3\nscript = v_user.py\n\n[user_group-2]\nthreads = 3\nscript = v_user.py\n\n', script_content="\nimport random\nimport time\n\n\nclass Transaction(object):\n def __init__(self):\n pass\n\n def run(self):\n r = random.uniform(1, 2)\n time.sleep(r)\n self.custom_timers['Example_Timer'] = r\n\n\nif __name__ == '__main__':\n trans = Transaction()\n trans.run()\n print trans.custom_timers\n")¶
- oct.multimechanize.utilities.newproject.main()¶
oct.multimechanize.utilities.run module¶
- class oct.multimechanize.utilities.run.UserGroupConfig(num_threads, name, script_file)¶
Bases: object
- oct.multimechanize.utilities.run.configure(project_name, cmd_opts, config_file=None)¶
- oct.multimechanize.utilities.run.main()¶
Main function to run multimechanize benchmark/performance test.
- oct.multimechanize.utilities.run.rerun_results(project_name, cmd_opts, results_dir)¶
- oct.multimechanize.utilities.run.run_test(project_name, cmd_opts, remote_starter=None)¶
Module contents¶
Submodules¶
oct.multimechanize.core module¶
- class oct.multimechanize.core.Agent(queue, process_num, thread_num, start_time, run_time, user_group_name, script_module, script_file)¶
Bases: threading.Thread
- run()¶
- class oct.multimechanize.core.UserGroup(queue, process_num, user_group_name, num_threads, script_file, run_time, rampup)¶
Bases: multiprocessing.process.Process
- run()¶
- oct.multimechanize.core.init(projects_dir, project_name)¶
Sanity check that all test scripts can be loaded.
- oct.multimechanize.core.load_script(script_file)¶
Load a test scripts as Python module.
Returns: Imported script as python module.
oct.multimechanize.dependency_checker module¶
script to verify all multi-mechanize dependencies are satisfied
oct.multimechanize.graph module¶
- oct.multimechanize.graph.resp_graph(avg_resptime_points_dict, percentile_80_resptime_points_dict, percentile_90_resptime_points_dict, image_name, dir='./')¶
- oct.multimechanize.graph.resp_graph_raw(nested_resp_list, image_name, dir='./')¶
- oct.multimechanize.graph.tp_graph(throughputs_dict, image_name, dir='./')¶
oct.multimechanize.progressbar module¶
oct.multimechanize.reportwriter module¶
oct.multimechanize.reportwriterxml module¶
- oct.multimechanize.reportwriterxml.write_jmeter_output(mm_data, output_path)¶
Take the list of ResponseStats objects and write a JMeter 2.1 formatted XML file to output_path.
JMeter JTL file documentation: http://jakarta.apache.org/jmeter/usermanual/listeners.html
oct.multimechanize.results module¶
- class oct.multimechanize.results.ResponseStats(request_num, elapsed_time, epoch_secs, user_group_name, trans_time, error, custom_timers)¶
Bases: object
- class oct.multimechanize.results.Results(results_file_name, run_time)¶
Bases: object
- oct.multimechanize.results.average(seq)¶
- oct.multimechanize.results.output_results(results_dir, results_file, run_time, rampup, ts_interval, user_group_configs=None, xml_reports=False, parent='../../')¶
- oct.multimechanize.results.percentile(seq, percentile)¶
- oct.multimechanize.results.split_series(points, interval)¶
- oct.multimechanize.results.standard_dev(seq)¶
oct.multimechanize.resultsloader module¶
oct.multimechanize.resultswriter module¶
oct.multimechanize.rpcserver module¶
- class oct.multimechanize.rpcserver.RemoteControl(project_name, run_callback)¶
Bases: object
- check_test_running()¶
- get_config()¶
- get_project_name()¶
- get_results()¶
- run_test()¶
- update_config(config)¶
- oct.multimechanize.rpcserver.launch_rpc_server(bind_addr, port, project_name, run_callback)¶
oct.multimechanize.script_loader module¶
REPLACE: multi-mechanize exec()/eval() magic with real imports.
- exception oct.multimechanize.script_loader.InvalidScriptError¶
Bases: exceptions.StandardError
Should be raised when a Script does not confirm to required interface.
- SCRIPT INTERFACE:
- Transaction class exists.
- Transaction.run() method exists.
- class oct.multimechanize.script_loader.ScriptLoader¶
Bases: object
Utility class to load scripts as python modules.
- static load(path)¶
Load a script by using a path.
Returns: Loaded script module- Raise: ImportError, when script module cannot be loaded.
- classmethod load_all(scripts_path, validate=False)¶
Load all python scripts in a path.
Returns: Loaded script modules as dictionary.
- class oct.multimechanize.script_loader.ScriptValidator¶
Bases: object
Utility class to ensure that scripts are valid and conforms to conventions.
- static check_module_invalid(module)¶
Check if a script module is invalid and does not comply w/ conventions
Returns: Problem as string, if any is found. Returns: None, if no problems are detected.
- classmethod ensure_module_valid(module)¶
Ensures that a script module is valid.
Raises: InvalidScriptError, if any convention is violated.
Module contents¶
oct.testing package¶
Module contents¶
- oct.testing.content.must_contain(resp, pattern)¶
Test if the pattern is in content
Parameters: - pattern (str) – pattern to find
- resp – a response object
Returns: None
Raise: AssertionError
- oct.testing.content.must_not_contain(resp, pattern)¶
Test if the pattern is not in content
Parameters: - pattern (str) – pattern to find
- resp – a response object
Returns: None
Raise: AssertionError
- oct.testing.response.check_response_status(resp, status)¶
This will check is the response_code is equal to the status
Parameters: - resp – a response object
- status (int) – the expected status
Returns: None
Raise: AssertionError
oct.tools package¶
oct.tools contain two functions. One who can be called directly in the shell
octtools-user-generator
email_generator_func
How to¶
octtools-user-generator¶
is the command line to generate either user or email WITH their password
occtools-user-generator must have a CSV file provided and have multiple optional arguments
MUST HAVE THIS ONE
-h [CSV File]
[[OPTIONAL]]
-n [nb_item] Number of items generated
-s [size] Size of each user/email/password generated
-w [type u = user, e = email] What you want to generate
Default value of each options
-n => 250 items
-s => item with lenght of 6
-w => e (generate email by default)
Exemple¶
octtools-user-generator userfile.csv -n 25000 -s 6 -w u
This command line will generate 25000 email/password with a lenght of 6 in “userfile.csv”
email_generator_func()¶
Is a function with multiple agruments some have a default value
csvfile
what = Define what you want to generate u = user, e = email.
number = Define how many items you want to generate
size = Define the size of each items
chars = Define with 'what' you want to generate you item
Default value of each options
number = 15
size = 6
char = string.ascii_lowercase
Exemple¶
email_generator_func("csvfile.csv", "u", 15000, 7):
This command line will generate 15000 user/password with a lenght of 7 in “csvfile.csv”
oct.tools.email_generator module¶
- oct.tools.email_generator.email_generator()¶
Command line tool for generating csv file containing user / password pairs
Parameters: - xmlfile – name of XML file provided
- number_of_email – Number of random generated email
- size – number of char generated
- chars – lower the generated char
Type: int
Type: int
Type: string
Returns: None
- oct.tools.email_generator.email_generator_func(csvfile, what, number_of_email=15, size=6, chars='abcdefghijklmnopqrstuvwxyz')¶
Parameters: - xmlfile – name of XML file provided
- number_of_email – Number of random generated email
- size – number of char generated
- chars – lower the generated char
Type: int
Type: int
Type: string
Returns: None
oct.tools.xmltocsv module¶
- oct.tools.xmltocsv.sitemap_to_csv()¶
Take as options: Xml file, CSV File
Parse the XML and write each value get from it inside the CSV file provided. :return: None
Module contents¶
oct.utilities package¶
This module provide you basics shell commands to easily use the OCT module.
newproject module¶
This module provide you a command to create a new project directory. Once the library is installed you can use :
oct-newproject <project-name>
It will create a new directory named project-name with basic stuff inside.
The project structure must look like this :
.
├── config.cfg
├── templates
│ ├── css
│ │ └── style.css
│ ├── footer.html
│ ├── head.html
│ ├── img
│ └── scripts
└── test_scripts
└── v_user.py
With this basic project structure you can start writing your scripts, customize your templates, etc...
This project can be run with the command
multimech-run <project>
But for testing your scripts you can simply run them by using your standard python interpreter
The file config.cfg contain all configuration variable for your project. By default it’s look like this
[global]
run_time = 30
rampup = 0
results_ts_interval = 10
progress_bar = on
console_logging = off
xml_report = off
base_url = http://localhost
default_sleep_time = 2
statics_enabled = 1
[user_group-1]
threads = 3
script = v_user.py
[user_group-2]
threads = 3
script = v_user.py
For explanations :
This file give you two virtual user groups, each group has 3 user, and the user script is the v_user.py file.
To see all configuration variables explained see the Configuration section
Module doc¶
- oct.utilities.newproject.create_project(project_name, config_name='config.cfg', script_name='v_user.py', scripts_dir='test_scripts', config_content='\n[global]\nrun_time = 30\nrampup = 0\nresults_ts_interval = 10\nprogress_bar = on\nconsole_logging = off\nxml_report = off\nbase_url = http://localhost\ndefault_sleep_time = 2\nstatics_enabled = 1\n\n\n[user_group-1]\nthreads = 3\nscript = v_user.py\n\n[user_group-2]\nthreads = 3\nscript = v_user.py\n\n', script_content="\nfrom oct.core.generic import GenericTransaction\nimport random\nimport time\nimport os\n\n\nCONFIG_PATH = os.path.join(os.path.dirname(os.path.abspath(__file__)), '../')\n\n\nclass Transaction(GenericTransaction):\n def __init__(self):\n GenericTransaction.__init__(self, True, CONFIG_PATH)\n\n def run(self):\n r = random.uniform(1, 2)\n time.sleep(r)\n self.custom_timers['Example_Timer'] = r\n\n\nif __name__ == '__main__':\n trans = Transaction()\n trans.run()\n print trans.custom_timers\n", template_dir='templates', head_content='\n<!DOCTYPE html>\n<html>\n\n<head>\n <title> OCT | Results </title>\n <link rel="stylesheet" type="text/css" href="css/style.css" />\n</head>\n\n<body id="main">\n', footer_content='\n</body>\n</html>\n')¶
- oct.utilities.newproject.main()¶
run module¶
TODO
WORK IN PROGRESS
This module give you access to the command :
oct-run <project>
This command will run your project using celery. For now the broker can only be configured in source code. It’s bind on a standard rabbitmq server.
The goal of this command is to run the same project in several rabbitmq instance.
Module doc¶
celery module¶
The configuration for running celery