wltp gear-shifts calculator¶

Overview¶

The calculator accepts as input the vehicle’s technical data, along with parameters for modifying the execution of the WLTC cycle, and it then spits-out the gear-shifts of the vehicle, the attained speed-profile, and any warnings. It does not calculate any CO₂ emissions.

An “execution” or a “run” of an experiment is depicted in the following diagram:

           .---------------------.                         .----------------------------.
          ;     Input-Model     ;                         ;        Output-Model        ;
         ;---------------------;                         ;----------------------------;
        ; +--vehicle          ;     ____________        ; +---...                    ;
       ;  +--params          ;     |            |      ;  +--cycle_run:             ;
      ;       +--wltc_data  ;  ==> | Experiment | ==> ;      t  v_class gear ...   ;
     ;                     ;       |____________|    ;      --------------------  ;
    ;                     ;                         ;       00      0.0    1     ;
   ;                     ;                         ;        01      1.3    1    ;
  ;                     ;                         ;         02      5.5    1   ;
 ;                     ;                         ;          ...               ;
'---------------------'                         '----------------------------.

The Input & Output Data are instances of pandas-model, trees of strings and numbers, assembled with:

• sequences,
• dictionaries,
• pandas.DataFrame,
• pandas.Series, and
• URI-references to other model-trees.

Quick-start¶

On Windows/OS X, it is recommended to use one of the following “scientific” python-distributions, as they already include the native libraries and can install without administrative priviledges:

• WinPython (Windows only),
• Anaconda,
• Canopy,

Assuming you have a working python-environment, open a command-shell, (in Windows use cmd.exe BUT ensure python.exe is in its PATH), you can try the following commands:

$ pip install wltp --pre
$ wltp --winmenus                       ## Adds StartMenu-items, Windows only.

$ wltp --version
0.0.9-alpha.3

$ wltp --help
...

$ wltp --gui`                           ## For exploring model, but not ready yet.

$ wltp --excelrun                       ## Windows & OS X only

from wltp.experiment import Experiment

input_model = { ... }           ## See also "Python Usage" for model contents.
exp = Experiment(input_model)
output_model = exp.run()
print('Results: \n%s' % output_model['cycle_run'])

* git, git-completion
* make, zip, unzip, bzip2
* openssh, curl, wget

Discussion¶

Older versions¶

An additional purpose of the versioning schema of the project is to track which specific version of the GTR it implements. Given a version number MAJOR.MINOR.PATCH, the MAJOR part tracks the GTR phase implemented. See the “GTR version matrix” section in Changes for the mapping of MAJOR-numbers to GTR versions.

To install an older version issue the console command:

$ pip install wltp=1.1.1                    ## Use `--pre` if version-string has a build-suffix.

If you have another version already installed, you have to use --ignore-installed (or -I). For using the specific version, check this (untested) stackoverflow question .

Of course it is better to install each version in a separate virtualenv (isolated Python environment) and shy away from all this.

Installing from sources¶

If you download the sources you have more options for installation. There are various methods to get hold of them:

• Download the source distribution from PyPi repo.

• Download a release-snapshot from github

• Clone the git-repository at github.

  Assuming you have a working installation of git you can fetch and install the latest version of the project with the following series of commands:

  $ git clone "https://github.com/ankostis/wltp.git" wltp.git
  $ cd wltp.git
  $ python setup.py install                                 ## Use `python3` if both python-2 & 3 installed.
  

When working with sources, you need to have installed all libraries that the project depends on:

$ pip install -r requirements/execution.txt .

The previous command installs a “snapshot” of the project as it is found in the sources. If you wish to link the project’s sources with your python environment, install the project in development mode:

$ python setup.py develop

Project files and folders¶

The files and folders of the project are listed below:

+--wltp/            ## (package) The python-code of the calculator
|   +--cycles/      ## (package) The python-code for the WLTC data
|   +--test/        ## (package) Test-cases and the wltp_db
|   +--model        ## (module) Describes the data and their schema for the calculation
|   +--experiment   ## (module) The calculator
|   +--plots        ## (module) Diagram-plotting code and utilities
+--docs/            ## Documentation folder
|   +--pyplots/     ## (scripts) Plot the metric diagrams embeded in the README
+--devtools/        ## (scripts) Preprocessing of WLTC data on GTR and the wltp_db
|   +--run_tests.sh ## (script) Executes all TestCases
+--wltp             ## (script) The cmd-line entry-point script for the calculator
+--setup.py         ## (script) The entry point for `setuptools`, installing, testing, etc
+--requirements/    ## (txt-files) Various pip-dependencies for tools.
+--README.rst
+--CHANGES.rst
+--LICENSE.txt

Discussion¶

Cmd-line usage¶

The command-line usage below requires the Python environment to be installed, and provides for executing an experiment directly from the OS’s shell (i.e. cmd in windows or bash in POSIX), and in a single command. To have precise control over the inputs and outputs (i.e. experiments in a “batch” and/or in a design of experiments) you have to run the experiments using the API python, as explained below.

The entry-point script is called wltp, and it must have been placed in your PATH during installation. This script can construct a model by reading input-data from multiple files and/or overriding specific single-value items. Conversely, it can output multiple parts of the resulting-model into files.

To get help for this script, use the following commands:

$ wltp --help                               ## to get generic help for cmd-line syntax
$ wltcmdp.py -M vehicle/full_load_curve     ## to get help for specific model-paths

and then, assuming vehicle.csv is a CSV file with the vehicle parameters for which you want to override the n_idle only, run the following:

$ wltp -v \
    -I vehicle.csv file_frmt=SERIES model_path=params header@=None \
    -m vehicle/n_idle:=850 \
    -O cycle.csv model_path=cycle_run

GUI usage¶

For a quick-‘n-dirty method to explore the structure of the model-tree and run an experiment, just run:

$ wltp --gui

Excel usage¶

In Windows and OS X you may utilize the excellent xlwings library to use Excel files for providing input and output to the experiment.

To create the necessary template-files in your current-directory you should enter:

$ wltp --excel

You could type instead wltp --excel file_path to specify a different destination path.

In windows/OS X you can type wltp --excelrun and the files will be created in your home-directory and the excel will open them in one-shot.

All the above commands creates two files:

wltp_excel_runner.xlsm

The python-enabled excel-file where input and output data are written, as seen in the screenshot below:

After opening it the first tie, enable the macros on the workbook, select the python-code at the left and click the Run Selection as Pyhon button; one sheet per vehicle should be created.

The excel-file contains additionally appropriate VBA modules allowing you to invoke Python code present in selected cells with a click of a button, and python-functions declared in the python-script, below, using the mypy namespace.

To add more input-columns, you need to set as column Headers the json-pointers path of the desired model item (see Python usage below,).

wltp_excel_runner.py

Utility python functions used by the above xls-file for running a batch of experiments.

The particular functions included reads multiple vehicles from the input table with various vehicle characteristics and/or experiment parameters, and then it adds a new worksheet containing the cycle-run of each vehicle . Of course you can edit it to further fit your needs.

Some general notes regarding the python-code from excel-cells:

• On each invocation, the predefined VBA module pandalon executes a dynamically generated python-script file in the same folder where the excel-file resides, which, among others, imports the “sister” python-script file. You can read & modify the sister python-script to import libraries such as ‘numpy’ and ‘pandas’, or pre-define utility python functions.
• The name of the sister python-script is automatically calculated from the name of the Excel-file, and it must be valid as a python module-name. Therefore do not use non-alphanumeric characters such as spaces(` ), dashes(-) and dots(.`) on the Excel-file.
• On errors, a log-file is written in the same folder where the excel-file resides, for as long as the message-box is visible, and it is deleted automatically after you click ‘ok’!
• Read http://docs.xlwings.org/quickstart.html

Python usage¶

Example python REPL example-commands are given below that setup and run an experiment.

First run python or ipython and try to import the project to check its version:

>>> import wltp

>>> wltp.__version__            ## Check version once more.
'0.0.9-alpha.3'

>>> wltp.__file__               ## To check where it was installed.         
/usr/local/lib/site-package/wltp-...

If everything works, create the pandas-model that will hold the input-data (strings and numbers) of the experiment. You can assemble the model-tree by the use of:

• sequences,
• dictionaries,
• pandas.DataFrame,
• pandas.Series, and
• URI-references to other model-trees.

For instance:

>>> from wltp import model
>>> from wltp.experiment import Experiment
>>> from collections import OrderedDict as odic         ## It is handy to preserve keys-order.

>>> mdl = odic(
...   vehicle = odic(
...     unladen_mass = 1430,
...     test_mass    = 1500,
...     v_max        = 195,
...     p_rated      = 100,
...     n_rated      = 5450,
...     n_idle       = 950,
...     n_min        = None,                            ## Manufacturers my overridde it
...     gear_ratios         = [120.5, 75, 50, 43, 37, 32],
...     resistance_coeffs   = [100, 0.5, 0.04],
...   )
... )

For information on the accepted model-data, check its JSON-schema:

>>> model.json_dumps(model.model_schema(), indent=2)                                
{
  "properties": {
    "params": {
      "properties": {
        "f_n_min_gear2": {
          "description": "Gear-2 is invalid when N :< f_n_min_gear2 * n_idle.",
          "type": [
            "number",
            "null"
          ],
          "default": 0.9
        },
        "v_stopped_threshold": {
          "description": "Velocity (Km/h) under which (<=) to idle gear-shift (Annex 2-3.3, p71).",
          "type": [
...

You then have to feed this model-tree to the Experiment constructor. Internally the Pandel resolves URIs, fills-in default values and validates the data based on the project’s pre-defined JSON-schema:

>>> processor = Experiment(mdl)         ## Fills-in defaults and Validates model.

Assuming validation passes without errors, you can now inspect the defaulted-model before running the experiment:

>>> mdl = processor.model               ## Returns the validated model with filled-in defaults.
>>> sorted(mdl)                         ## The "defaulted" model now includes the `params` branch.
['params', 'vehicle']
>>> 'full_load_curve' in mdl['vehicle'] ## A default wot was also provided in the `vehicle`.
True

Now you can run the experiment:

>>> mdl = processor.run()               ## Runs experiment and augments the model with results.
>>> sorted(mdl)                         ## Print the top-branches of the "augmented" model.
['cycle_run', 'params', 'vehicle']

To access the time-based cycle-results it is better to use a pandas.DataFrame:

>>> import pandas as pd
>>> df = pd.DataFrame(mdl['cycle_run']); df.index.name = 't'
>>> df.shape                            ## ROWS(time-steps) X COLUMNS.
(1801, 11)
>>> df.columns
Index(['v_class', 'v_target', 'clutch', 'gears_orig', 'gears', 'v_real', 'p_available', 'p_required', 'rpm', 'rpm_norm', 'driveability'], dtype='object')
>>> 'Mean engine_speed: %s' % df.rpm.mean()
'Mean engine_speed: 1917.0407829'
>>> df.describe()
           v_class     v_target     clutch   gears_orig        gears  \
count  1801.000000  1801.000000       1801  1801.000000  1801.000000
mean     46.506718    46.506718  0.0660744     3.794003     3.683509
std      36.119280    36.119280  0.2484811     2.278959     2.278108
...

            v_real  p_available   p_required          rpm     rpm_norm
count  1801.000000  1801.000000  1801.000000  1801.000000  1801.000000
mean     50.356222    28.846639     4.991915  1917.040783     0.214898
std      32.336908    15.833262    12.139823   878.139758     0.195142
...

>>> processor.driveability_report()                                             
...
  12: (a: X-->0)
  13: g1: Revolutions too low!
  14: g1: Revolutions too low!
...
  30: (b2(2): 5-->4)
...
  38: (c1: 4-->3)
  39: (c1: 4-->3)
  40: Rule e or g missed downshift(40: 4-->3) in acceleration?
...
  42: Rule e or g missed downshift(42: 3-->2) in acceleration?
...

You can export the cycle-run results in a CSV-file with the following pandas command:

>>> df.to_csv('cycle_run.csv')                                                      

For more examples, download the sources and check the test-cases found under the /wltp/test/ folder.

IPython notebook usage¶

The list of IPython notebooks for wltp is maintained at the wiki of the project.

Discussion¶

Sources & Dependencies¶

To get involved with development, you need a POSIX environment to fully build it (Linux, OSX or Cygwin on Windows).

First you need to download the latest sources:

$ git clone https://github.com/ankostis/wltp.git wltp.git
$ cd wltp.git

Then you can install all project’s dependencies in `development mode using the setup.py script:

$ python setup.py --help                           ## Get help for this script.
Common commands: (see '--help-commands' for more)

  setup.py build      will build the package underneath 'build/'
  setup.py install    will install the package

Global options:
  --verbose (-v)      run verbosely (default)
  --quiet (-q)        run quietly (turns verbosity off)
  --dry-run (-n)      don't actually do anything
...

$ python setup.py develop                           ## Also installs dependencies into project's folder.
$ python setup.py build                             ## Check that the project indeed builds ok.

You should now run the test-cases (see ref:metrics, below) to check that the sources are in good shape:

$ python setup.py test

$ deactivate
$ python setup.py develop

$ python setup.py install                # May require admin-rights

Development procedure¶

For submitting code, use UTF-8 everywhere, unix-eol(LF) and set git --config core.autocrlf = input.

The typical development procedure is like this:

1. Modify the sources in small, isolated and well-defined changes, i.e. adding a single feature, or fixing a specific bug.

2. Add test-cases “proving” your code.

3. Rerun all test-cases to ensure that you didn’t break anything, and check their coverage remain above 80%:

   $ python setup.py nosetests --with-coverage --cover-package wltp.model,wltp.experiment --cover-min-percentage=80
   

   Tip

   You can enter just: python setup.py test_all instead of the above cmd-line since it has been aliased in the setup.cfg file. Check this file for more example commands to use during development.

4. If you made a rather important modification, update also the Changes file and/or other documents (i.e. README.rst). To see the rendered results of the documents, issue the following commands and read the result html at build/sphinx/html/index.html:

   $ python setup.py build_sphinx                  # Builds html docs
   $ python setup.py build_sphinx -b doctest       # Checks if python-code embeded in comments runs ok.
   

5. If there are no problems, commit your changes with a descriptive message.

6. Repeat this cycle for other bugs/enhancements.

7. When you are finished, push the changes upstream to github and make a merge_request. You can check whether your merge-request indeed passed the tests by checking its build-status on the integration-server’s site (TravisCI).

   Hint

   Skim through the small IPython developer’s documentantion on the matter: The perfect pull request

Specs & Algorithm¶

This program was implemented from scratch based on this GTR specification (included in the docs/ folder). The latest version of this GTR, along with other related documents can be found at UNECE’s site:

• http://www.unece.org/trans/main/wp29/wp29wgs/wp29grpe/grpedoc_2013.html
• https://www2.unece.org/wiki/pages/viewpage.action?pageId=2523179
• Probably a more comprehensible but older spec is this one: https://www2.unece.org/wiki/display/trans/DHC+draft+technical+report

The WLTC-profiles for the various classes in the devtools/data/cycles/ folder were generated from the tables of the specs above using the devtools/csvcolumns8to2.py script, but it still requires an intermediate manual step involving a spreadsheet to copy the table into ands save them as CSV.

Then use the devtools/buildwltcclass.py to construct the respective python-vars into the wltp/model.py sources.

Data-files generated from Steven Heinz’s ms-access vehicle info db-table can be processed with the devtools/preprocheinz.py script.

Cycles¶

Tests, Metrics & Reports¶

In order to maintain the algorithm stable, a lot of effort has been put to setup a series of test-case and metrics to check the sanity of the results and to compare them with the Heinz-db tool or other datasets included in the project. These tests can be found in the wltp/test/ folders.

Additionally, below are auto-generated representative diagrams with the purpose to track the behavior and the evolution of this project.

You can reuse the plotting code here for building nice ipython-notebooks reports, and (optionally) link them in the wiki of the project (see section above). The actual code for generating diagrams for these metrics is in wltp.plots and it is invoked by scripts in the docs/pyplot/ folder.

Development team¶

• Author:

  • Kostis Anagnostopoulos

• Contributing Authors:

  • Heinz Steven (test-data, validation and review)
  • Georgios Fontaras (simulation, physics & engineering support)
  • Alessandro Marotta (policy support)

Discussion¶

General¶

Technical¶

Module: wltp.experiment¶

Module: wltp.model¶

Module: wltp.pandel¶

Module: wltp.test.samples_db_tests¶

Module: wltp.test.wltp_db_tests¶

GTR version matrix¶

Given a version number MAJOR.MINOR.PATCH, the MAJOR part tracks the GTR phase implemented. The following matrix shows these correspondences:

Known deficiencies¶

• (!) Driveability-rules not ordered as defined in the latest task-force meeting.
• (!) The driveability-rules when speeding down to a halt is broken, and human-drivers should improvise.
• (!) The n_min_drive is not calculated as defined in the latest task-force meeting, along with other recent updates.
• (!) The n_max is calculated for ALL GEARS, resulting in “clipped” velocity-profiles, leading to reduced rpm’s for low-powered vehicles.
• Clutching-points and therefore engine-speed are very preliminary (ie rpm when starting from stop might be < n_idle).

TODOs¶

• Add cmd-line front-end.
• Automatically calculate masses from H & L vehicles, and regression-curves from categories.
• wltp_db: Improve test-metrics with group-by classes/phases.
• model: Enhance model-preprocessing by interleaving “octapus” merging stacked-models between validation stages.
• model: finalize data-schema (renaming columns and adding name fields in major blocks).
• model/core: Accept units on all quantities.
• core: Move calculations as class-methods to provide for overriding certain parts of the algorithm.
• core: Support to provide and override arbitrary model-data, and ask for arbitrary output-ones by topologically sorting the graphs of the calculation-dependencies.
• build: Separate wltpdb tests as a separate, optional, plugin of this project (~650Mb size).

Releases¶

Glossary¶

WLTP

The Worldwide harmonised Light duty vehicles Test Procedure, a GRPE informal working group

UNECE

The United Nations Economic Commission for Europe, which has assumed the steering role on the WLTP.

GRPE

UNECE Working party on Pollution and Energy - Transport Programme

GS Task-Force

The Gear-shift Task-force of the GRPE. It is the team of automotive experts drafting the gear-shifting strategy for vehicles running the WLTP cycles.

WLTC

The family of pre-defined driving-cycles corresponding to vehicles with different PMR. Classes 1,2, 3a & 3b are split in 2, 4, 4 and 4 parts respectively.

Unladen mass

UM or Curb weight, the weight of the vehicle in running order minus the mass of the driver.

Test mass

TM, the representative weight of the vehicle used as input for the calculations of the simulation, derived by interpolating between high and low values for the CO₂-family of the vehicle.

Downscaling

Reduction of the top-velocity of the original drive trace to be followed, to ensure that the vehicle is not driven in an unduly high proportion of “full throttle”.

pandas-model

The container of data that the gear-shift calculator consumes and produces. It is implemented by wltp.pandel.Pandel as a mergeable stack of JSON-schema abiding trees of strings and numbers, formed with sequences, dictionaries, pandas-instances and URI-references.

JSON-schema

The JSON schema is an IETF draft that provides a contract for what JSON-data is required for a given application and how to interact with it. JSON Schema is intended to define validation, documentation, hyperlink navigation, and interaction control of JSON data. You can learn more about it from this excellent guide, and experiment with this on-line validator.

JSON-pointer

JSON Pointer(RFC 6901) defines a string syntax for identifying a specific value within a JavaScript Object Notation (JSON) document. It aims to serve the same purpose as XPath from the XML world, but it is much simpler.