tox-envreport

tox-envreport is a plugin for tox to document the setup of used virtual environments.

By installing this plugin, 2 types of files are created automatically during each tox run:

env-report.json

A virtual environment specific file, which is created for each used virtual environment.

Stored inside each virtual environment folder. E.g: .tox/py34/env-report.json.

tox-report.json

A global report file, which bundles data from all virtual environment specific files.

Stored inside .tox folder. E.g: .tox/tox-report.json.

Warning

This plugin is an early alpha version and under heavy development. The used structure inside export files may change in future.

A report contains the following information and more:

• System: host, platform,
• Virtual environment: name, path, tested package (incl. hashes)
• Python: executable, version, installed packages (incl. version)
• Executed commands : setup, test (all incl. output and return code)

Installation

pip install tox-envreport

Usage

The installation of the plugin is enough to automatically activate it for each tox run.

There is nothing more to do or to configure.

Access reports

tox-envreport creates one common tox-report.json file, which contains all information about all used virtual environments (venv) by tox.

Beside this a venv specific file is generated as well and contains information about the related venv only. You can find this file in the venv-specific tox-folder. E.g: MY_PROJECT/.tox/py27/env-report.json for a venv called py27.

Motivation

tox-envreport was created for an automotive project, which needs to archive beside test results also used test environments. The goal is to provide enough information to be able to setup an identical test environment in 20+ years.

tox-envreport is part of a software bundle, which was designed to fulfill the parameters of the ISO 26262 standard for safety critical software in automotive companies. Other tools are: sphinx-needs.

Content

Examples

env-report.json

{
    "host": "daniel-dev",
    "installed_packages": [
        "attrs==17.4.0",
        "more-itertools==4.1.0",
        "pluggy==0.6.0",
        "py==1.5.3",
        "pytest==3.5.0",
        "six==1.11.0",
        "tox==3.0.0",
        "tox-envreport==0.1.0",
        "virtualenv==15.2.0"
    ],
    "installpkg": {
        "basename": "tox-envreport-0.1.0.zip",
        "md5": "5046419f076f6fd825319874614a90d0",
        "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
    },
    "name": "py35",
    "path": "/home/daniel/workspace/tox-envreport/.tox/py35",
    "platform": "linux",
    "python": {
        "executable": "/home/daniel/workspace/tox-envreport/.tox/py35/bin/python",
        "version": "3.5.2 (default, Nov 23 2017, 16:37:01) \n[GCC 5.4.0 20160609]",
        "version_info": [
            3,
            5,
            2,
            "final",
            0
        ]
    },
    "reportversion": "1",
    "setup": [
        {
            "command": [
                "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip",
                "install",
                "-U",
                "--no-deps",
                "/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip"
            ],
            "output": "actionid: py35\nmsg: installpkg\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip', 'install', '-U', '--no-deps', '/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip']\n\nProcessing ./.tox/dist/tox-envreport-0.1.0.zip\nBuilding wheels for collected packages: tox-envreport\n  Running setup.py bdist_wheel for tox-envreport: started\n  Running setup.py bdist_wheel for tox-envreport: finished with status 'done'\n  Stored in directory: /home/daniel/.cache/pip/wheels/da/40/e3/1954e2a10b42ec16b0a8ecb694e438fc3894a7a3c233810269\nSuccessfully built tox-envreport\nInstalling collected packages: tox-envreport\n  Found existing installation: tox-envreport 0.1.0\n    Uninstalling tox-envreport-0.1.0:\n      Successfully uninstalled tox-envreport-0.1.0\nSuccessfully installed tox-envreport-0.1.0\n",
            "retcode": "0"
        },
        {
            "command": [
                "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip",
                "freeze"
            ],
            "output": "actionid: py35\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip', 'freeze']\n\nattrs==17.4.0\nmore-itertools==4.1.0\npluggy==0.6.0\npy==1.5.3\npytest==3.5.0\nsix==1.11.0\ntox==3.0.0\ntox-envreport==0.1.0\nvirtualenv==15.2.0\n",
            "retcode": "0"
        }
    ],
    "test": [
        {
            "command": [
                "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pytest",
                "tests"
            ],
            "output": null,
            "retcode": "0"
        }
    ],
    "toxversion": "3.0.0"
}

tox-report.json

The following json-code contains information about 5 virtual environments: flake8, py27, py34, py35, py36.

{
    "flake8": {
        "host": "daniel-dev",
        "installed_packages": [
            "flake8==3.5.0",
            "mccabe==0.6.1",
            "pycodestyle==2.3.1",
            "pyflakes==1.6.0"
        ],
        "installpkg": {
            "basename": "tox-envreport-0.1.0.zip",
            "md5": "5046419f076f6fd825319874614a90d0",
            "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
        },
        "name": "flake8",
        "path": "/home/daniel/workspace/tox-envreport/.tox/flake8",
        "platform": "linux",
        "python": {
            "executable": "/home/daniel/workspace/tox-envreport/.tox/flake8/bin/python",
            "version": "3.5.2 (default, Nov 23 2017, 16:37:01) \n[GCC 5.4.0 20160609]",
            "version_info": [
                3,
                5,
                2,
                "final",
                0
            ]
        },
        "reportversion": "1",
        "setup": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/flake8/bin/pip",
                    "freeze"
                ],
                "output": "actionid: flake8\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/flake8/bin/pip', 'freeze']\n\nflake8==3.5.0\nmccabe==0.6.1\npycodestyle==2.3.1\npyflakes==1.6.0\n",
                "retcode": "0"
            }
        ],
        "test": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/flake8/bin/flake8",
                    "--show-source"
                ],
                "output": null,
                "retcode": "0"
            }
        ],
        "toxversion": "3.0.0"
    },
    "py27": {
        "host": "daniel-dev",
        "installed_packages": [
            "attrs==17.4.0",
            "funcsigs==1.0.2",
            "more-itertools==4.1.0",
            "pluggy==0.6.0",
            "py==1.5.3",
            "pytest==3.5.0",
            "six==1.11.0",
            "tox==3.0.0",
            "tox-envreport==0.1.0",
            "virtualenv==15.2.0"
        ],
        "installpkg": {
            "basename": "tox-envreport-0.1.0.zip",
            "md5": "5046419f076f6fd825319874614a90d0",
            "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
        },
        "name": "py27",
        "path": "/home/daniel/workspace/tox-envreport/.tox/py27",
        "platform": "linux",
        "python": {
            "executable": "/home/daniel/workspace/tox-envreport/.tox/py27/bin/python",
            "version": "2.7.12 (default, Nov 20 2017, 18:23:56) \n[GCC 5.4.0 20160609]",
            "version_info": [
                2,
                7,
                12,
                "final",
                0
            ]
        },
        "reportversion": "1",
        "setup": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py27/bin/pip",
                    "install",
                    "-U",
                    "--no-deps",
                    "/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip"
                ],
                "output": "actionid: py27\nmsg: installpkg\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py27/bin/pip', 'install', '-U', '--no-deps', '/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip']\n\nProcessing ./.tox/dist/tox-envreport-0.1.0.zip\nBuilding wheels for collected packages: tox-envreport\n  Running setup.py bdist_wheel for tox-envreport: started\n  Running setup.py bdist_wheel for tox-envreport: finished with status 'done'\n  Stored in directory: /home/daniel/.cache/pip/wheels/da/40/e3/1954e2a10b42ec16b0a8ecb694e438fc3894a7a3c233810269\nSuccessfully built tox-envreport\nInstalling collected packages: tox-envreport\n  Found existing installation: tox-envreport 0.1.0\n    Uninstalling tox-envreport-0.1.0:\n      Successfully uninstalled tox-envreport-0.1.0\nSuccessfully installed tox-envreport-0.1.0\n",
                "retcode": "0"
            },
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py27/bin/pip",
                    "freeze"
                ],
                "output": "actionid: py27\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py27/bin/pip', 'freeze']\n\nattrs==17.4.0\nfuncsigs==1.0.2\nmore-itertools==4.1.0\npluggy==0.6.0\npy==1.5.3\npytest==3.5.0\nsix==1.11.0\ntox==3.0.0\ntox-envreport==0.1.0\nvirtualenv==15.2.0\n",
                "retcode": "0"
            }
        ],
        "test": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py27/bin/pytest",
                    "tests"
                ],
                "output": null,
                "retcode": "0"
            }
        ],
        "toxversion": "3.0.0"
    },
    "py34": {
        "host": "daniel-dev",
        "installed_packages": [
            "attrs==17.4.0",
            "more-itertools==4.1.0",
            "pluggy==0.6.0",
            "py==1.5.3",
            "pytest==3.5.0",
            "six==1.11.0",
            "tox==3.0.0",
            "tox-envreport==0.1.0",
            "virtualenv==15.2.0"
        ],
        "installpkg": {
            "basename": "tox-envreport-0.1.0.zip",
            "md5": "5046419f076f6fd825319874614a90d0",
            "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
        },
        "name": "py34",
        "path": "/home/daniel/workspace/tox-envreport/.tox/py34",
        "platform": "linux",
        "python": {
            "executable": "/home/daniel/workspace/tox-envreport/.tox/py34/bin/python",
            "version": "3.4.5 (default, Jul 15 2016, 16:39:07) \n[GCC 5.4.0 20160609]",
            "version_info": [
                3,
                4,
                5,
                "final",
                0
            ]
        },
        "reportversion": "1",
        "setup": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py34/bin/pip",
                    "install",
                    "-U",
                    "--no-deps",
                    "/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip"
                ],
                "output": "actionid: py34\nmsg: installpkg\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py34/bin/pip', 'install', '-U', '--no-deps', '/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip']\n\nProcessing ./.tox/dist/tox-envreport-0.1.0.zip\nBuilding wheels for collected packages: tox-envreport\n  Running setup.py bdist_wheel for tox-envreport: started\n  Running setup.py bdist_wheel for tox-envreport: finished with status 'done'\n  Stored in directory: /home/daniel/.cache/pip/wheels/da/40/e3/1954e2a10b42ec16b0a8ecb694e438fc3894a7a3c233810269\nSuccessfully built tox-envreport\nInstalling collected packages: tox-envreport\n  Found existing installation: tox-envreport 0.1.0\n    Uninstalling tox-envreport-0.1.0:\n      Successfully uninstalled tox-envreport-0.1.0\nSuccessfully installed tox-envreport-0.1.0\n",
                "retcode": "0"
            },
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py34/bin/pip",
                    "freeze"
                ],
                "output": "actionid: py34\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py34/bin/pip', 'freeze']\n\nattrs==17.4.0\nmore-itertools==4.1.0\npluggy==0.6.0\npy==1.5.3\npytest==3.5.0\nsix==1.11.0\ntox==3.0.0\ntox-envreport==0.1.0\nvirtualenv==15.2.0\n",
                "retcode": "0"
            }
        ],
        "test": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py34/bin/pytest",
                    "tests"
                ],
                "output": null,
                "retcode": "0"
            }
        ],
        "toxversion": "3.0.0"
    },
    "py35": {
        "host": "daniel-dev",
        "installed_packages": [
            "attrs==17.4.0",
            "more-itertools==4.1.0",
            "pluggy==0.6.0",
            "py==1.5.3",
            "pytest==3.5.0",
            "six==1.11.0",
            "tox==3.0.0",
            "tox-envreport==0.1.0",
            "virtualenv==15.2.0"
        ],
        "installpkg": {
            "basename": "tox-envreport-0.1.0.zip",
            "md5": "5046419f076f6fd825319874614a90d0",
            "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
        },
        "name": "py35",
        "path": "/home/daniel/workspace/tox-envreport/.tox/py35",
        "platform": "linux",
        "python": {
            "executable": "/home/daniel/workspace/tox-envreport/.tox/py35/bin/python",
            "version": "3.5.2 (default, Nov 23 2017, 16:37:01) \n[GCC 5.4.0 20160609]",
            "version_info": [
                3,
                5,
                2,
                "final",
                0
            ]
        },
        "reportversion": "1",
        "setup": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip",
                    "install",
                    "-U",
                    "--no-deps",
                    "/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip"
                ],
                "output": "actionid: py35\nmsg: installpkg\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip', 'install', '-U', '--no-deps', '/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip']\n\nProcessing ./.tox/dist/tox-envreport-0.1.0.zip\nBuilding wheels for collected packages: tox-envreport\n  Running setup.py bdist_wheel for tox-envreport: started\n  Running setup.py bdist_wheel for tox-envreport: finished with status 'done'\n  Stored in directory: /home/daniel/.cache/pip/wheels/da/40/e3/1954e2a10b42ec16b0a8ecb694e438fc3894a7a3c233810269\nSuccessfully built tox-envreport\nInstalling collected packages: tox-envreport\n  Found existing installation: tox-envreport 0.1.0\n    Uninstalling tox-envreport-0.1.0:\n      Successfully uninstalled tox-envreport-0.1.0\nSuccessfully installed tox-envreport-0.1.0\n",
                "retcode": "0"
            },
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip",
                    "freeze"
                ],
                "output": "actionid: py35\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py35/bin/pip', 'freeze']\n\nattrs==17.4.0\nmore-itertools==4.1.0\npluggy==0.6.0\npy==1.5.3\npytest==3.5.0\nsix==1.11.0\ntox==3.0.0\ntox-envreport==0.1.0\nvirtualenv==15.2.0\n",
                "retcode": "0"
            }
        ],
        "test": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py35/bin/pytest",
                    "tests"
                ],
                "output": null,
                "retcode": "0"
            }
        ],
        "toxversion": "3.0.0"
    },
    "py36": {
        "host": "daniel-dev",
        "installed_packages": [
            "attrs==17.4.0",
            "more-itertools==4.1.0",
            "pluggy==0.6.0",
            "py==1.5.3",
            "pytest==3.5.0",
            "six==1.11.0",
            "tox==3.0.0",
            "tox-envreport==0.1.0",
            "virtualenv==15.2.0"
        ],
        "installpkg": {
            "basename": "tox-envreport-0.1.0.zip",
            "md5": "5046419f076f6fd825319874614a90d0",
            "sha256": "a410ac82328f6089e7e6ffc826a81a1526c91395a793bee265430c0a9cd7d28e"
        },
        "name": "py36",
        "path": "/home/daniel/workspace/tox-envreport/.tox/py36",
        "platform": "linux",
        "python": {
            "executable": "/home/daniel/workspace/tox-envreport/.tox/py36/bin/python",
            "version": "3.6.2 (default, Jul 17 2017, 23:14:31) \n[GCC 5.4.0 20160609]",
            "version_info": [
                3,
                6,
                2,
                "final",
                0
            ]
        },
        "reportversion": "1",
        "setup": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py36/bin/pip",
                    "install",
                    "-U",
                    "--no-deps",
                    "/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip"
                ],
                "output": "actionid: py36\nmsg: installpkg\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py36/bin/pip', 'install', '-U', '--no-deps', '/home/daniel/workspace/tox-envreport/.tox/dist/tox-envreport-0.1.0.zip']\n\nProcessing ./.tox/dist/tox-envreport-0.1.0.zip\nBuilding wheels for collected packages: tox-envreport\n  Running setup.py bdist_wheel for tox-envreport: started\n  Running setup.py bdist_wheel for tox-envreport: finished with status 'done'\n  Stored in directory: /home/daniel/.cache/pip/wheels/da/40/e3/1954e2a10b42ec16b0a8ecb694e438fc3894a7a3c233810269\nSuccessfully built tox-envreport\nInstalling collected packages: tox-envreport\n  Found existing installation: tox-envreport 0.1.0\n    Uninstalling tox-envreport-0.1.0:\n      Successfully uninstalled tox-envreport-0.1.0\nSuccessfully installed tox-envreport-0.1.0\n",
                "retcode": "0"
            },
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py36/bin/pip",
                    "freeze"
                ],
                "output": "actionid: py36\nmsg: envreport\ncmdargs: ['/home/daniel/workspace/tox-envreport/.tox/py36/bin/pip', 'freeze']\n\nattrs==17.4.0\nmore-itertools==4.1.0\npluggy==0.6.0\npy==1.5.3\npytest==3.5.0\nsix==1.11.0\ntox==3.0.0\ntox-envreport==0.1.0\nvirtualenv==15.2.0\n",
                "retcode": "0"
            }
        ],
        "test": [
            {
                "command": [
                    "/home/daniel/workspace/tox-envreport/.tox/py36/bin/pytest",
                    "tests"
                ],
                "output": null,
                "retcode": "0"
            }
        ],
        "toxversion": "3.0.0"
    }
}

Contribute

Please check our github project at https://github.com/useblocks/tox-envreport.

Changelog

0.2.1

• Bugfix: Support for tox >= 3.8.0 (#3)

0.2.0

• Improvement: Switch file names to tox-reports.json and env-report.json
• Improvement: Added more examples, a changelog and a logo to documentation

0.1.1

• Added Readme.rst for PyPi

0.1.0

• Initial version

Acknowledgments

Thanks to Lazur URH , who designed the tox-envreport logo.