tomahawk 0.7.4.1 documentation

What is tomahawk?

tomahawk enables following 3 things.

  • Executes a command into many remote hosts
  • Copy local files to many remote hosts
  • Copy files from remote hosts to local

Executes a command into many remote hosts

tomahawk is a program that enables to execute a command into many hosts.

$ tomahawk -h host1,host2,host3 uptime

—> ‘uptime’ command will be executed in host1, host2, and host3 with following output.

oinume@host1 % uptime
22:41:27 up 10 days,  3:26,  1 users,  load average: 1.11, 1.13, 1.11

oinume@host2 % uptime
22:41:28 up 20 days,  4:26,  2 users,  load average: 2.11, 2.13, 2.11

oinume@host3 % uptime
22:41:29 up 30 days,  5:26,  3 users,  load average: 3.11, 3.13, 3.11

Copy files to many remote hosts or copy files from remote hosts to local

tomahawk-rsync is a program that enables to copy files <into/from> many hosts.

$ tomahawk-rsync -h host1,host2,host3 test.py /tmp/test.py

—> ‘test.py’ is copied to host1, host2 and host3.

$ tomahawk-rsync -f web.list /usr/local/apache2/conf/httpd.conf /tmp/httpd.conf

—> ‘httpd.conf’ is copied to hosts which listed in ‘web.list’.

$ tomahawk-rsync -h host1,host2 -m pull /usr/local/apache2/conf/httpd.conf /tmp/conf/

—> ‘httpd.conf’ is copied from host1 and host2 to local directory /tmp/conf as ‘host1__httpd.conf’ and ‘host2__httpd.conf’.

Contents:

How to install tomahawk

Requirements

  • python >= 2.4
  • argparse (required only with python < 2.7)
  • multiprocessing (required only with python < 2.6)
  • pexpect
  • pytest (required for testing)
  • flexmock (required for testing)

Installation

tomahawk distributions is in the pypi, so the easiest way is using pip or easy_install

$ sudo pip install tomahawk

or

$ sudo easy_install tomahawk

Or you can use traditional way

$ tar xvzf tomahawk-x.y.z.tar.gz
$ cd tomahawk-x.y.z
$ sudo python setup.py install

tomahawk manual page

SYNOPSIS

tomahawk [options] command

DESCRIPTION

tomahawk is a program that enables to execute a command into many hosts.

$ tomahawk -h host1,host2,host3 uptime

—> ‘uptime’ command will be executed in host1, host2, and host3 with following output.

oinume@host1 % uptime
22:41:27 up 10 days,  3:26,  1 users,  load average: 1.11, 1.13, 1.11

oinume@host2 % uptime
22:41:28 up 20 days,  4:26,  2 users,  load average: 2.11, 2.13, 2.11

oinume@host3 % uptime
22:41:29 up 30 days,  5:26,  3 users,  load average: 3.11, 3.13, 3.11
ssh

tomahawk executes a command via ‘ssh’. You can specify options for ssh with -o/–ssh-options and can configure ssh behavior with $HOME/.ssh/config.

hosts file

-h option enables you to specify hosts, another option ‘-f’, which is specifying hosts files. hosts file is listing host names like this

host1
host2
host3
#host4

Starting with “#” means commenting the host out.

shell operators

tomahawk executes commands via shell(/bin/sh), so you can use ‘|’ (pipe), ‘&&’, ‘||’ operators and so on.

$ tomahawk -h host1,host2 'ps auxww | grep python'

OPTIONS

These programs follow the usual GNU command line syntax, with long options starting with two dashes (‘–’). A summary of options is included below. For a complete description, see the Info files.

-H, –hosts

Specifies host names for sending commands. You can specify multiple hosts with ‘,’.

-f, –hosts-files

Specifies hosts files which listed host names for sending commands. You can specify multiple hosts files with ‘,’.

Format of hosts file is below.

web01
web02
#web03
web04

A line of starting with ‘#’ disables a host.

-l, –prompt-login-password

Prompts a password for ssh authentication at first. If the password is all the same between target hosts, you’ll input a password just once.

–login-password-stdin

Read a SSH password from stdin instead of prompting.

-s, –prompt-sudo-password

Prompts a password for sudo.

-c, –continue-on-error

Continues to send commands even if any errors. The default behavior is fail-safe, means that tomahawk will stop if any errors.

-p, –parallel

Specifies a number of processes for parallel command execution. (default: 1) If your machine has multiple cpu cores, –parallel 2 .. N might be faster.

-t, –timeout

Specifies timeout seconds for a command.

-u, –ssh-user

Specifies ssh user. The default is a current logged in user.

-o, –ssh-options

Specifies ssh options.

-F, –output-format

Specifies command output format. The default is '${user}@${host} % ${command}\n${output}\n'

-V, –verify-output

Verify command output of all hosts. For additional information, see Checking files on remote hosts

-C, –conf

Specifies configuration file path. For additional information, see Omit command line options by a configuration file.

ENVIRONMENT VARIABLES

tomahawk’s behavior is affected by the following environment variables.

TOMAHAWK_ENV

This variable specifies an environment of ‘production’, ‘testing’ or ‘development’. If TOMAHAWK_ENV=production specified, tomahawk or tomahawk-rsync prompt as “command “%s” will be executed %s hosts. Are you sure? [yes/NO]: ”. The environment variable exists for mis-execution of a command.

SEE ALSO

  • tomahawk-rsync(1)
  • ssh(1)

tomahawk-rsync manual page

SYNOPSIS

tomahawk-rsync [options] source destination

DESCRIPTION

tomahawk-rsync is a program that enables to copy files <into/from> many hosts.

$ tomahawk-rsync -h host1,host2,host3 test.py /tmp/test.py

—> ‘test.py’ is copied to host1, host2 and host3.

$ tomahawk-rsync -f web.list /usr/local/apache2/conf/httpd.conf /tmp/httpd.conf

—> ‘httpd.conf’ is copied to hosts which listed in ‘web.list’.

$ tomahawk-rsync -h host1,host2 -m pull /usr/local/apache2/conf/httpd.conf /tmp/conf/

—> ‘httpd.conf’ is copied from host1 and host2 to local directory /tmp/conf as ‘host1__httpd.conf’ and ‘host2__httpd.conf’.

rsync

tomahawk-rsync copies files via ‘rsync’. You can specify options for rsync with -o/–rsync-ooptions.

OPTIONS

These programs follow the usual GNU command line syntax, with long options starting with two dashes (‘–’). A summary of options is included below. For a complete description, see the Info files.

-H, –hosts

Specifies host names for sending commands. You can specify multiple hosts with ‘,’.

-f, –hosts-files

Specifies hosts files which listed host names for sending commands. You can specify multiple hosts files with ‘,’.

Format of hosts file is below.

web01
web02
#web03
web04

A line of starting with ‘#’ disables a host.

-l, –prompt-login-password

Prompts a password for ssh authentication of rsync at first. If the password is all the same between target hosts, you’ll input a password just once.

-c, –continue-on-error

Continues to send commands even if any errors. The default behavior is fail-safe, means that tomahawk will stop if any errors.

-p, –parallel

Specifies a number of processes for parallel command execution. (default: 1) If your machine has many cpu cores, –parallel 2 .. N might be faster.

-t, –timeout

Specifies timeout seconds for a command.

-F, –output-format

Specifies command output format. The default is '${user}@${host} % ${command}n${output}n’

-u, –rsync-user

Specifies rsync user. The default is a current logged in user.

-o, –rsync-options

Specifies rsync options. The default is ‘-av’

-m, –mirror-mode

Selection of ‘push’ or ‘pull’. ‘pull’ means copy files from remote to local. The default is ‘push’.

-C, –conf

Specifies configuration file path. For additional information, see Omit command line options by a configuration file.

ENVIRONMENT VARIABLES

tomahawk’s behavior is affected by the following environment variables.

TOMAHAWK_ENV

This variable specifies an environment of ‘production’, ‘testing’ or ‘development’. If TOMAHAWK_ENV=production specified, tomahawk or tomahawk-rsync prompt as “command “%s” will be executed %s hosts. Are you sure? [yes/NO]: ”. The environment variable exists for mis-execution of a command.

SEE ALSO

  • tomahawk(1)
  • ssh(1)
  • rsync(1)

tomahawk Recipes

Describes how to use tomahawk.

Formatting output

-F/--output-format option can change tomahawk’s output.

$ tomahawk -h <hosts> -F '[${host}] ${output}' -h <hosts> 'date'

[localhost] Sat Jun  2 02:21:39 JST 2012
[127.0.0.1] Sat Jun  2 02:21:40 JST 2012

You can speficy following variables.

  • ${user}
  • ${host}
  • ${command}
  • ${output}

Checking files on remote hosts

Since v0.6.0, able to check files on remote hosts are all the same. Let’s check httpd.conf is all the same with md5sum command.

$ tomahawk -h <hosts> -V 'md5sum /usr/local/apache2/conf/httpd.conf'

If output of some hosts are different, you’ll get following errors.

...
[error] Detected different command output on following hosts.
...

Omit command line options by a configuration file.

Since v0.6.0, you can omit command line options by a configuration file. If -c/--conf option is specified, tomahawk and tomahawk-rsync read command line options from a configuration file. It is good to define commonly-used options in a configuration file.

Configuration file is just ini file like below.

[tomahawk]
options = --parallel 1

[tomahawk-rsync]
options = --parallel 1

That is equivalent to

$ tomahawk --parallel 1
$ tomahawk-rsync --parallel 1

Changes

0.7.4.1 <2016-08-21>

  • bug: #57 UnicodeEncodeError (Thanks @sekino for reporting)

0.7.3 <2016-04-24>

  • #54 Add Python 3.5 support
  • #55 Fix line break problem when expect (Thanks @sekino)

0.7.2 <2014-07-27>

  • bug: #50 pexpect-3.3 installation problem (ValueError: I/O operation on closed file)
  • #49 Add Python 3.4 support

0.7.1 <2014-04-17>

  • bug: #46 Cannot install latest `six’ on Python 2.4 (Thanks @n0ts)
  • #47 Test various environments with Vagrant
  • #48 Wrong command line options in documentation

0.7.0 <2014-03-02>

  • bug: #41 Ignores User configuration on $HOME/.ssh/config

0.7.0-rc1 <2013-11-16>

  • feature: #42 Python3 support
  • change: #43 An alias of –hosts should be -H (not -h)
  • change: #44 Delete duplicated options –expect-timeout, –no-sudo-password

0.6.0 <2013-04-21>

  • document: Add “For developers” section in README.rst

0.6.0-rc1 <2013-04-06>

  • feature: #36 –verify-output option
  • feature: #37 A short-cut ‘-F’ for –output-format option
  • feature: #39 -C/–conf: Specify a configuration file
  • document: #38 tomahawk recipes in documentation

0.5.2 <2012-06-01>

  • bug: #35 A duplicated command output.

0.5.1 <2012-05-26>

  • bug: #33 tomahawk should escape ‘$’ and ‘`’
  • bug: #34 parallel numer should be smaller than specified value when count of hosts is smal
  • document: updateUpdated documentation.
  • test: Updated testing codes (now uses pytest and flexmock)

0.5.0 <2012-04-01>

  • Nothing changed from 0.5.0-rc3

0.5.0-rc3 <2012-03-24>

  • More bug fixes
  • change: Removed option in 0.5.0-rc1 ‘–prompt-login-password’ were back.
  • feature: #16 New options –login-password-stdin, –sudo-password-stdin is now available
  • change: #32 tomahawk now doesn’t prompt sudo password even if ‘sudo ...’ command given. If -s/–prompt-sudo-password is given, tomahawk prompts sudo password.

0.5.0-rc2 <2012-03-13>

  • bug : tomahawk doesn’t stop completely when SIGINT(Ctrl-C) received

0.5.0-rc1< 2012-03-04>

  • feature: #22 Output OS version, Python version, libraries versions
  • bug: #27 Should call Pool#join() after all processes are finished
  • bug: #26 tomahawk-rsync should say an easier error message when rsync command is not found.
  • change: #16 Now tomahawk sends command with ‘ssh -t’, allocates pseudo-tty. This changes default behavior of `tomahawk`. Sended commands on remote host will be cancelled after tomahawk stops. (Ctrl-C or something) Old behavior is that remote commands never stop even if tomahawk stops. If you want old behavior, use –ssh-options=’-T’. It disables pseudo-tty allocation.

0.4.5 <2012-01-15>

  • bug: Fixed installation error for 0.4.4

0.4.4 <2012-01-14>

  • document: New documentation with sphinx.
  • bin/tomahawk_bootstrap.py was moved to tomahawk directory. (Thanks @mkouhei <https://github.com/mkouhei>)
  • Created tools directory for developers.

0.4.3 <2011-12-03>

  • bug: #21 Support sudo prompt in Ubuntu. (Thanks t9md)
  • Added files. AUTHORS, COPYING

0.4.2 <2011-11-27>

  • Updated a license(LGPL -> LGPL 2.1)
  • Added man page (Thanks @mkouhei)

0.4.1 <2011-09-18>

  • bug: #15 setup.py is not ready for python 2.4

0.4.0 <2011-07-14>

  • feature: #13 Coloring output
  • feature: #14 Support python 2.4
  • bug: #12 Unknown distribution option: ‘test_require’
  • Refactoring whole source and adding more tests

0.3.4 <2011-07-05>

  • feature: #8 –output-format option for tomahawk

0.3.3 <2011-07-04>

  • bug: #6 Timeout problem when both ssh authentication and sudo password required
  • bug: #10 tomahawk times out when japanese sudo password prompt
  • change: #11 –expect-encoding is now obsoleted
  • change: #9 release.py problem

0.3.2 <2011-06-18>

  • bug: #6 Timeout problem when both ssh authentication and sudo password required
  • bug: #7 Password is not masked even –debug

0.3.1 <2011-06-16>

  • bug: #4 Pass a directory to -f option, traceback occurrs
  • bug: #6 Timeout problem when both ssh authentication and sudo password required

0.3.0 <2011-05-15>

  • Improve handling keyboard interrupt.
  • Fix tests.

0.3.0-rc1 <2011-04-17>

  • bug: When execution timed out, looks like just failure.
  • bug: All host names are not displayed when tomahawk-rsync with –continue-on-error fails.
  • bug fix: Displays error hosts with specified order when –continue-on-error option specified.
  • change: –expect-timeout becomes duplicated. Use –timeout instead.
  • change: Default –timeout seconds is changed from 5 to 10.
  • change: Changed filename format from ‘%(filename)__%(host)’ to ‘%(host)__%(filename)’ when tomahawk-rsync –mirror-mode=pull.
  • Output further debug messages when –debug.

0.2.6 <2011-04-13>

  • feature: –no-sudo-password is now available.

0.2.5 <2011-02-24>

  • bug: When no sudo password, no command output.

0.2.4 <2011-02-08>

0.2.3 <2011-02-07>

0.2.2 <2011-01-25>

  • bug: when tomahawk-rsync -m pull specified, always appends hostname to local filename.

0.2.1 <2011-01-24>

  • bug: tomahawk-rsync always outputs same hostname.

0.2.0 <2010-12-07>

  • feature: When environment “TOMAHAWK_ENV” is “production”, tomahawk confirms command execution.
  • feature: –version option is now available.

0.1.2 <2010-12-06>

  • bug: tomahawk prints a password to stdout.

0.1.1 <2010-12-02>

  • change: Renamed. tomahawk.py -> tomahawk, tomahawk-rsync.py -> tomahawk-rsync
  • Fixed many bugs

0.1.0 <2010-11-26>

  • The first release of python version.
  • feature: Brand new option: -p (–parallel)
  • change: -C option is now obsoleted. Use -c

Indices and tables