Steth

A network inspection tool for OpenStack.

Description

Steth is an inspection tool that can aid in pinpointing issues before deployment and during operation of an OpenStack environment.

It is modelled as agent(s)/client in which a controller interacts with agents deployed in your environment.

Background

OpenStack networking can be deloyed as different architectures, such as ML2 with OVS(legacy and DVR), Linux bridge, OVN, Dragonflow and so forth. However, they all need enviromental prerequisites. For instance, VLAN needs to be configured as we expect; bandwidth should meet our requirements; connection between nodes should be active, etc.

Besides, with some well-deployed architectures, troubleshooting for VM networking is difficult. For instance, why VM cannot get an IP address; or why it cannot connect to Internet, etc. Steth integrates useful scripts and third party tools(like iperf, tcpdump, etc.) to help operators keep tracking on VM networking.

Mission

Steth is an introspection tool for OpenStack networking. Only proved to be working in ML2 with OVS for now.

Table of Contents

Quickstart

Steth is an inspection tool that can aid in pinpointing issues before deployment and during operation of an OpenStack environment.

It is modelled as agent(s)/client in which a controller interacts with agents deployed in your environment. Let me introduce how to use steth.

Download code

Download the latest code from git repository. And run python setup.py install to install steth. After running that, you can steth --help to confirm Steth is installed correctly.

Deploy Steth Agent

Steth Agent listens in 0.0.0.0:9698 on any node you want. It will wait for RPC request. Currently we support CentOS 6.5, CentOS 7.0 and CentOS 7.1 only. In CentOS 6.5, you should run service steth-agent start to start steth-agent. In CentOS 7.0 and 7.1, you should run systemctl start steth to start steth-agent.

Deploy Steth Client

Steth Client is a stateless program. You can run steth --help to show all steth commands that you can run.

Configuration File

On start the client will read a configuration file. By default the configuration file is located at /etc/steth/steth.conf.

Here is an example of the configuration file

# (ListOpt) list of networks types.
# We may have multi network types in one node, such as mgmt, net and stroage.
# so this value should be a list.
# We seperate each item by ":". Treat first item as network type.
# The second is physical nic name. And the third is network_prefix.
# Example: "mgmt:eth0:1.1.1.,net:eth1:2.2.2.,storage:eth2:3.3.3."
network_types=mgmt:eth0:1.1.1.,net:eth1:2.2.2.,storage:eth2:3.3.3.

# (ListOpt) All nodes info. Just need sequence number.
# Example: 64, 65, 66
nodes_id=39,233,64,65,66

# (StrOpt) Name prefix of every node. By default, this value
# is "server". We combine "node_name_prefix" with
# "nodes_id", to define nodes. Such as "server-64", "server-68"
# and so on. In every region, we give every node a specific name.
# Ensure that DNS can resolve the nodes.
node_name_prefix=server-

Multi-node Architecture

                                                                   note that steth does not save
                                                                   any state, it acts as a rpc
                                                                   client which makes requests to steth
                                    +--------------------------+   agent(s) and analyses the result.
                                    |                          |
                                    |   +------CLI---------+   |
                                    |   |                  |   |
             +--------------------------+     steth        +--------------------------+
             |                      |   |                  |   |                      |
             |                      |   +--------+---------+   |                      |
             |                      |            |             |                      |
             |                      +--------------------------+                      |
             |                                   |                                    |
             v                                   v                                    v
+-------+port:9698---------+        +-------+port:9698---------+         +-------+port:9698---------+
|            ^             |        |            ^             |         |            ^             |
|            |             |        |            |             |         |            |             |
| +----------+-------+     |        | +----------+-------+     |         | +----------+-------+     |
| |                  |     |        | |                  |     |         | |                  |     |
| |   steth-agent    |     |        | |   steth-agent    |     |         | |   steth-agent    |     |
| |                  |     |        | |                  |     |         | |                  |     |
| +-----------+------+     |        | +-----------+------+     |         | +-----------+------+     |
|             |            |        |             |            |         |             |            |
|             |            |        |             |            |         |             |            |
|  +----------v----------+ |        |  +----------v----------+ |         |  +----------v----------+ |
|  | run command like:   | |        |  | run command like:   | |         |  | run command like:   | |
|  | ping, iperf, tcpdump| |        |  | ping, iperf, tcpdump| |         |  | ping, iperf, tcpdump| |
|  | or use scapy to send| |        |  | or use scapy to send| |         |  | or use scapy to send| |
|  | packet              | |        |  | packet              | |         |  | packet              | |
|  +---------------------+ |        |  +---------------------+ |         |  +---------------------+ |
|                          |        |                          |         |                          |
|                          |        |                          |         |                          |
+--------------------------+        +--------------------------+         +--------------------------+

In a scenario using multiple nodes, Steth is a stateless CLI and controller. It knows each steth agent and will read config files, interact with OpenStack, and sending instructions to agents when needed.

Steth Agent is introduced to manage processes or run commands. It should be installed in each compute and network node, and their IPs should be specified in the config file of steth controller.

Steth Agent

Listening on 0.0.0.0:9698 and waiting for the rpc request.

Note: for get_interface() agent API, we use ifconfig to get full information. However, the output of ifconfig varies from a Linux distribution to another. The API has only been tested on CentOS 6.5 and 7.0. Any other distribution has not been tested. If it works, please let us know.