Suricata用户手册¶
什么是Suricata¶
Suricata是一个高性能的网络IDS/IPS和网络安全监控引擎。它是开源的，归名叫开放信息安全基金会（OISF）的社区运营的非营利基金会所有。Suricata由OISF开发。
关于开放信息安全基金会¶
开放信息安全基金会是一个非营利性基金会，旨在建立社区并支持开源安全技术，如世界一流的IDS/IPS引擎Suricata。
许可证¶
Suricata源码以第二版本 GNU General Public License 授权许可，这份文档以 Creative Commons Attribution-NonCommercial 4.0 International Public License 授权许可。
安装¶
在使用Suricata之前，我们必须安装它。. Suricata可以通过二进制包: 二进制包 在各种不同的发行版上安装。
对于熟悉自己编译软件的人来说，推荐使用自己编译源代码方式安装的。
高级用户可以查看高级指南, 参见 高级安装.
源码方式¶
通过源码方式安装可以最大程序地控制Suricata安装过程.
基本步骤
tar xzvf suricata-4.0.0.tar.gz
cd suricata-4.0.0
./configure
make
make install
这将Suricata安装到 /usr/local/bin/ 目录, 默认配置文件位于 /usr/local/etc/suricata/ 目录，日志输出到
/usr/local/var/log/suricata 目录
常规configure选项¶
-
--disable-gccmarch-native¶ 不根据硬件来优化生成的二进制代码，加上这个选项意味着生成的Suricata程序是可移植的或者在VM中使用.
-
--prefix=/usr/¶ Suricata二进制文件将安装到/usr/bin/目录. 默认是安装到
/usr/local/目录
-
--sysconfdir=/etc¶ Suricata配置文件目录将设定为 /etc/suricata/ ，默认是
/usr/local/etc/目录
-
--localstatedir=/var¶ Suricata日志输出目录将设定为 /var/log/suricata/. 默认是
/usr/local/var/log/suricata目录
-
--enable-lua¶ 为检测和输出模块启用Lua支持.
-
--enable-geopip¶ 为检测模块启用GeoIP支持.
-
--enable-rust¶ 启用试验性Rust支持
依赖¶
编译Suricata你需要安装以下开发库和头文件:
libpcap, libpcre, libmagic, zlib, libyaml
下面这几个工具也是必须的:
make gcc (or clang) pkg-config
若要编译的程序具有所有功能特性, 需要添加这些:
libjansson, libnss, libgeoip, liblua5.1, libhiredis, libevent
Rust支持 (试验性):
rustc, cargo
Ubuntu/Debian¶
最小依赖:
apt-get install libpcre3 libpcre3-dbg libpcre3-dev build-essential libpcap-dev \
libyaml-0-2 libyaml-dev pkg-config zlib1g zlib1g-dev \
make libmagic-dev
建议:
apt-get install libpcre3 libpcre3-dbg libpcre3-dev build-essential libpcap-dev \
libnet1-dev libyaml-0-2 libyaml-dev pkg-config zlib1g zlib1g-dev \
libcap-ng-dev libcap-ng0 make libmagic-dev libjansson-dev \
libnss3-dev libgeoip-dev liblua5.1-dev libhiredis-dev libevent-dev
对于iptables/nftables IPS集成，额外添加:
apt-get install libnetfilter-queue-dev libnetfilter-queue1 \
libnetfilter-log-dev libnetfilter-log1 \
libnfnetlink-dev libnfnetlink0
Rust支持 (仅Ubuntu):
apt-get install rustc cargo
二进制包¶
Ubuntu¶
针对Ubuntu, OISF维护了一个问题包含最新的稳定发行版本源，地址是 suricata-stable .
使用这个源:
sudo add-apt-repository ppa:oisf/suricata-stable
sudo apt-get update
sudo apt-get install suricata
Debian¶
在Debian 9 (Stretch)下，运行:
apt-get install suricata
在Debian Jessie中，Suricata已经过期, 但在Debian Backports中，有一个更新的版本.
以root身份运行:
echo "deb http://http.debian.net/debian jessie-backports main" > \
/etc/apt/sources.list.d/backports.list
apt-get update
apt-get install suricata -t jessie-backports
Fedora¶
dnf install suricata
RHEL/CentOS¶
对于RedHat Enterprise Linux 7 和 CentOS 7 ，可以使用 EPEL 源.
yum install epel-release
yum install suricata
高级安装¶
从GIT和其他操作系统安装的各种安装指南都保存在: https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Suricata_Installation
命令行选项¶
Suricata命令行选项:
-
-h¶ 显示使用概要
-
-V¶ 显示Suricata版本
-
-c<path>¶ 指定配置文件路径
-
-T¶ 测试配置
-
-v¶ -v 选项让Suricata输出详细信息。 多次使用这个选项可以得到更多详细信息。
-
-r<path>¶ 以pcap离线模式运行从pcap文件中读取数据。 如果<path>指定一个目录，则将按照修改时间的顺序处理该目录中的所有文件，以保持流状态。
-
--pcap-file-continuous¶ 与-r 选项一起使用以指示运行模式应该保持活动直到被中断。 在以目录方式添加处理文件时，加上这个选项不会重置流状态。
-
-i<interface>¶ -i 选项指定用于嗅探数据包的网卡，此选项将尝试可用的最佳数据包捕获方法。
-
--pcap[=<device>]¶ 以PCAP模式运行。 如果未指定设备，则将使用配置文件中*pcap*部分提供的接口。
-
--af-packet[=<device>]¶ 指定Linux上使用AF_PACKET进行数据包捕获。 如果没有指定设备，则使用yaml配置文件中af-packet部分的设备列表。
-
-q<queue id>¶ 根据NFQUEUE队列ID运行在inline nfqueue模式，这个ID可能会多次提供
-
-s<filename.rules>¶ -s 选项指定一个规则文件，这个文件可以与yaml配置文件中指定的规则文件一起加载使用。
-
-S<filename.rules>¶ -S 选项指定一个规则文件，这个文件会单独加载使用，而不加载yaml配置文件中指定的规则文件。
-
-l<directory>¶ -l 选项指定默认日志目录。 即使您已经在yaml配置文件中设置了default-log-dir，Suricata也会忽略这个设置。如果您没有使用-l选项，Suricata将使用yaml中设置的目录。
-
-D¶ 通常当你使用console运行suricata时，这个console会被suricata占用而不能用来处理其他工作，console关闭时suricata也会被关闭。使用-D选项，suricata会在后台运行，这时你可以使用console干处理其他工作而不会影响到suricata运行。
-
--runmode<runmode>¶ –runmode 选项指定你想要使用的运行模式，此命令行选项可以覆盖yaml中的运行模式设置。
运行模式包括: workers, autofp and single.
更多关于运行模式的资料请参考手册中的 Runmodes .
-
-F<bpf filter file>¶ 指定BPF过滤器文件.
-
-k[all|none]¶ 强制（all）或禁用（none）校验和检查。
-
--pidfile<file>¶ 将进程ID写到指定文件，这个选项会覆盖配置文件中*pid-file*部分的设定值，并且当进程不是以后台进程运行时，强制写文件。
-
--init-errors-fatal¶ 设置若加载规则文件时出错，就失败退出。
-
--disable-detection¶ 禁用检测引擎
-
--dump-config¶ 将配置文件中加载的配置转储到终端并退出。
-
--build-info¶ 显示Suricata编译时的编译选项信息。
-
--list-app-layer-protos¶ 列出支持的所有应用层协议。
-
--list-keywords=[all|csv|<kword>]¶ 列出支持的所有规则关键字。
-
--list-runmodes¶ 列出支持的所有运行模式。
-
--set<key>=<value>¶ 设置一个配置值来覆盖基本配置参数。例如，更改默认日志目录:
--set default-log-dir=/var/tmp
-
--engine-analysis¶ 打印关于引擎中不同部分的分析报告，要知道可以打印什么报告请参考配置文件engine-analysis部分。
-
--pcap-buffer-size=<size>¶ 指定PCAP包捕获缓冲区大小 (0 - 2147483647).
-
--netmap[=<device>]¶ 在FreeBSD或Linux上启用NETMAP数据包捕获，如果未指定设备，则使用yaml配置文件中netmap部分设置的设备列表。
-
--pfring[=<device>]¶ 启用PF_RING数据包捕获，如果未指定设备，将使用Suricata配置文件中指定的设备。
-
--pfring-cluster-id<id>¶ 设置PF_RING集群ID.
-
--pfring-cluster-type<type>¶ 设置PF_RING集群类型 (cluster_round_robin, cluster_flow).
-
-d<divert-port>¶ 使用IPFW转发模式运行
-
--dag<device>¶ 启用DAG卡数据包捕获。可使用诸如“dag0：4”之类的设备名称来选择捕获特定流，这个选项可以多次使用以指定多个设备和/或流。
-
--napatech¶ 使用Napatech Streams API进行数据包捕获。
-
--mpipe¶ 使用TileGX mpipe接口进行数据包捕获.
-
--erf-in=<file>¶ 从指定的ERF(Endace可扩展记录格式)文件中读取数据以离线模式运行
-
--simulate-ips¶ 以非IPS模式运行时伪装成IPS模式
单元测试¶
内置单元测试仅在Suricata以–enable-unittests选项编译生成时可用。
运行unittests不会使用配置文件，使用-l选项指定输出目录。
-
-u¶ 运行单元测试并退出，要求Suricata编译时添加 –enable-unittests 选项.
-
-U,--unittest-filter=REGEX¶ -U 选项指定将要运行的单元测试，这个选项可以使用正则表达式，例如: suricata -u -U http
-
--list-unittests¶ 列出所有单元测试
-
--fatal-unittests¶ 单元测试出错时启动致命失败，Suricata将会退出而不是继续进行测试。
-
--unittests-coverage¶ 显示单元测试覆盖报告。
Suricata规则¶
规则格式¶
规则在Suricata中起着非常重要的作用，大多数情况下人们使用现有的规则集.
官方途径来安装规则集请参考 Rule Management with Suricata-Update.
本Suricata规则文件解释了所有关于规则的内容; 如何阅读，调整和创建它们。
规则/签名包括以下内容：
- action, 确定规则匹配时执行的动作
- header, 定义协议，IP地址，端口和方向.
- rule options, 定义规则细节.
下面是一个规则的例子:
在这个例子里, 红色 是动作, 绿色 规则头部， 是规则选项。
我们将在本节中使用上述规则作为示例，高亮显示规则的不同部分。它是来自Emerging Threats数据库的签名，这是一个开放式数据库，其中包含许多规则，您可以在Suricata实例中免费下载和使用这些规则。
动作¶
更多信息请参见 Action-order.
协议¶
这个规则关键字告诉Suricata什么协议是它关注的，你可以选择以下四种基础协议:
- tcp (for tcp-traffic)
- udp
- icmp
- ip (ip 意味着 ‘all’ or ‘any’)
还有一些所谓的应用层协议或者第七层协议可以选择，它们包括:
- http
- ftp
- tls (this includes ssl)
- smb
- dns
- dcerpc
- ssh
- smtp
- imap
- msn
- modbus (默认禁用)
- dnp3 (默认禁用)
- enip (默认禁用)
- nfs (取决于rust是否可用)
- ikev2 (取决于rust是否可用)
- krb5 (取决于rust是否可用)
- ntp (取决于rust是否可用)
- dhcp (取决于rust是否可用)
这些协议是否可用取决于协议是否在suricata.yaml配置文件中启用。
如果你有一个HTTP协议的规则，Suricata确认这条规则只会匹配HTTP相关的流量
源和目的¶
第一个高亮部分就是源,第二个是目的(注意箭头的方向).
源和目的分别表示流量的来源和流量的目的，可以使用IP地址（支持IPv4和IPv6）和IP范围来定义源和目的，并且可以用以下操作符来进行组合使用:
| 操作符 | 描述 |
|---|---|
| ../.. | IP范围 (CIDR表示法) |
| ! | 排除/取反 |
| [.., ..] | 组合 |
通常，我们可以使用变量, 如 $HOME_NET 和
$EXTERNAL_NET. 配置文件中指定了这些变量的IP地址，用于代替规则中的变量。
更多信息请参考 Rule-vars
例如:
| 例子 | 含义 |
|---|---|
| ! 1.1.1.1 | 除了1.1.1.1的任意IP |
| ![1.1.1.1, 1.1.1.2] | 除了1.1.1.1 和 1.1.1.2 的任意IP |
| $HOME_NET | 配置文件中设置的HOME_NET变量 |
| [$EXTERNAL_NET, !$HOME_NET] | 包含EXTERNAL_NET但不包含HOME_NET的地址 |
| [10.0.0.0/24, !10.0.0.5] | 10.0.0.0/24但不包括10.0.0.5 |
| […, [….]] | |
| […, ![…..]] |
警告
如果你在配置文件中做了如下设置:
HOME_NET: any
EXTERNAL_NET: ! $HOME_NET
那么你不能在规则中使用 $EXTERNAL_NET ，这是因为它代表
‘not any’. 这是一个无效设置。
端口 (源和目的)¶
第一个高亮部分是源端口，第二是是目的端口 (注意箭头方向).
流量通过端口进入和流出，不同的端口具有不同的端口号。例如，HTTP的默认端口是80，而443通常是HTTPS的端口。但请注意，端口不代表通信中使用的协议。相反，它确定哪个应用程序正在接收数据。
上面提到的端口通常是目标端口。 源端口，即发送数据包的应用程序使用的端口，通常由操作系统随机分配。 在为自己的HTTP服务编写规则时，通常会编写``any - > 80``，因为这意味着从任何源端口到HTTP应用程序（在端口80上运行）的任何数据包都是匹配的。
在端口设置中你也可以像上面描述的那样使用特定操作符, 例如:
| 操作符 | 描述 |
|---|---|
| : | 商品范围 |
| ! | 排除/取反 |
| [.., ..] | 组合 |
例如:
| 例子 | 含义 |
|---|---|
| [80, 81, 82] | 端口 80, 81 and 82 |
| [80: 82] | 从80到82端口 |
| [1024: ] | 从1024到最高端口号 |
| !80 | 除了80的任意端口 |
| [80:100,!99] | 从80到100，99除外 |
| [1:80,![2,4]] | 从1到80, 除开2和4 |
| [.., [..,..]] |
方向¶
方向告诉规则匹配哪个方向的流量，几乎所有的规则都有一个右箭头(->)来指示方向，这意味着只有相同方向的流量才会被匹配。但是，匹配双向流量的规则也是存在的 (<>):
source -> destination
source <> destination (both directions)
警告
没有相反方向, 即没有 <- 符号。
下图中的例子说明了这个问题， 比如，有一个IP地址为1.2.3.4和端口1024的客户端，以及一个IP地址为5.6.7.8的服务器，监听端口80（通常是HTTP）。客户端向服务器发送消息，服务器回复应答。
现在，我们有一条规则包含如下头部:
alert tcp 1.2.3.4 1024 -> 5.6.7.8 80
只有第一个数据包才会被这条规则匹配，因为方向确定了我们无法匹配应答数据包。
规则选项¶
规则剩下的部分由规则选项组成。 这些选项用括号括起来并用分号分隔。有些选项有设置内容(如 msg),
它由选项关键字指定, 后面跟上冒号,再后面就是设置值。其它的没有设置内容,仅仅只有选项关键字(如 nocase):
<keyword>: <settings>;
<keyword>;
规则选项具有特定的顺序，更改其顺序将改变规则的含义。
注解
字符 ; 和 " 在Suricata规则语言中具有特殊的含义，如果要在规则选项设置值中使用它们，必须要进行转义。例如:
msg:"Message with semicolon\;";
因此，您还必须转义反斜杠，因为它充当转义字符。
本章的剩下部分记录各种关键字的使用.
关键字的一些通用细节如下.
修饰符关键字¶
一些关键字会当作修饰符来工作。有两种类型的修饰符：
老一点的 ‘content修饰符’ 可以回顾一下规则, 如:
alert http any any -> any any (content:"index.php"; http_uri; sid:1;)
在上面的例子中，模式 ‘index.php’ 被强行用来检查HTTP uri缓冲区，而不是整个数据包内容.
比较新的类型被称作 ‘sticky buffer’. 它将缓冲区名称放在第一位，其后面的所有关键字都作用于该缓冲区, 例如:
alert http any any -> any any (http_response_line; content:"403 Forbidden"; sid:1;)
在上面的例子中，模式 ‘403 Forbidden’ 被用来检查HTTP响应头，这是因为他跟在
http_response_line关键字后面.
标准化缓冲区¶
数据包由原始数据组成,HTTP和重组会复制这些类型的数据包数据。它们清除异常内容，组合数据包等。 余下部分叫做“标准化缓冲区”:
因为数据正在规范化，所以它的内容发生了变化，这种规范化也是一种解析。 规范化缓冲区包括：所有HTTP关键字，重组流，TLS-，SSL-，SSH-，FTP-和dcerpc-缓冲区。
请注意有一些例外情况，例如 ``http_raw_uri``关键字， 更多信息参见 http_uri and http_raw_uri
元关键字¶
元关键字设置对Suricata的检测没有影响; 但是它却对Suricata的事件输出有影响。
msg (消息)¶
msg关键字指定规则产生的告警文本信息。
msg格式:
msg: "一些描述信息";
例如:
msg:"ATTACK-RESPONSES 403 Forbidden";
msg:"ET EXPLOIT SMB-DS DCERPC PnP bind attempt";
继续上一章的例子, 这是实际规则中的关键字:
小技巧
一般习惯于将规则选项的第一部分大写，并且用来表示规则类型。
同样习惯于将 msg 作为第一个规则选项关键字。
注解
在msg中下列字符必须转义:
; \ "
sid (规则ID)¶
sid关键字指定每条规则的id号，这个id用一个数字来表示，其格式是:
sid:123;
规则中sid的例子:
小技巧
一般习惯于将规则 sid 放到最后一个规则选项关键字 (或者如果有 ``rev``的话，就放到倒数第二个)。
rev (revision)¶
sid关键字几乎每次都伴随着rev，Rev代表规则的版本。如果修改了规则，则规则编写者应当增加rev的数值。rev规则选项关键字的格式是:
rev:123;
规则中rev的例子:
小技巧
一般习惯于将sid放在rev的前面, 并且它们一起放在所有关键字的最后。
gid (组ID)¶
gid关键字可用于为不同的签名组提供另一个id值（像sid一样）。 Suricata默认使用gid 1，我们可以修改这个gid。它通常不会被改变，改变它没有技术含义。你只能在告警中看到它。
fast.log告警中的gid示例。 在[1:2008124:2]中，1是gid（2008124是sid，2是rev）。
classtype¶
classtype关键字提供有关规则和警报分类的信息，它由短名称，长名称和优先级组成，它用来标识规则是仅仅是信息性的还是关于黑客的，classification.config指定了规则中使用的每个classtype的优先级。
classtype定义举例:
config classification: web-application-attack,Web Application Attack,1
config classification: not-suspicious,Not Suspicious Traffic,3
现在我们在配置中定义了这两个classtypes，就可以在规则中使用它们了。一条classtype为web-application-attack的规则，它的优先级被设置为1，并且产生的告警内容中会包含 ‘Web Application Attack’:
| classtype | 告警 | 优先级 |
|---|---|---|
| web-application-attack | Web Application Attack | 1 |
| not-suspicious | Not Suspicious Traffic | 3 |
接下来的例子也包含classtype, 它是 trojan-activity:
小技巧
通常习惯于将classtype放在sid and rev前面，其它的关键字后面.
参考¶
参考关键字表示指向规则有关的信息以及试图解决规则相关问题的位置。参考关键字可以在规则中多次出现，此关键字适用于规则编写者和分析师，他们会研究规则匹配的原因。 它的格式如下:
reference: type, reference
一个典型的指向www.info.com的参考是这样的:
reference: url, www.info.com
但是，参考类型有很多种，一个众所周知的例子是CVE数据库，它为漏洞分配编号。为了避免你多次输入相同URL的麻烦，你可以这样来定义参考:
reference: cve, CVE-2014-1234
这会定义一个指向http://cve.mitre.org/cgi-bin/cvename.cgi?name=CVE-2014-1234 的参考。 所有参考类型都在reference.config配置文件中定义。
下面的例子也定义了一个参考:
优先级¶
优先级关键字用一个从1到255数字表示，但是通常只使用从1到4，高优先级的规则将会优先检查，最高优先级是1。classtype中指定的优先级会被priority关键字优先级覆盖，priority关键字格式如下:
priority:1;
metadata¶
meatadata关键字允许将其他非功能性信息添加到规则中，虽然它的格式很自由，但建议使用key, value键值对，这是因为Suricata可以在eve告警中包含这些值。其格式是:
metadata: key value;
metadata: key value, key value;
target¶
target关键字允许规则编写者指定告警的哪一侧是攻击的目标，如果指定了，告警事件中会包含有关源和目标的信息。
其格式是:
target:[src_ip|dest_ip]
如果设置成src_ip，那么生成的事件(JSON中的src_ip字段) 中源IP就是攻击目标，如果设置成dest_ip，生成事件中目的IP就是攻击目标。
IP关键字¶
ttl¶
ttl关键字用于检查IP数据包标头中time-to-live值， 其格式是:
ttl:<number>
例如:
ttl:10;
在ttl关键字的末尾，您可以输入要匹配的值。Time-to-live值决定定数据包在网络中传输的最长时间，如果此字段设置为0，则必须销毁数据包，time-to-live基于跳数，数据包每经过一跳，TTL值就减一。这个机制的目的是限制数据包无限循环路由。
规则中ttl关键字例子:
ipopts¶
ipopts关键字用来检查特定的IP选项是否设定。Ipopts必须在规则的开头使用，每条规则只能匹配一个选项。 这些选项包括:
| IP选项 | 描述 |
|---|---|
| rr | 记录路由 |
| eol | 列表结束 |
| nop | 无操作 |
| ts | 时间戳 |
| sec | IP安全 |
| esec | IP安全扩展 |
| lsrr | 宽松源路由选项 |
| ssrr | 严格源路由选项 |
| satid | 流标识符 |
| any | 任意IP选项 |
ipopts关键字格式:
ipopts: <name>
例如:
ipopts: lsrr;
规则中使用ipopts的例子:
sameip¶
每个数据包都有一个源IP地址和一个目的IP地址。 可以是源IP与目的IP相同。 使用sameip关键字，您可以检查源IP地址是否与目的IP地址相同。 sameip 关键字的格式是:
sameip;
规则中使用sameip的例子:
ip_proto¶
使用ip_proto关键字，您可以匹配数据包头中的IP协议。您可以使用协议的名称或编号来匹配下面的协议:
1 ICMP 因特网消息控制协议
6 TCP 传输控制协议
17 UDP 用户数据报协议
47 GRE 通用路由封装协议
50 ESP IPv6 Encap安全有效负载
51 AH Ipv6身份验证标头
58 IPv6-ICMP ICMP for Ipv6
完整的协议和协议号列表请参考 http://en.wikipedia.org/wiki/List_of_IP_protocol_numbers
规则中使用ip_proto的例子:
例子中的命名变量为:
ip_proto:PIM
id¶
使用id关键字，您可以匹配指定的IP ID值。 ID标识主机发送的每个数据包，并且通常每发送一个数据包就递增一个数值。 IP ID用作片段标识号。 每个数据包都有一个IP ID，当数据包被分段时，该数据包的所有分段都具有相同的ID。 以这种方式，分组的接收器知道哪些分段属于同一数据包。 （IP ID不关心ID号顺序，分段重组时使用偏移来完成，它阐明了分段的顺序。）
id关键字格式为:
id:<number>;
规则中使用id的例子:
geoip¶
geoip关键字使您能够匹配源和目标IP地址来自哪个网络以及它所属的国家/地区。Suricata使用Maxmind的GeoIP API来实现这个功能。
geoip的语法格式为:
geoip: src, RU;
geoip: both, CN, RU;
geoip: dst, CN, RU, IR;
geoip: both, US, CA, UK;
geoip: any, CN, IR;
因此，您可以使用以下标识符来明确匹配的方向:
both: 使用指定geoip (geopip’s)来匹配双向
any: 使用指定geoip (’s)来匹配任意一个方向.
dest: 使用指定geoip来匹配目的.
src: 使用指定geoip来匹配源.
这个关键字仅支持IPv4. 由于他使用了Maxmind的GeoIP API, 必须将libgeoip编译入suricata.
fragbits (IP分段)¶
使用fragbits关键字，可以检查IP头中是否设置了分段和保留位。fragbits关键字应放在规则的开头。Fragbits用于IP分段机制。 在将数据包从一个Internet网络路由到另一个Internet网络期间，可能发生数据包大于网络可以处理的最大数据包长度的情况。 在这种情况下，可以分段发送数据包。网络中数据包长度的最大值称为最大传输单位（MTU）.
您可以匹配以下比特位:
M - 更多分段
D - 不要分段
R - 保留位
可以使用以下修饰符更多地指定对此位的匹配:
+ 匹配指定的位，以及任何其他位
* 如果设置了任何指定位，则匹配
! 如果未设置指定的位，则匹配
格式:
fragbits:[*+!]<[MDR]>;
规则中使用fragbits的例子:
fragoffset¶
使用fragoffset关键字，您可以匹配IP分段偏移字段的指定十进制值。 如果要检查会话的第一个分段，则必须将fragoffset 0与More Fragment选项组合在一起来使用。分段的偏移字段便于重组。 id用于确定哪些分段属于哪个数据包，而分段的偏移字段用于说明分段的顺序.
您可以使用以下修饰符:
< 如果小于指定值，则匹配
> 如果大于指定值，则匹配
! 如果指定的值不存在则匹配
fragoffset格式:
fragoffset:[!|<|>]<number>;
规则中使用fragoffset的例子:
TCP关键字¶
seq¶
可以在规则中使用seq关键字来检查指定的TCP序列号。序列号是TCP连接两端随机生成的数字。客户端和服务器都创建一个序列号，随着各方发送字节的增加而增加，所以双方这个序列号是不同的。该序列号必须由连接的双方确认。TCP通过序列号处理数据包的确认、顺序和重传。它的数字着发送方发送的每个数据字节而增加。seq有助于跟踪字节在所属的数据流中处于哪个位置。如果SYN标志设置为1，则数据的第一个字节的序列号是此数字加1（以此类推，2）
例子:
seq:0;
规则中使用seq的例子:
数据包中seq的例子 (Wireshark):
ack¶
ack是对TCP连接另一端发送的所有先前（数据）字节的接收的确认。 在大多数情况下，TCP连接的每个数据包在第一个SYN之后都有一个ACK标志，而ack-number随着每个新数据字节的接收而增加。 可以在规则中使用ack关键字来检查指定的TCP确认号.
ack的格式:
ack:1;
规则中使用ack的例子:
数据包中ack的例子(Wireshark):
window¶
window关键字用于检查指定的TCP窗口大小。TCP窗口大小是一种控制数据流的机制。窗口由接收端设置（接收端通告的窗口大小），并指示可以接收的字节数。在发送方可以发送相同数量的新数据之前，接收端必须先确认此数据量。该机制用于防止接收端数据溢出。窗口大小的值是有限的，可以是2到65535字节。为了更多地利用带宽，您可以使用更大的TCP窗口。
window关键字格式:
window:[!]<number>;
规则中使用window的例子:
ICMP关键字¶
ICMP（Internet消息控制协议）是IP协议栈的一部分。 在传输数据（数据报）方面，IP协议本身并不可靠。ICMP协议可以在出现问题时提供反馈。它不会阻止问题的发生，但有助于理解出错的原因和地点。如果需要可靠性，使用IP的协议必须自己处理可靠性。很多情况下会发送ICMP消息。 例如，当目的地不可达时，如果没有足够的缓冲容量来转发数据，或者当数据报发送时不能进行分段，则等等。 更多信息可参考ICMP消息类型列表。
ICMP消息有四个重要内容，可以与相应的ICMP关键字匹配。 它们是：消息的类型，代码，id和序列号。
itype¶
itype关键字用于匹配指定的ICMP类型（数字）。ICMP有几种消息，并使用代码来表示这些消息。不同的消息由不同的名称区分，但更重要的是类型代码。 有关更多信息，请参阅包含消息类型和代码列表。
itype关键字格式:
itype:min<>max;
itype:[<|>]<number>;
例如 这个例子匹配类型代码大于10的ICMP数据包:
itype:>10;
规则中使用itype关键字的例子:
下面列出了撰写本文时所有的已知ICMP类型。可以在 IANA的网站 找到最新的列表
| ICMP类型 | Name |
|---|---|
| 0 | 回显应答 |
| 3 | 目标不可达 |
| 4 | 源端被关闭（基本流控制） |
| 5 | 重定向 |
| 6 | Alternate Host Address |
| 8 | 回显请求 |
| 9 | 路由器通告 |
| 10 | 路由器请求 |
| 11 | TTL为0 |
| 12 | 参数错误 |
| 13 | 时间戳请求 |
| 14 | 时间戳请求应答 |
| 15 | 信息请求 |
| 16 | 信息请求应答 |
| 17 | 地址掩码请求 |
| 18 | 地址掩码应答 |
| 30 | Traceroute |
| 31 | 数据报转换错误 |
| 32 | Mobile Host Redirect |
| 33 | IPv6 Where-Are-You |
| 34 | IPv6 I-Am-Here |
| 35 | Mobile Registration Request |
| 36 | Mobile Registration Reply |
| 37 | Domain Name Request |
| 38 | Domain Name Reply |
| 39 | SKIP |
| 40 | Photuris |
| 41 | Experimental mobility protocols such as Seamoby |
icode¶
使用icode关键字，您可以匹配指定的ICMP代码。 ICMP消息的代码阐明了该消息。与ICMP类型一起，它表明您正在处理什么样的问题。每个ICMP类型的代码都有不同的用途。
icode关键字格式:
icode:min<>max;
icode:[<|>]<number>;
例如: 这个例子匹配ICMP代码大于5的数据包:
icode:>5;
规则中使用icode关键字的例子:
下面列出了所有ICMP类型的含义。如果未列出类型，则仅定义代码0，并且具有上表中ICMP类型的含义。 最新的列表可以在 IANA的网站 上找到。
| ICMP类型 | ICMP代码 | 描述 |
|---|---|---|
| 3 |
|
|
| 5 |
|
|
| 9 |
|
|
| 11 |
|
|
| 12 |
|
|
| 40 |
|
|
icmp_id¶
使用icmp_id关键字，您可以匹配指定的ICMP id值。每个ICMP数据包在发送时都会获得一个id。在接收方收到数据包时，它将使用相同的ID发送回复，以便发送方识别它并将其与正确的ICMP请求关联。
icmp_id关键字格式:
icmp_id:<number>;
例如: 这个例子匹配ICMP ID为0的数据包:
icmp_id:0;
规则中使用icmp_id关键字的例子:
icmp_seq¶
您可以使用icmp_seq关键字检查ICMP序列号。 ICMP消息都有序列号。 这可以用于（与id一起）用于检查哪个回复消息属于哪个请求消息。
icmp_seq关键字格式:
icmp_seq:<number>;
例如: 这个例子匹配ICMP序列号为0的数据包:
icmp_seq:0;
规则中使用icmp_seq关键字的例子:
Payload关键字¶
Payload关键字检查数据包或流的有效载荷的内容。
content¶
content关键字在规则中非常重要，您可以在引号之间写下您希望规则匹配的content。 最简单的content格式是:
content: ”............”;
在一条规则中使用多个content是有可能的。
content基于字节进行匹配。一个字节有256个不同的值（0-255）。你可以匹配所有字符; 从a到z，大写和小写以及所有特殊标记。但并非所有字节都是可打印字符。对于这些字节，可以使用十六进制来表示，许多编程语言使用0x00表示法，这里0x表示它涉及二进制值，但是规则语言使用 |00| 表示法。 这种表示法也可用于可打印字符。
例如:
|61| 表示 a
|61 61| 表示 aa
|41| 表示 A
|21| 表示 !
|0D| 表示 回车符
|0A| 表示 换行符
有些字符不允许在content中使用，因此这些字符是规则定义表示符，要匹配这些字符，你必须使用十六进制表示法。这些字符包括:
“ |22|
; |3B|
: |3A|
| |7C|
用大写字符来书写16进制符号是一种惯例。
例如，你要在规则的content中写 http:// , 应当这样写: content: “http|3A|//”; 如果你在规则中使用16进制, 一定要这些字符放在管道符之间。否则16进制的内容会被当作content普通文本的一部分。
一些例子:
content:“a|0D|bc”;
content:”|61 0D 62 63|";
content:”a|0D|b|63|”;
规则可以定义检查整个载荷与content匹配，或者仅检查载荷的特定部分，晚点我们会讨论这个问题。 如果你没有在规则中定义特殊的选项，那么将在整个载荷中进行匹配。
默认情况下，模式匹配区分大小写。content定义必须准确，否则不会匹配。
图例:
同样，可以使用 ! 来表示content内容取反（例外情况）匹配。
例如:
alert http $HOME_NET any -> $EXTERNAL_NET any (msg:"Outdated Firefox on
Windows"; content:"User-Agent|3A| Mozilla/5.0 |28|Windows|3B| ";
content:"Firefox/3."; distance:0; content:!"Firefox/3.6.13";
distance:-10; sid:9000000; rev:1;)
如 content:!”Firefox/3.6.13”;, 这意味着使用的Firefox版本不是3.6.13的时候，会产生一条报警。
注解
在content中，下列字符必须进行转义:
; \ "
nocase¶
如果您不想区分大小写字符，可以使用nocase。nocase关键字是content修饰符。
这个关键字的格式是:
nocase;
你必须把它放在你想修饰的content后面, 像这样:
content: “abc”; nocase;
nocase例子:
它对规则中的其他内容没有影响。
depth¶
depth关键字是一个绝对content修饰符，他跟在content后面。depth作为content修饰符必须要跟一个数字, 如:
depth:12;
depth后的数字表示将检查载荷从开头到多少字节的内容。
如:
startswith¶
startswith关键字类似于depth. 它没有任何参数并且必须跟在一个content关键字后面。它表示content将从一个缓冲区的开头进行匹配。
如:
content:"GET|20|"; startswith;
startswith 相当于下面规则的简写:
content:"GET|20|"; depth:4; offset:0;
对于同一个content定义， startswith 不能和 depth, offset, within 或 distance 写在一起。
offset¶
关键字指定从载荷中的哪个字节开始查找匹配。例如offset:3; 从第四个字节开始查找匹配。
offset和depth关键字可以并且经常组合在一起使用。
例如:
content:“def”; offset:3; depth:3;
如果在规则中使用图中组合，那么将检查载荷中的第四到第六个字节。
distance¶
distance关键字是相对content修饰符。意思是它表示当前content关键字与前一个content之间的关系。跟在前一个匹配后面，Distance有它的作用。distance关键字必须要跟一个数字。你指定的distance数值, 决定了载荷中从上一个匹配到当前匹配之间的字节数。Distance仅仅指定Suricata从哪里开始查找匹配。因此， distance:5; 表示可以在上一个匹配后面5字节的任何地方进行匹配。为了限制Suricata匹配载荷中的最后字节，使用 ‘within’ 关键字。
distance的例子:
Distance也可以是一个负值。这用来检测多个匹配之间部分相同的内容(参考例子)或者是完全在前一匹配之前的内容。但是这并不经常使用，因为有可能使用其它关键字来达到相同的目的。
within¶
within关键字是相对于前一个匹配而言的，它必须跟一个数字。使用within关键字需要确认的是，在载荷的前一个匹配到within设定的字节之间只有一个匹配。within不能设置为0。
如:
使用within进行匹配的例子:
第二部分内容（整体）必须距离第一部分内容3个字节以内。
如前所述，distance和within可以在规则中很好地组合在一起。如果你希望Suricata检查载荷中某个特定的部分，使用within.
isdataat¶
isdataat关键字的目的是检查载荷的特定部分是否存在数据。这个关键字以一个数字（表示位置）开始，然后可能跟上一个’relative’，以逗号与前面的数字分隔。这里的’relative’表示距离前一个匹配指定字节的位置是否存在数据。
你可以像下面两个例子这样使用:
isdataat:512;
isdataat:50, relative;
第一个例子表示将检查载荷的第512个字节是否存在数据，第二个例子表示载荷中上一个匹配后的第50个字节是否存在数据。
你可以在isdataat前面使用 (!) 来表示不存在数据则匹配。
dsize¶
你可以使用dsize关键字来检查整个载荷的长度，如可以检查载荷异常的长度。这在检测缓冲区溢出时会非常有用。
格式:
dsize:<number>;
规则中使用dsize的例子:
rpc¶
rpc关键字可用来在SUNRPC CALL中匹配RPC过程编号和RPC版本。
您可以使用通配符表示匹配模式，使用*通配符可以匹配所有版本和/或过程编号。
RPC（远程过程调用）是一种允许计算机程序在另一台计算机（或地址空间）上执行过程的应用程序,它用于进程间通信。参见 http://en.wikipedia.org/wiki/Inter-process_communication
格式:
rpc:<application number>, [<version number>|*], [<procedure number>|*]>;
规则中使用rpc关键字的例子:
replace¶
replace作为content修饰符只能在ips中使用，它会改变网络流量，会将后面的内容从 (‘abc’) 替换成(‘def’)。如:
replace修饰符必须包含与其替换的内容一样多的字符,它只能用于单个数据包，不能在 标准化缓冲区 如HTTP uri中使用或重组流中进行匹配。
在使用replace关键字后，Suricata会重新计算校验和并更改。
pcre (兼容Perl的正则表达式)¶
关键字pcre使用正则表达式进行特定匹配。有关正则表达式的更多信息，请访问此处 http://en.wikipedia.org/wiki/Regular_expression.
pcre复杂度高，并且对性能影响大。因此，为了避免Suricata经常进行pcre正则匹配，pcre大多与 ‘content’一起使用。在这种情况下，必须要先匹配content，再进行pcre检查（正则匹配）。
pcre的格式:
pcre:"/<regex>/opts";
pcre的例子。在此示例中，如果载荷中包含六个数字，则将匹配:
pcre:"/[0-9]{6}/";
规则中使用pcre的例子:
有一些pcre的特性可以修改:
- pcre默认是大小写敏感的.
- .（点）是正则表达式的一部分，它匹配除换行符之外的每个字节.
- 默认情况下，载荷将被当作一行来检查.
可以使用以下字符修改这些特性:
i pcre大小写不敏感
s pcre会检查换行符
m 可以使一行（载荷）计为两行
这些选项是perl兼容修饰符。要使用这些修饰符，您应该将它们添加到正则表达式后面。像这样:
pcre: “/<regex>/i”;
Pcre兼容修饰符
还有一些pcre兼容的修饰符可以改变它的特性，它们是:
A: 模式必须从缓冲区的开头匹配. (在pcre中 ^ 跟这里的 A 类似.)E: 忽略缓冲区/载荷末尾的换行符.G: 切换贪婪模式匹配.
注解
必须在内容中转义以下字符:
; \ "
Suricata的修饰符¶
Suricata有自己特定的pcre修饰符。它们是:
R: 相对于前一个模式匹配的匹配。类似于distance:0;U: 使用pcre匹配规范化的uri。它匹配uri_buffer就像uricontent和content组合http_uri.U一样，可以和/R一起组合使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-uri缓冲区内，更多信息请参考 HTTP URI Normalization.
I: 在HTTP-raw-uri上进行pcre匹配，在http_raw_uri的同一个缓冲区上进行匹配，可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-raw-uri缓冲区内，更多信息请参考 HTTP URI Normalization.P: 在HTTP-request-body上进行pcre匹配，在http_client_body的同一个缓冲区上进行匹配， P 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-request-body内.Q: 在HTTP-response-body上进行pcre匹配，在http_server_body的同一个缓冲区上进行匹配， Q 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-response-body内.H: 在HTTP-header上进行pcre匹配， Q 可以和/R一起使用。注意 H 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-response-header内.D: 在非标准化的头部上进行匹配，在http_raw_header的同一个缓冲区上进行匹配， D 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-raw-header内。M: 在request-method上进行匹配，在http_method的同一个缓冲区上进行匹配， M 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-method内。C: 在HTTP-cookie上进行匹配，在http_cookie的同一个缓冲区上进行匹配， C 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-cookie内。S: 在HTTP-stat-code上进行匹配，在http_stat_code的同一个缓冲区上进行匹配， S 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-stat-code内。Y: 在HTTP-stat-msg上进行匹配，在http_stat_msg的同一个缓冲区上进行匹配， Y 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-stat-msg内。B: You can encounter B in signatures but this is just for compatibility. So, Suricata does not use B but supports it so it does not cause errors.您可能在规则中遇到B，但这只是为了兼容性。 因此，Suricata不使用B但支持它，这样不会导致错误。O: 覆盖pcre匹配限制的配置。V: 在HTTP-User-Agent上进行匹配，在http_user_agent的同一个缓冲区上进行匹配， V 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-User-Agent内。W: 在HTTP-Host上进行匹配，在http_host的同一个缓冲区上进行匹配， W 可以和/R一起使用。注意 R 是相对于上一个匹配而言的，所以两个匹配都必须在HTTP-Host内。
预过滤关键字¶
fast_pattern¶
Suricata快速匹配模式选择详解¶
如果在规则中明确设置了’fast_pattern’关键字，Suricata将采用快速模式匹配。’fast_pattern’关键字在一条规则中只能设置一次。如果规则中没有设置’fast_pattern’，Suricata将自动选择一个content来使用快速模式匹配。
以下说明Suricata快速模式匹配自动选择的逻辑。
请注意，如果存在正（即非否定）content匹配，则忽略否定content匹配以进行快速模式选择。 如果不存在，才考虑否定的content匹配。
fast_pattern选择标准如下:
- Suricata首先匹配规则中具有最高“优先级”的所有content，优先级基于匹配的缓冲区，通常’http_*’ 缓冲区具有更高的优先级 (更小的数字代表更高的优先级)。关于什么样的缓冲区拥有什么样的优先级请参阅 附录 B .
- 在步骤1中进行的content匹配（最高优先级content匹配）中，最长（就字符/字节长度而言）content匹配使用快速模式匹配。
- 如果多个content匹配具有相同的最高优先级并且都是最长长度，则具有最高字符/字节多样性评分（“模式强度”）的那个匹配用作快速模式匹配。关于确定模式强度的算法详情请参见 附录 C .
- 如果多个content匹配具有相同的最高优先级，都是最长长度，且具有相同的最高模式强度，则*最后注册*的缓冲区(“list_id”) 使用快速模式匹配。 不同的缓冲区/列表注册顺序详情请参阅 附录 B .
- 如果多个content匹配具有相同的最高优先级，都是最长长度，具有相同的最高模式强度，并且具有相同的list_id(即在同一个缓冲区内匹配)，那么content按照在规则中从左到右的顺序，第一个content使用快速匹配。
值得注意的是，对于具有相同优先级，长度和模式强度的content匹配，’http_stat_msg’，’http_stat_code’和’http_method’优先于常规’content’匹配。
附录¶
在Suricata 1.1.x - 1.4.x中，这些应该都是一样的.
| list_id | 内容修饰符关键字 | 缓冲区名 | 注册顺序 |
|---|---|---|---|
| 1 | <none> (regular content match) | DETECT_SM_LIST_PMATCH | 1 (first) |
| 2 | http_uri | DETECT_SM_LIST_UMATCH | 2 |
| 6 | http_client_body | DETECT_SM_LIST_HCBDMATCH | 3 |
| 7 | http_server_body | DETECT_SM_LIST_HSBDMATCH | 4 |
| 8 | http_header | DETECT_SM_LIST_HHDMATCH | 5 |
| 9 | http_raw_header | DETECT_SM_LIST_HRHDMATCH | 6 |
| 10 | http_method | DETECT_SM_LIST_HMDMATCH | 7 |
| 11 | http_cookie | DETECT_SM_LIST_HCDMATCH | 8 |
| 12 | http_raw_uri | DETECT_SM_LIST_HRUDMATCH | 9 |
| 13 | http_stat_msg | DETECT_SM_LIST_HSMDMATCH | 10 |
| 14 | http_stat_code | DETECT_SM_LIST_HSCDMATCH | 11 |
| 15 | http_user_agent | DETECT_SM_LIST_HUADMATCH | 12 (last) |
注意: 在决定Suricata 1.3.4的快速模式匹配时，注册顺序无关紧要，但list_id值起作用。
在Suricata 2.0.x 中，这些应该都是一样的.
| 优先级 (数字小代表高优先级) | 注册顺序 | 内容修饰符关键字 | 缓冲区名字 | list_id |
|---|---|---|---|---|
| 3 | 11 | <none> (regular content match) | DETECT_SM_LIST_PMATCH | 1 |
| 3 | 12 | http_method | DETECT_SM_LIST_HMDMATCH | 12 |
| 3 | 13 | http_stat_code | DETECT_SM_LIST_HSCDMATCH | 9 |
| 3 | 14 | http_stat_msg | DETECT_SM_LIST_HSMDMATCH | 8 |
| 2 | 1 (first) | http_client_body | DETECT_SM_LIST_HCBDMATCH | 4 |
| 2 | 2 | http_server_body | DETECT_SM_LIST_HSBDMATCH | 5 |
| 2 | 3 | http_header | DETECT_SM_LIST_HHDMATCH | 6 |
| 2 | 4 | http_raw_header | DETECT_SM_LIST_HRHDMATCH | 7 |
| 2 | 5 | http_uri | DETECT_SM_LIST_UMATCH | 2 |
| 2 | 6 | http_raw_uri | DETECT_SM_LIST_HRUDMATCH | 3 |
| 2 | 7 | http_host | DETECT_SM_LIST_HHHDMATCH | 10 |
| 2 | 8 | http_raw_host | DETECT_SM_LIST_HRHHDMATCH | 11 |
| 2 | 9 | http_cookie | DETECT_SM_LIST_HCDMATCH | 13 |
| 2 | 10 | http_user_agent | DETECT_SM_LIST_HUADMATCH | 14 |
| 2 | 15 (last) | dns_query | DETECT_SM_LIST_DNSQUERY_MATCH | 20 |
注意: 在决定Suricata 2.0.7的快速模式匹配时，list_id无关紧要，但注册顺序值起作用.
来自detect-engine-mpm.c. 基本上，模式强度“评分”从零开始，从左到右检查传入的字节数组中的每个字符/字节。如果当前字符/字节是第一次出现，接下来根据当前字符/字节类型来加分，如果当前字符是字母，则分数加3，如果不是字母字符，但是可打印字符或者是0x00, 0x01, 0xFF，则分数加4，如果不是这些字符，那么分数加6。如果不是第一次出现的话，分数加1。最终得到模式强度评分。
/** \brief Predict a strength value for patterns
*
* Patterns with high character diversity score higher.
* Alpha chars score not so high
* Other printable + a few common codes a little higher
* Everything else highest.
* Longer patterns score better than short patters.
*
* \param pat pattern
* \param patlen length of the pattern
*
* \retval s pattern score
*/
uint32_t PatternStrength(uint8_t *pat, uint16_t patlen) {
uint8_t a[256];
memset(&a, 0 ,sizeof(a));
uint32_t s = 0;
uint16_t u = 0;
for (u = 0; u < patlen; u++) {
if (a[pat[u]] == 0) {
if (isalpha(pat[u]))
s += 3;
else if (isprint(pat[u]) || pat[u] == 0x00 || pat[u] == 0x01 || pat[u] == 0xFF)
s += 4;
else
s += 6;
a[pat[u]] = 1;
} else {
s++;
}
}
return s;
}
在多模式匹配器（MPM）中仅使用规则中的一个content，如果有多个content，那么Suricata会使用’strongest’ content，strong由长度、content的变化程度以及正在检查什么样的缓冲区这样一个组合来决定。通常情况下，长度越长和变化程度越高越好。有关Suricata如何确定快速模式匹配的完整详细信息参见 Suricata快速匹配模式选择详解.
有时规则的作者会得出这样的结论：他宁愿让Suricata选择另外一个content，而不是按它默认的选择去做。
例如:
User-agent: Mozilla/5.0 Badness;
content:”User-Agent|3A|”;
content:”Badness”; distance:0;
在这个例子中，您看到第一个content比第二个content更长且更具多变性，因此您知道Suricata将使MPM使用此content进行匹配。但是由于“User-Agent：”会经常被匹配到，并且“Badness”在网络流量中出现得较少，因此您可以使Suricata使“fast_pattern”使用第二个content进行匹配。
content:”User-Agent|3A|”;
content:”Badness”; distance:0; fast_pattern;
fast_pattern关键字对他前面的content起作用。
Fast-pattern也可以和前面提到的所有关键字以及所有提到的HTTP修饰符组合在一起使用。
fast_pattern:only¶
有时候一条规则只有一个content,在这种情况下，使用MPM找到匹配后，Suricata将不再进行检查。如果只有一个content,那么整条规则匹配了，Suricata会自动注意到这一点。在某些规则中，仍然用 ‘fast_pattern:only;’表示。尽管Suricata没有必要使用fast_pattern:only,但是仍然支持这种写法.
fast_pattern:’chop’¶
如果您不希望MPM使用整个content，可以使用fast_pattern ‘chop’。
如:
content: “aaaaaaaaabc”; fast_pattern:8,4;
这样的话， MPM只使用从第8字节开始的4个字节（也就是最后4个字节）进行匹配.
预过滤¶
可以使用’prefilter’关键字在特定规则中启用其他非MPM关键字的预过滤引擎。
在下面的规则中，TTL检测将用于预过滤而不是那个单字节的模式匹配:
alert ip any any -> any any (ttl:123; prefilter; content:"a"; sid:1;)
有关如何配置预过滤引擎的更多信息请参见 Prefilter Engines
流关键字¶
flowbits¶
Flowbits由两部分组成。 第一部分描述了它将要执行的操作，第二部分是flowbit的名称。
一个流有多个数据包，Suricata将这些流保存在内存中。更多信息请参阅 Flow Settings. Flowbits可以确保在多数据包匹配时生成告警。例如一条规则决定只有两个数据包都匹配时才会生成告警，因此，当第二个数据包匹配时，Suricata必须知道第一个数据包是否也匹配。如果第一个数据包匹配，Flowbits标记流，那么Suricata’知道’它应该在第二个数据包也匹配时生成告警。
Flowbits有不同的动作。包括:
flowbits: set, name 可能的话，在流中设置条件/'name'
flowbits: isset, name 在规则匹配并且设置条件的时候产生一条告警
flowbits: toggle, name 反转当前设置。所以如果设置了某个条件，那么将取消设置，反之亦然。
flowbits: unset, name 用来取消流中的条件设置.
flowbits: isnotset, name 在规则匹配并且未设置条件的时候产生一条告警
flowbits: noalert 禁止规则产生告警.
如:
当你查看第一条规则时，你会发现如果不是规则末尾的’flowbits: noalert’，那么这条规则在匹配时会产生一条告警。这条规则的目的是匹配’userlogin’并在流中标记，因此没有必要生成告警。 如果没有第一条规则，第二条规则则无效。如果第一条规则匹配，flowbits在流中设置一个特定的条件成立，现在使用第二条规则可以检查先前的数据包是否满足该特定条件。 如果满足并且第二条规则匹配，则会产生一条告警。
可以在规则中多次使用flowbits并组合不同的功能。
flow¶
flow关键字可用于匹配流的方向，客户端<=>服务器，还可以匹配流是否established。flow关键字也可用于表示规则必须仅在流上(only_stream) 或仅在数据包上匹配(no_stream)。
所以，你可以用flow关键来匹配:
- to_client
- 匹配从服务器到客户端的数据包.
- to_server
- 匹配从客户端到服务器的数据包.
- from_client
- 匹配从客户端到服务器的数据包（与to_server相同）.
- from_server
- 匹配从服务器到客户端的数据包（与to_client相同）.
- established
- 匹配established连接.
- not_established
- 匹配不属于established连接的数据包.
- stateless
- 匹配无论是否属于established连接的数据包.
- only_stream
- 匹配已由流引擎重新组装的数据包.
- no_stream
- 匹配尚未由流引擎重组的数据包。 将不匹配已重新组装的数据包.
- only_frag
- 匹配已从分段重组的数据包.
- no_frag
- 匹配尚未进行分段重组的数据包.
多个flow选项可以组合使用,如
flow:to_client, established
flow:to_server, established, only_stream
flow:to_server, not_established, no_frag
*established* 的确认依赖于协议:
TCP连接在三次握手后变成established.
其它协议(如UDP), 只要看见通信双方的流量就可以认为连接建立(established)了.
flowint¶
我们将很快在引擎中添加全局变量任务，flowint是这个任务的前身，这将允许在变量中捕获，存储和比较数据。顾名思义他是全局的，因此，您可以比较来自不相关流中的数据包的数据。
Flowint允许使用变量进行存储和数学运算。它的运行方式与flowbits非常相似，但增加了数学功能，并且可以存储和操作整数，而不仅仅是标志的集合。我们可以将它用于许多非常有用的事情，例如统计次数，添加或减去次数，或者在流中对多个因子进行阈值处理。 这将很快扩展到全局上下文，因此用户可以在流之间执行这些操作。
它的语法如下:
flowint: , ;
定义一个变量(非必须), 或检查变量是否设置.
flowint: , , ;
flowint: , < +,-,=,>,<,>=,<=,==, != >, ;
比较或改变变量。 加，减，大于或小于，大于等于，小于等于都可以使用。比较项可以是整数或其他变量。
例如，如果您想要统计在特定流中某用户名出现的次数，次数超过5时告警.
alert tcp any any -> any any (msg:"Counting Usernames"; content:"jonkman"; \
flowint: usernamecount, +, 1; noalert;)
这条规则规定用户名每出现一次，变量usernamecount就加1，不会每次出现产生告警。
现在我们想让它在流中出现超过5次时告警.
alert tcp any any -> any any (msg:"More than Five Usernames!"; content:"jonkman"; \
flowint: usernamecount, +, 1; flowint:usernamecount, >, 5;)
因此，仅当usernamecount大于5时，我们会得到一条告警.
So now let’s say we want to get an alert as above but NOT if there have been more occurrences of that username logging out. 现在我们可以说在没有用户注销的情况下，可以以上面的规则得到一条告警。假设此特定协议使用”jonkman logout”表示注销，尝试:
alert tcp any any -> any any (msg:"Username Logged out"; content:"logout jonkman"; \
flowint: usernamecount, -, 1; flowint:usernamecount, >, 5;)
所以现在只有当这个特定用户名有超过五个活动登录时，才会产生告警。
这是一个相当简单的例子，但我相信它显示了这样一个简单的函数可以为规则编写做些什么。在登录跟踪，IRC状态机，恶意软件跟踪和暴力破解检测等方面都可以应用。
假设我们正在跟踪一个通常允许每个连接有五次登录失败的协议，但是我们有一个漏洞，攻击者可以在五次尝试之后继续登录，我们需要了解它之后的登录。
alert tcp any any -> any any (msg:"Start a login count"; content:"login failed"; \
flowint:loginfail, notset; flowint:loginfail, =, 1; noalert;)
第一击，如果尚未设置变量，我们会检测到第一次登陆失败，定义变量将并其设置为1。
alert tcp any any -> any any (msg:"Counting Logins"; content:"login failed"; \
flowint:loginfail, isset; flowint:loginfail, +, 1; noalert;)
如果设置变量，我们就将其加1.
alert tcp any any -> any any (msg:"More than Five login fails in a Stream"; \
content:"login failed"; flowint:loginfail, isset; flowint:loginfail, >, 5;)
如果在同一个流中超过5次登陆失败就产生一条告警.
但是，我们想在两次成功登录并且之后登录失败的情况下，也产生告警.
alert tcp any any -> any any (msg:"Counting Good Logins"; content:"login successful"; \
flowint:loginsuccess, +, 1; noalert;)
这里我们统计成功登陆，现在我们统计成功登陆相关的失败登陆:
alert tcp any any -> any any (msg:"Login fail after two successes"; \
content:"login failed"; flowint:loginsuccess, isset; flowint:loginsuccess, =, 2;)
下面是一些其它的例子:
alert tcp any any -> any any (msg:"Setting a flowint counter"; content:"GET"; \
flowint:myvar, notset; flowint:maxvar,notset; \
flowint:myvar,=,1; flowint: maxvar,=,6;)
alert tcp any any -> any any (msg:"Adding to flowint counter"; \
content:"Unauthorized"; flowint:myvar,isset; flowint: myvar,+,2;)
alert tcp any any -> any any (msg:"if the flowint counter is 3 create a new counter"; \
content:"Unauthorized"; flowint:myvar, isset; flowint:myvar,==,3; \
flowint:cntpackets,notset; flowint:cntpackets, =, 0;)
alert tcp any any -> any any (msg:"count the rest without generating alerts"; \
flowint:cntpackets,isset; flowint:cntpackets, +, 1; noalert;)
alert tcp any any -> any any (msg:"fire this when it reach 6"; \
flowint: cntpackets, isset; \
flowint: maxvar,isset; flowint: cntpackets, ==, maxvar;)
stream_size¶
流大小选项根据序列号注册的字节数匹配流量。这个关键字有多个修饰符:
> 大于
< 小于
= 等于
!= 不等于
>= 大于等于
<= 小于等于
格式
stream_size:<server|client|both|either>, <modifier>, <number>;
规则中使用stream-size关键字的例子:
HTTP关键字¶
还有其他内容修饰符可以在应用程序层提供特定于协议的功能。可以在以下位置找到更多信息 Payload关键字 .这些关键字确保规则仅检查网络流量的特定部分。例如，特别检查请求URI，cookie或HTTP请求或响应正文等。
所有HTTP关键字都是修饰符,注意内容修饰符和粘滞缓冲区（sticky buffer）之间的区别。 有关更多信息，请参阅 修饰符关键字 . 对于新手:
‘content modifiers’ 回顾规则，例如:
alert http any any -> any any (content:"index.php"; http_uri; sid:1;)
‘sticky buffers’ 前置并且它后面所有的关键字都作用在该缓冲区, 例如:
alert http any any -> any any (http_response_line; content:"403 Forbidden"; sid:1;)
request 关键字如下:
| 关键字 | Sticky或修饰符 | 方向 |
|---|---|---|
| http_uri | Modifier | Request |
| http_raw_uri | Modifier | Request |
| http_method | Modifier | Request |
| http_request_line | Sticky Buffer | Request |
| http_client_body | Modifier | Request |
| http_header | Modifier | Both |
| http_raw_header | Modifier | Both |
| http_cookie | Modifier | Both |
| http_user_agent | Modifier | Request |
| http_host | Modifier | Request |
| http_raw_host | Modifier | Request |
| http_accept | Sticky Buffer | Request |
| http_accept_lang | Sticky Buffer | Request |
| http_accept_enc | Sticky Buffer | Request |
| http_referer | Sticky Buffer | Request |
| http_connection | Sticky Buffer | Request |
| http_content_type | Sticky Buffer | Both |
| http_content_len | Sticky Buffer | Both |
| http_start | Sticky Buffer | Both |
| http_protocol | Sticky Buffer | Both |
| http_header_names | Sticky Buffer | Both |
response 关键字如下:
| 关键字 | Sticky或修饰符 | 方向 |
|---|---|---|
| http_stat_msg | Modifier | Response |
| http_stat_code | Modifier | Response |
| http_response_line | Sticky Buffer | Response |
| http_header | Modifier | Both |
| http_raw_header | Modifier | Both |
| http_cookie | Modifier | Both |
| http_server_body | Modifier | Response |
| file_data | Sticky Buffer | Response |
| http_content_type | Sticky Buffer | Both |
| http_content_len | Sticky Buffer | Both |
| http_start | Sticky Buffer | Both |
| http_protocol | Sticky Buffer | Both |
| http_header_names | Sticky Buffer | Both |
HTTP 入门¶
了解HTTP请求和响应的结构非常重要。.一个简单的HTTP请求和响应的例子如下:
HTTP request
GET /index.html HTTP/1.0\r\n
GET 是请求 method. 请求的例子有: GET, POST, PUT, HEAD等等. URI路径为 /index.html， HTTP版本是 HTTP/1.0. 过去的这些前，使用过多个HTTP版本; 包括0.9, 1.0 和 1.1, 现在用得最广泛的是1.0 和 1.1.
HTTP response
HTTP/1.0 200 OK\r\n
<html>
<title> some page </title>
</HTML>
在这个例子里, HTTP/1.0 是HTTP版本, 200 是响应状态码， OK是响应状态信息.
另一个更加详细的例子:
请求:
响应:
请求:
尽管cookie在HTTP头部里发送，但是你不能使用 http_header 关键字来匹配它，cookie使用它自己的关键字来进行匹配，叫 http_cookie.
表的每个部分属于一个所谓的 buffer. HTTP方法属于方法缓冲区，HTTP头部属于头部缓冲区等。缓冲区是Suricata在内存中提取以进行检查的请求或响应的特定部分。
所有先前描述的关键字可以规则中的缓冲区组合使用。关键字 distance 和 within 是相对修饰符，因此它们只能在同一个缓冲区中使用。您不能使用相对修饰符将content匹配与不同缓冲区相关联。
http_method¶
通过 http_method 修饰符, 可以仅在HTTP方法缓冲区上进行匹配，这个关键字可以和所有之前提到过的content修饰符组合使用，如: depth, distance, offset, nocase and within.
HTTP方法的例子有: GET, POST, PUT, HEAD, DELETE, TRACE, OPTIONS, CONNECT and PATCH.
HTTP请求中的方法:
方法的目的:
http_uri and http_raw_uri¶
通过 http_uri 和 http_raw_uri content修饰符, 可以仅在请求URI缓冲区上匹配。这个关键字可以和所有之前提到过的content修饰符组合使用，如: depth, distance, offset, nocase and within.
uri在Suricata中有两种形式：raw_uri和规范化的uri。例如，空格可以用十六进制符号％20表示，将这个十六进制转换成空格，就表示将其规范化。虽然可以匹配uri中字符％20的特定内容，但这意味着匹配raw_uri。raw_uri和规范化的uri是不同的缓冲区。因此，raw_uri检查raw_uri缓冲区并且无法检查规范化缓冲区。
HTTP请求中URI的例子:
uricontent¶
uricontent关键字和http_uricontent关键字具有完全一样的作用.uricontent是一个过时的(但是仍然支持的) 方法，表示仅在请求URI缓冲区上匹配.
uricontent示例:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS (msg:”ET TROJAN Possible Vundo Trojan Variant reporting to Controller”; flow:established,to_server; content:”POST “; depth:5; uricontent:”/frame.html?”; urilen: > 80; classtype:trojan-activity; reference:url,doc.emergingthreats.net/2009173; reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/VIRUS/TROJAN_Vundo; sid:2009173; rev:2;)
http_uri and uricontent 的区别在于语法:
在编写新规则时, 建议使用 http_uri content修饰符，而不是已经过时的 uricontent 关键字.
urilen¶
urilen关键字用来匹配请求URI的长度，可以使用<和>操作符, 分别表示 小于 和 大于.
urilen格式是:urilen:3;
其它可能的格式:
urilen:1;
urilen:>1;
urilen:<10;
urilen:10<>20; (大于 10, 小于 20)
如:
规则中使用 urilen 的例子:
你可以添加 norm 或 raw 来定义你想使用什么类型的缓冲区(规范化的或者raw缓冲区).
http_protocol¶
http_protocol检查HTTP请求或者响应中的协议字段。 如果请求行是 ‘GET / HTTP/1.0rn’, 那么这个缓冲区就是 ‘HTTP/1.0’.
如:
alert http any any -> any any (flow:to_server; http_protocol; content:"HTTP/1.0"; sid:1;)
http_request_line¶
http_request_line强制检查整个HTTP请求行.
如:
alert http any any -> any any (http_request_line; content:"GET / HTTP/1.0"; sid:1;)
http_header 和 http_raw_header¶
通过 http_header content修饰符, 可以仅在HTTP头部缓冲区匹配，这包含提取到单个缓冲区中的所有头部，除开文档中指示的那些无法通过此缓冲区匹配并具有自己的content修饰符的头部(如 http_cookie). 这个关键字可以和所有之前提到过的content修饰符组合使用，如: depth, distance, offset, nocase and within.
Note: 头部缓冲区是 规范化的. 末尾的所有空格和制表符都已经删除。 详见: https://lists.openinfosecfoundation.org/pipermail/oisf-users/2011-October/000935.html. 要避免这一点，请使用http_raw_header关键字.
HTTP请求中头部的例子:
http_cookie¶
通过 http_cookie content修饰符, 可以仅在cookie缓冲区上匹配，这个关键字可以和所有之前提到过的content修饰符组合使用，如: depth, distance, offset, nocase and within.
注意cookie在HTTP头部里传输，但是会提取到一个单独的缓冲区，通过他自己特有的content修饰符来匹配。
HTTP请求中cookie的例子:
http_user_agent¶
http_user_agentcontent修饰符是HTTP请求头部的一部分，可以仅在User-Agent请求头上匹配。它是规范化的，这意味着它不包括 _”User-Agent: “_ 头部名称和分隔符，也不包括末尾的回车换行符(CRLF)。 T这个关键字可以和所有之前提到过的content修饰符组合使用，如:depth,distance,offset,nocaseandwithin. 注意pcre关键字在使用/V修饰符时，也可以检查这个缓冲区。
规范化: 开头的空白字符 不是 这个缓冲区的一部分，因此. “User-Agent: rn” 会导致一个空的 http_user_agent 缓冲区。
HTTP请求中User-Agent的例子:
http_user_agent 示例:
注意¶
The
http_user_agentbuffer will NOT include the header name, colon, or leading whitespace. i.e. it will not include “User-Agent: “.The
http_user_agentbuffer does not include a CRLF (0x0D 0x0A) at the end. If you want to match the end of the buffer, use a relativeisdataator a PCRE (although PCRE will be worse on performance).If a request contains multiple “User-Agent” headers, the values will be concatenated in the
http_user_agentbuffer, in the order seen from top to bottom, with a comma and space (“, “) between each of them.Example request:
GET /test.html HTTP/1.1 User-Agent: SuriTester/0.8 User-Agent: GGGG
http_user_agentbuffer contents:SuriTester/0.8, GGGG
Corresponding PCRE modifier:
VUsing the
http_user_agentbuffer is more efficient when it comes to performance than using thehttp_headerbuffer (~10% better).https://blog.inliniac.net/2012/07/09/suricata-http_user_agent-vs-http_header/
http_accept¶
Sticky buffer to match on the HTTP Accept header. Only contains the header value. The \r\n after the header are not part of the buffer.
Example:
alert http any any -> any any (http_accept; content:"image/gif"; sid:1;)
http_accept_enc¶
Sticky buffer to match on the HTTP Accept-Encoding header. Only contains the header value. The \r\n after the header are not part of the buffer.
Example:
alert http any any -> any any (http_accept_enc; content:"gzip"; sid:1;)
http_accept_lang¶
Sticky buffer to match on the HTTP Accept-Language header. Only contains the header value. The \r\n after the header are not part of the buffer.
Example:
alert http any any -> any any (http_accept_lang; content:"en-us"; sid:1;)
http_connection¶
Sticky buffer to match on the HTTP Connection header. Only contains the header value. The \r\n after the header are not part of the buffer.
Example:
alert http any any -> any any (http_connection; content:"keep-alive"; sid:1;)
http_content_type¶
Sticky buffer to match on the HTTP Content-Type headers. Only contains the header value. The \r\n after the header are not part of the buffer.
Use flow:to_server or flow:to_client to force inspection of request or response.
Examples:
alert http any any -> any any (flow:to_server; \
http_content_type; content:"x-www-form-urlencoded"; sid:1;)
alert http any any -> any any (flow:to_client; \
http_content_type; content:"text/javascript"; sid:2;)
http_content_len¶
Sticky buffer to match on the HTTP Content-Length headers. Only contains the header value. The \r\n after the header are not part of the buffer.
Use flow:to_server or flow:to_client to force inspection of request or response.
Examples:
alert http any any -> any any (flow:to_server; \
http_content_len; content:"666"; sid:1;)
alert http any any -> any any (flow:to_client; \
http_content_len; content:"555"; sid:2;)
To do a numeric inspection of the content length, byte_test can be used.
Example, match if C-L is equal to or bigger than 8079:
alert http any any -> any any (flow:to_client; \
http_content_len; byte_test:0,>=,8079,0,string,dec; sid:3;)
http_referer¶
Sticky buffer to match on the HTTP Referer header. Only contains the header value. The \r\n after the header are not part of the buffer.
Example:
alert http any any -> any any (http_referer; content:".php"; sid:1;)
http_start¶
Inspect the start of a HTTP request or response. This will contain the request/reponse line plus the request/response headers. Use flow:to_server or flow:to_client to force inspection of request or response.
Example:
alert http any any -> any any (http_start; content:"HTTP/1.1|0d 0a|User-Agent"; sid:1;)
The buffer contains the normalized headers and is terminated by an extra \r\n to indicate the end of the headers.
http_header_names¶
Inspect a buffer only containing the names of the HTTP headers. Useful for making sure a header is not present or testing for a certain order of headers.
Buffer starts with a \r\n and ends with an extra \r\n.
Example buffer:
\\r\\nHost\\r\\n\\r\\n
Example rule:
alert http any any -> any any (http_header_names; content:"|0d 0a|Host|0d 0a|"; sid:1;)
Example to make sure only Host is present:
alert http any any -> any any (http_header_names; \
content:"|0d 0a|Host|0d 0a 0d 0a|"; sid:1;)
Example to make sure User-Agent is directly after Host:
alert http any any -> any any (http_header_names; \
content:"|0d 0a|Host|0d 0a|User-Agent|0d 0a|"; sid:1;)
Example to make sure User-Agent is after Host, but not necessarily directly after:
alert http any any -> any any (http_header_names; \
content:"|0d 0a|Host|0d 0a|"; content:"|0a 0d|User-Agent|0d 0a|"; \
distance:-2; sid:1;)
http_client_body¶
With the http_client_body content modifier, it is possible to
match specifically and only on the HTTP request body. The keyword can
be used in combination with all previously mentioned content modifiers
like distance, offset, nocase, within, etc.
Example of http_client_body in a HTTP request:
Example of the purpose of http_client_body:
Note: how much of the request/client body is inspected is controlled
in the libhtp configuration section via the request-body-limit
setting.
http_stat_code¶
With the http_stat_code content modifier, it is possible to match
specifically and only on the HTTP status code buffer. The keyword can
be used in combination with all previously mentioned content modifiers
like distance, offset, nocase, within, etc.
Example of http_stat_code in a HTTP response:
Example of the purpose of http_stat_code:
http_stat_msg¶
With the http_stat_msg content modifier, it is possible to match
specifically and only on the HTTP status message buffer. The keyword
can be used in combination with all previously mentioned content
modifiers like depth, distance, offset, nocase and
within.
Example of http_stat_msg in a HTTP response:
Example of the purpose of http_stat_msg:
http_response_line¶
The http_response_line forces the whole HTTP response line to be inspected.
Example:
alert http any any -> any any (http_response_line; content:"HTTP/1.0 200 OK"; sid:1;)
http_server_body¶
With the http_server_body content modifier, it is possible to
match specifically and only on the HTTP response body. The keyword can
be used in combination with all previously mentioned content modifiers
like distance, offset, nocase, within, etc.
Note: how much of the response/server body is inspected is controlled
in your libhtp configuration section via the response-body-limit
setting.
Notes¶
- Using
http_server_bodyis similar to having content matches that come afterfile_dataexcept that it doesn’t permanently (unless reset) set the detection pointer to the beginning of the server response body. i.e. it is not a sticky buffer. http_server_bodywill match on gzip decoded data just likefile_datadoes.- Since
http_server_bodymatches on a server response, it can’t be used with theto_serverorfrom_clientflow directives. - Corresponding PCRE modifier:
Q - further notes at the
file_datasection below.
http_host and http_raw_host¶
With the http_host content modifier, it is possible to
match specifically and only the normalized hostname.
The http_raw_host inspects the raw hostname.
The keyword can be used in combination with most of the content modifiers
like distance, offset, within, etc.
The nocase keyword is not allowed anymore. Keep in mind that you need
to specify a lowercase pattern.
Notes¶
The
http_hostandhttp_raw_hostbuffers are populated from either the URI (if the full URI is present in the request like in a proxy request) or the HTTP Host header. If both are present, the URI is used.The
http_hostandhttp_raw_hostbuffers will NOT include the header name, colon, or leading whitespace if populated from the Host header. i.e. they will not include “Host: “.The
http_hostandhttp_raw_hostbuffers do not include a CRLF (0x0D 0x0A) at the end. If you want to match the end of the buffer, use a relative ‘isdataat’ or a PCRE (although PCRE will be worse on performance).The
http_hostbuffer is normalized to be all lower case.The content match that
http_hostapplies to must be all lower case or have thenocaseflag set.http_raw_hostmatches the unnormalized buffer so matching will be case-sensitive (unlessnocaseis set).If a request contains multiple “Host” headers, the values will be concatenated in the
http_hostandhttp_raw_hostbuffers, in the order seen from top to bottom, with a comma and space (“, “) between each of them.Example request:
GET /test.html HTTP/1.1 Host: ABC.com Accept: */* Host: efg.net
http_hostbuffer contents:abc.com, efg.net
http_raw_hostbuffer contents:ABC.com, efg.net
Corresponding PCRE modifier (
http_host):WCorresponding PCRE modifier (
http_raw_host):Z
file_data¶
With file_data, the HTTP response body is inspected, just like
with http_server_body. The file_data keyword works a bit
differently from the normal content modifiers; when used in a rule,
all content matches following it in the rule are affected (modified)
by it.
Example:
alert http any any -> any any (file_data; content:"abc"; content:"xyz";)
The file_data keyword affects all following content matches, until
the pkt_data keyword is encountered or it reaches the end of the
rule. This makes it a useful shortcut for applying many content
matches to the HTTP response body, eliminating the need to modify each
content match individually.
As the body of a HTTP response can be very large, it is inspected in smaller chunks.
How much of the response/server body is inspected is controlled
in your libhtp configuration section via the response-body-limit
setting.
If the HTTP body is a flash file compressed with ‘deflate’ or ‘lzma’,
it can be decompressed and file_data can match on the decompress data.
Flash decompression must be enabled under libhtp configuration:
# Decompress SWF files.
# 2 types: 'deflate', 'lzma', 'both' will decompress deflate and lzma
# compress-depth:
# Specifies the maximum amount of data to decompress,
# set 0 for unlimited.
# decompress-depth:
# Specifies the maximum amount of decompressed data to obtain,
# set 0 for unlimited.
swf-decompression:
enabled: yes
type: both
compress-depth: 0
decompress-depth: 0
Notes¶
- If a HTTP body is using gzip or deflate,
file_datawill match on the decompressed data. - Negated matching is affected by the chunked inspection. E.g.
‘content:!”<html”;’ could not match on the first chunk, but would
then possibly match on the 2nd. To avoid this, use a depth setting.
The depth setting takes the body size into account.
Assuming that the
response-body-minimal-inspect-sizeis bigger than 1k, ‘content:!”<html”; depth:1024;’ can only match if the pattern ‘<html’ is absent from the first inspected chunk. file_datacan also be used with SMTP
File Keywords¶
Suricata comes with several rule keywords to match on various file properties. They depend on properly configured File Extraction.
filemagic¶
Matches on the information libmagic returns about a file.
Syntax:
filemagic:<string>;
Example:
filemagic:"executable for MS Windows";
Note: as libmagic versions differ between installations, the returned information may also slightly change. See also #437.
filestore¶
Stores files to disk if the signature matched.
Syntax:
filestore:<direction>,<scope>;
direction can be:
- request/to_server: store a file in the request / to_server direction
- response/to_client: store a file in the response / to_client direction
- both: store both directions
scope can be:
- file: only store the matching file (for filename,fileext,filemagic matches)
- tx: store all files from the matching HTTP transaction
- ssn/flow: store all files from the TCP session/flow.
If direction and scope are omitted, the direction will be the same as the rule and the scope will be per file.
filemd5¶
Match file MD5 against list of MD5 checksums.
Syntax:
filemd5:[!]filename;
The filename is expanded to include the rule dir. In the default case it will become /etc/suricata/rules/filename. Use the exclamation mark to get a negated match. This allows for white listing.
Examples:
filemd5:md5-blacklist;
filemd5:!md5-whitelist;
File format
The file format is simple. It’s a text file with a single md5 per line, at the start of the line, in hex notation. If there is extra info on the line it is ignored.
Output from md5sum is fine:
2f8d0355f0032c3e6311c6408d7c2dc2 util-path.c
b9cf5cf347a70e02fde975fc4e117760 util-pidfile.c
02aaa6c3f4dbae65f5889eeb8f2bbb8d util-pool.c
dd5fc1ee7f2f96b5f12d1a854007a818 util-print.c
Just MD5’s are good as well:
2f8d0355f0032c3e6311c6408d7c2dc2
b9cf5cf347a70e02fde975fc4e117760
02aaa6c3f4dbae65f5889eeb8f2bbb8d
dd5fc1ee7f2f96b5f12d1a854007a818
Memory requirements
Each MD5 uses 16 bytes of memory. 20 Million MD5’s use about 310 MiB of memory.
See also: https://blog.inliniac.net/2012/06/09/suricata-md5-blacklisting/
filesha1¶
Match file SHA1 against list of SHA1 checksums.
Syntax:
filesha1:[!]filename;
The filename is expanded to include the rule dir. In the default case it will become /etc/suricata/rules/filename. Use the exclamation mark to get a negated match. This allows for white listing.
Examples:
filesha1:sha1-blacklist;
filesha1:!sha1-whitelist;
File format
Same as md5 file format.
filesha256¶
Match file SHA256 against list of SHA256 checksums.
Syntax:
filesha256:[!]filename;
The filename is expanded to include the rule dir. In the default case it will become /etc/suricata/rules/filename. Use the exclamation mark to get a negated match. This allows for white listing.
Examples:
filesha256:sha256-blacklist;
filesha256:!sha256-whitelist;
File format
Same as md5 file format.
filesize¶
Match on the size of the file as it is being transferred.
Syntax:
filesize:<value>;
Possible units are KB, MB and GB, without any unit the default is bytes.
Examples:
filesize:100; # exactly 100 bytes
filesize:100<>200; # greater than 100 and smaller than 200
filesize:>100MB; # greater than 100 megabytes
filesize:<100MB; # smaller than 100 megabytes
Note: For files that are not completely tracked because of packet loss or stream.depth being reached on the “greater than” is checked. This is because Suricata can know a file is bigger than a value (it has seen some of it already), but it can’t know if the final size would have been within a range, an exact value or smaller than a value.
DNS Keywords¶
There are some more content modifiers (If you are unfamiliar with content modifiers, please visit the page Payload关键字 These ones make sure the signature checks a specific part of the network-traffic.
dns_query¶
With dns_query the DNS request queries are inspected. The dns_query keyword works a bit different from the normal content modifiers. When used in a rule all contents following it are affected by it. Example:
alert dns any any -> any any (msg:”Test dns_query option”; dns_query; content:”google”; nocase; sid:1;)
The dns_query keyword affects all following contents, until pkt_data is used or it reaches the end of the rule.
Normalized Buffer¶
Buffer contains literal domain name
- <length> values (as seen in a raw DNS request) are literal ‘.’ characters
- no leading <length> value
- No terminating NULL (0x00) byte (use a negated relative
isdataatto match the end)
Example DNS request for “mail.google.com” (for readability, hex values are encoded between pipes):
DNS query on the wire (snippet):
|04|mail|06|google|03|com|00|
dns_query buffer:
mail.google.com
SSL/TLS Keywords¶
Suricata comes with several rule keywords to match on various properties of TLS/SSL handshake. Matches are string inclusion matches.
tls_cert_subject¶
Match TLS/SSL certificate Subject field.
Examples:
tls_cert_subject; content:"CN=*.googleusercontent.com"; isdataat:!1,relative;
tls_cert_subject; content:"google.com"; nocase; pcre:"/google.com$/";
tls_cert_subject is a ‘Sticky buffer’.
tls_cert_subject can be used as fast_pattern.
tls_cert_issuer¶
Match TLS/SSL certificate Issuer field.
Examples:
tls_cert_issuer; content:"WoSign"; nocase; isdataat:!1,relative;
tls_cert_issuer; content:"StartCom"; nocase; pcre:"/StartCom$/";
tls_cert_issuer is a ‘Sticky buffer’.
tls_cert_issuer can be used as fast_pattern.
tls_cert_serial¶
Match on the serial number in a certificate.
Example:
alert tls any any -> any any (msg:"match cert serial"; \
tls_cert_serial; content:"5C:19:B7:B1:32:3B:1C:A1"; sid:200012;)
tls_cert_serial is a ‘Sticky buffer’.
tls_cert_serial can be used as fast_pattern.
tls_cert_fingerprint¶
Match on the SHA-1 fingerprint of the certificate.
Example:
alert tls any any -> any any (msg:"match cert fingerprint"; \
tls_cert_fingerprint; \
content:"4a:a3:66:76:82:cb:6b:23:bb:c3:58:47:23:a4:63:a7:78:a4:a1:18"; \
sid:200023;)
tls_cert_fingerprint is a ‘Sticky buffer’.
tls_cert_fingerprint can be used as fast_pattern.
tls_sni¶
Match TLS/SSL Server Name Indication field.
Examples:
tls_sni; content:"oisf.net"; nocase; isdataat:!1,relative;
tls_sni; content:"oisf.net"; nocase; pcre:"/oisf.net$/";
tls_sni is a ‘Sticky buffer’.
tls_sni can be used as fast_pattern.
tls_cert_notbefore¶
Match on the NotBefore field in a certificate.
Example:
alert tls any any -> any any (msg:"match cert NotBefore"; \
tls_cert_notbefore:1998-05-01<>2008-05-01; sid:200005;)
tls_cert_notafter¶
Match on the NotAfter field in a certificate.
Example:
alert tls any any -> any any (msg:"match cert NotAfter"; \
tls_cert_notafter:>2015; sid:200006;)
tls_cert_expired¶
Match returns true if certificate is expired. It evaluates the validity date from the certificate.
Usage:
tls_cert_expired;
tls_cert_valid¶
Match returns true if certificate is not expired. It only evaluates the
validity date. It does not do cert chain validation. It is the opposite
of tls_cert_expired.
Usage:
tls_cert_valid;
tls.subject¶
Match TLS/SSL certificate Subject field.
example:
tls.subject:"CN=*.googleusercontent.com"
Case sensitve, can’t use ‘nocase’.
Legacy keyword. tls_cert_subject is the replacement.
tls.issuerdn¶
match TLS/SSL certificate IssuerDN field
example:
tls.issuerdn:!"CN=Google-Internet-Authority"
Case sensitve, can’t use ‘nocase’.
Legacy keyword. tls_cert_issuer is the replacement.
tls.fingerprint¶
match TLS/SSL certificate SHA1 fingerprint
example:
tls.fingerprint:!"f3:40:21:48:70:2c:31:bc:b5:aa:22:ad:63:d6:bc:2e:b3:46:e2:5a"
Case sensitive, can’t use ‘nocase’.
The tls.fingerprint buffer is lower case so you must use lower case letters for this to match.
tls.store¶
store TLS/SSL certificate on disk
ssl_state¶
The ssl_state keyword matches the state of the SSL connection. The possible states
are client_hello, server_hello, client_keyx, server_keyx and unknown.
You can specify several states with | (OR) to check for any of the specified states.
Negation support is not available yet, see https://redmine.openinfosecfoundation.org/issues/1231
JA3 Keywords¶
Suricata comes with a JA3 integration (https://github.com/salesforce/ja3). JA3 is used to fingerprint TLS clients.
JA3 must be enabled in the Suricata config file (set ‘app-layer.protocols.tls.ja3-fingerprints’ to ‘yes’).
ja3_hash¶
Match on JA3 hash (md5).
Example:
alert tls any any -> any any (msg:"match JA3 hash"; \
ja3_hash; content:"e7eca2baf4458d095b7f45da28c16c34"; \
sid:100001;)
ja3_hash is a ‘Sticky buffer’.
ja3_hash can be used as fast_pattern.
ja3_string¶
Match on JA3 string.
Example:
alert tls any any -> any any (msg:"match JA3 string"; \
ja3_string; content:"19-20-21-22"; \
sid:100002;)
ja3_string is a ‘Sticky buffer’.
ja3_string can be used as fast_pattern.
Modbus Keyword¶
The modbus keyword can be used for matching on various properties of Modbus requests.
There are three ways of using this keyword:
- matching on functions properties with the setting “function”;
- matching on directly on data access with the setting “access”;
- matching on unit identifier with the setting “unit” only or with the previous setting “function” or “access”.
With the setting function, you can match on:
- an action based on a function code field and a sub-function code when applicable;
- one of three categories of Modbus functions;
- public functions that are publicly defined (setting “public”)
- user-defined functions (setting “user”)
- reserved functions that are dedicated to proprietary extensions of Modbus (keyword “reserved”)
- one of the two sub-groups of public functions:
- assigned functions whose definition is already given in the Modbus specification (keyword “assigned”);
- unassigned functions, which are reserved for future use (keyword “unassigned”).
Syntax:
modbus: function <value>
modbus: function <value>, subfunction <value>
modbus: function [!] <assigned | unassigned | public | user | reserved | all>
Sign ‘!’ is negation
Examples:
modbus: function 21 # Write File record function
modbus: function 4, subfunction 4 # Force Listen Only Mode (Diagnostics) function
modbus: function assigned # defined by Modbus Application Protocol Specification V1.1b3
modbus: function public # validated by the Modbus.org community
modbus: function user # internal use and not supported by the specification
modbus: function reserved # used by some companies for legacy products and not available for public use
modbus: function !reserved # every function but reserved function
With the access setting, you can match on:
- a type of data access (read or write);
- one of primary tables access (Discretes Input, Coils, Input Registers and Holding Registers);
- a range of addresses access;
- a written value.
Syntax:
modbus: access <read | write>
modbus: access read <discretes | coils | input | holding>
modbus: access read <discretes | coils | input | holding>, address <value>
modbus: access write < coils | holding>
modbus: access write < coils | holding>, address <value>
modbus: access write < coils | holding>, address <value>, value <value>
With _<value>_ setting matches on the address or value as it is being accessed or written as follows:
address 100 # exactly address 100
address 100<>200 # greater than address 100 and smaller than address 200
address >100 # greater than address 100
address <100 # smaller than address 100
Examples:
modbus: access read # Read access
modbus: access write # Write access
modbus: access read input # Read access to Discretes Input table
modbus: access write coils # Write access to Coils table
modbus: access read discretes, address <100 # Read access at address smaller than 100 of Discretes Input table
modbus: access write holding, address 500, value >200 # Write value greater than 200 at address 500 of Holding Registers table
With the setting unit, you can match on:
- a MODBUS slave address of a remote device connected on the sub-network behind a bridge or a gateway. The destination IP address identifies the bridge itself and the bridge uses the MODBUS unit identifier to forward the request to the right slave device.
Syntax:
modbus: unit <value>
modbus: unit <value>, function <value>
modbus: unit <value>, function <value>, subfunction <value>
modbus: unit <value>, function [!] <assigned | unassigned | public | user | reserved | all>
modbus: unit <value>, access <read | write>
modbus: unit <value>, access read <discretes | coils | input | holding>
modbus: unit <value>, access read <discretes | coils | input | holding>, address <value>
modbus: unit <value>, access write < coils | holding>
modbus: unit <value>, access write < coils | holding>, address <value>
modbus: unit <value>, access write < coils | holding>, address <value>, value <value>
With _<value>_ setting matches on the address or value as it is being accessed or written as follows:
unit 10 # exactly unit identifier 10
unit 10<>20 # greater than unit identifier 10 and smaller than unit identifier 20
unit >10 # greater than unit identifier 10
unit <10 # smaller than unit identifier 10
Examples:
modbus: unit 10 # Unit identifier 10
modbus: unit 10, function 21 # Unit identifier 10 and write File record function
modbus: unit 10, function 4, subfunction 4 # Unit identifier 10 and force Listen Only Mode (Diagnostics) function
modbus: unit 10, function assigned # Unit identifier 10 and assigned function
modbus: unit 10, function !reserved # Unit identifier 10 and every function but reserved function
modbus: unit 10, access read # Unit identifier 10 and Read access
modbus: unit 10, access write coils # Unit identifier 10 and Write access to Coils table
modbus: unit >10, access read discretes, address <100 # Greater than unit identifier 10 and Read access at address smaller than 100 of Discretes Input table
modbus: unit 10<>20, access write holding, address 500, value >200 # Greater than unit identifier 10 and smaller than unit identifier 20 and Write value greater than 200 at address 500 of Holding Registers table
(cf. http://www.modbus.org/docs/Modbus_Application_Protocol_V1_1b3.pdf)
Note: Address of read and write are starting at 1. So if your system is using a start at 0, you need to add 1 the address values.
Note: According to MODBUS Messaging on TCP/IP Implementation Guide V1.0b, it is recommended to keep the TCP connection opened with a remote device and not to open and close it for each MODBUS/TCP transaction. In that case, it is important to set the depth of the stream reassembling as unlimited (stream.reassembly.depth: 0)
Note: According to MODBUS Messaging on TCP/IP Implementation Guide V1.0b, the MODBUS slave device addresses on serial line are assigned from 1 to 247 (decimal). Address 0 is used as broadcast address.
(cf. http://www.modbus.org/docs/Modbus_Messaging_Implementation_Guide_V1_0b.pdf)
Paper and presentation (in french) on Modbus support are available : http://www.ssi.gouv.fr/agence/publication/detection-dintrusion-dans-les-systemes-industriels-suricata-et-le-cas-modbus/
DNP3 Keywords¶
The DNP3 keywords can be used to match on fields in decoded DNP3 messages. The keywords are based on Snort’s DNP3 keywords and aim to be 100% compatible.
dnp3_func¶
This keyword will match on the application function code found in DNP3 request and responses. It can be specified as the integer value or the symbolic name of the function code.
Syntax¶
dnp3_func:<value>;
Where value is one of:
- An integer value between 0 and 255 inclusive.
- Function code name:
- confirm
- read
- write
- select
- operate
- direct_operate
- direct_operate_nr
- immed_freeze
- immed_freeze_nr
- freeze_clear
- freeze_clear_nr
- freeze_at_time
- freeze_at_time_nr
- cold_restart
- warm_restart
- initialize_data
- initialize_appl
- start_appl
- stop_appl
- save_config
- enable_unsolicited
- disable_unsolicited
- assign_class
- delay_measure
- record_current_time
- open_file
- close_file
- delete_file
- get_file_info
- authenticate_file
- abort_file
- activate_config
- authenticate_req
- authenticate_err
- response
- unsolicited_response
- authenticate_resp
dnp3_ind¶
This keyword matches on the DNP3 internal indicator flags in the response application header.
Syntax¶
dnp3_ind:<flag>{,<flag>...}
Where flag is the name of the internal indicator:
- all_stations
- class_1_events
- class_2_events
- class_3_events
- need_time
- local_control
- device_trouble
- device_restart
- no_func_code_support
- object_unknown
- parameter_error
- event_buffer_overflow
- already_executing
- config_corrupt
- reserved_2
- reserved_1
This keyword will match of any of the flags listed are set. To match on multiple flags (AND type match), use dnp3_ind for each flag that must be set.
dnp3_obj¶
This keyword matches on the DNP3 application data objects.
Syntax¶
dnp3_obj:<group>,<variation>
Where <group> and <variation> are integer values between 0 and 255 inclusive.
dnp3_data¶
This keyword will cause the following content options to match on the re-assembled application buffer. The reassembled application buffer is a DNP3 fragment with CRCs removed (which occur every 16 bytes), and will be the complete fragment, possibly reassembled from multiple DNP3 link layer frames.
Syntax¶
dnp3_data;
Example¶
dnp3_data; content:|c3 06|;
ENIP/CIP Keywords¶
The enip_command and cip_service keywords can be used for matching on various properties of ENIP requests.
There are three ways of using this keyword:
- matching on ENIP command with the setting “enip_command”;
- matching on CIP Service with the setting “cip_service”.
- matching both the ENIP command and the CIP Service with “enip_command” and “cip_service” together
For the ENIP command, we are matching against the command field found in the ENIP encapsulation.
For the CIP Service, we use a maximum of 3 comma separated values representing the Service, Class and Attribute. These values are described in the CIP specification. CIP Classes are associated with their Service, and CIP Attributes are associated with their Service. If you only need to match up until the Service, then only provide the Service value. If you want to match to the CIP Attribute, then you must provide all 3 values.
Syntax:
enip_command:<value>
cip_service:<value(s)>
enip_command:<value>, cip_service:<value(s)>
Examples:
enip_command:99
cip_service:75
cip_service:16,246,6
enip_command:111, cip_service:5
(cf. http://read.pudn.com/downloads166/ebook/763211/EIP-CIP-V1-1.0.pdf)
Information on the protocol can be found here: http://literature.rockwellautomation.com/idc/groups/literature/documents/wp/enet-wp001_-en-p.pdf
FTP/FTP-DATA Keywords¶
ftpdata_command¶
Filter ftp-data channel based on command used on the FTP command channel. Currently supported commands are RETR (get on a file) and STOR (put on a file).
Syntax:
ftpdata_command:(retr|stor)
Examples:
ftpdata_command:retr
ftpdata_command:stor
Signature example:
alert ftp-data any any -> any any (msg:"FTP store password"; filestore; filename:"password"; ftpdata_command:stor; sid:3; rev:1;)
Kerberos Keywords¶
krb5_msg_type¶
Kerberos message type (integer).
Values are defined in RFC4120. Common values are
- 10 (AS-REQ)
- 11 (AS-REP)
- 12 (TGS-REQ)
- 13 (TGS-REP)
- 14 (AP-REQ)
- 15 (AP-REP)
- 30 (ERROR)
Syntax:
krb5_msg_type:<number>
Signature example:
alert krb5 any any -> any any (msg:"Kerberos 5 AS-REQ message"; krb5_msg_type:10; sid:3; rev:1;)
krb5_cname¶
Kerberos client name, provided in the ticket (for AS-REQ and TGS-REQ messages).
If the client name from the Kerberos message is composed of several parts, the name is compared to each part and the match will succeed if any is identical.
Comparison is case-sensitive.
Syntax:
krb5_cname; content:"name";
Signature example:
alert krb5 any any -> any any (msg:"Kerberos 5 des server name"; krb5_cname; content:"des"; sid:4; rev:1;)
krb5_cname is a ‘sticky buffer’.
krb5_cname can be used as fast_pattern.
krb5_sname¶
Kerberos server name, provided in the ticket (for AS-REQ and TGS-REQ messages) or in the error message.
If the server name from the Kerberos message is composed of several parts, the name is compared to each part and the match will succeed if any is identical.
Comparison is case-sensitive.
Syntax:
krb5_sname; content:"name";
Signature example:
alert krb5 any any -> any any (msg:"Kerberos 5 krbtgt server name"; krb5_sname; content:"krbtgt"; sid:5; rev:1;)
krb5_sname is a ‘sticky buffer’.
krb5_sname can be used as fast_pattern.
krb5_err_code¶
Kerberos error code (integer). This field is matched in Kerberos error messages only.
For a list of error codes, refer to RFC4120 section 7.5.9.
Syntax:
krb5_err_code:<number>
Signature example:
alert krb5 any any -> any any (msg:"Kerberos 5 error C_PRINCIPAL_UNKNOWN"; krb5_err_code:6; sid:6; rev:1;)
krb5.weak_encryption (event)¶
Event raised if the encryption parameters selected by the server are weak or deprecated. For example, using a key size smaller than 128, or using deprecated ciphers like DES.
Syntax:
app-layer-event:krb5.weak_encryption
Signature example:
alert krb5 any any -> any any (msg:"SURICATA Kerberos 5 weak encryption parameters"; flow:to_client; app-layer-event:krb5.weak_encryption; classtype:protocol-command-decode; sid:2226001; rev:1;)
krb5.malformed_data (event)¶
Event raised in case of a protocol decoding error.
Syntax:
app-layer-event:krb5.malformed_data
Signature example:
alert krb5 any any -> any any (msg:"SURICATA Kerberos 5 malformed request data"; flow:to_server; app-layer-event:krb5.malformed_data; classtype:protocol-command-decode; sid:2226000; rev:1;)
Generic App Layer Keywords¶
app-layer-protocol¶
Match on the detected app-layer protocol.
Syntax:
app-layer-protocol:[!]<protocol>;
Examples:
app-layer-protocol:ssh;
app-layer-protocol:!tls;
app-layer-protocol:failed;
A special value ‘failed’ can be used for matching on flows in which protocol detection failed. This can happen if Suricata doesn’t know the protocol or when certain ‘bail out’ conditions happen.
Bail out conditions¶
Protocol detection gives up in several cases:
- both sides are inspected and no match was found
- side A detection failed, side B has no traffic at all (e.g. FTP data channel)
- side A detection failed, side B has so little data detection is inconclusive
In these last 2 cases the app-layer-event:applayer_proto_detection_skipped
is set.
app-layer-event¶
Match on events generated by the App Layer Parsers and the protocol detection engine.
Syntax:
app-layer-event:<event name>;
Examples:
app-layer-event:applayer_mismatch_protocol_both_directions;
app-layer-event:http.gzip_decompression_failed;
Protocol Detection¶
applayer_mismatch_protocol_both_directions¶
The toserver and toclient directions have different protocols. For example a client talking HTTP to a SSH server.
applayer_wrong_direction_first_data¶
Some protocol implementations in Suricata have a requirement with regards to the first data direction. The HTTP parser is an example of this.
applayer_detect_protocol_only_one_direction¶
Protocol detection only succeeded in one direction. For FTP and SMTP this is expected.
applayer_proto_detection_skipped¶
Protocol detection was skipped because of Bail out conditions.
Xbits Keyword¶
Set, unset, toggle and check for bits stored per host or ip_pair.
Syntax:
xbits:noalert;
xbits:<set|unset|isset|toggle>,<name>,track <ip_src|ip_dst|ip_pair>;
xbits:<set|unset|isset|toggle>,<name>,track <ip_src|ip_dst|ip_pair> \
[,expire <seconds>];
xbits:<set|unset|isset|toggle>,<name>,track <ip_src|ip_dst|ip_pair> \
[,expire <seconds>];
Notes¶
- No difference between using
hostbitsandxbitswithtrack ip_<src|dst> - If you
seton a client request and usetrack ip_dst, if you want to match on the server response, you check it (isset) withtrack ip_src. - To not alert, use
noalert; - See also:
YAML settings¶
Bits that are stored per host are stored in the Host table. This means that host table settings affect hostsbits and xbits per host.
Bits that are stored per IP pair are stored in the IPPair table. This means that ippair table settings, especially memcap, affect xbits per ip_pair.
Threading¶
Due to subtle timing issues between threads the order of sets and checks can be slightly unpredictable.
Unix Socket¶
Hostbits can be added, removed and listed through the unix socket.
Add:
suricatasc -c "add-hostbit <ip> <bit name> <expire in seconds>"
suricatasc -c "add-hostbit 1.2.3.4 blacklist 3600"
If an hostbit is added for an existing hostbit, it’s expiry timer is updated.
Remove:
suricatasc -c "remove-hostbit <ip> <bit name>"
suricatasc -c "remove-hostbit 1.2.3.4 blacklist"
List:
suricatasc -c "list-hostbit <ip>"
suricatasc -c "list-hostbit 1.2.3.4"
This results in:
{
"message":
{
"count": 1,
"hostbits":
[{
"expire": 89,
"name": "blacklist"
}]
},
"return": "OK"
}
Examples¶
Creating a SSH blacklist¶
Below is an example of rules incoming to a SSH server.
The first 2 rules match on a SSH software version often used in bots. They drop the traffic and create an ‘xbit’ ‘badssh’ for the source ip. It expires in an hour:
drop ssh any any -> $MYSERVER 22 (msg:"DROP libssh incoming"; \
flow:to_server,established; ssh.softwareversion:"libssh"; \
xbits:set, badssh, track ip_src, expire 3600; sid:4000000005;)
drop ssh any any -> $MYSERVER 22 (msg:"DROP PUTTY incoming"; \
flow:to_server,established; ssh.softwareversion:"PUTTY"; \
xbits:set, badssh, track ip_src, expire 3600; sid:4000000007;)
Then the following rule simply drops any incoming traffic to that server that is on that ‘badssh’ list:
drop ssh any any -> $MYSERVER 22 (msg:"DROP BLACKLISTED"; \
xbits:isset, badssh, track ip_src; sid:4000000006;)
Thresholding Keywords¶
Thresholding can be configured per rule and also globally, see Global-Thresholds.
Note: mixing rule and global thresholds is not supported in 1.3 and before. See bug #425. For the state of the support in 1.4 see Global thresholds vs rule thresholds
threshold¶
The threshold keyword can be used to control the rule’s alert frequency. It has 3 modes: threshold, limit and both.
Syntax:
threshold: type <threshold|limit|both>, track <by_src|by_dst>, count <N>, seconds <T>
type “threshold”¶
This type can be used to set a minimum threshold for a rule before it generates alerts. A threshold setting of N means on the Nth time the rule matches an alert is generated.
Example:
alert tcp !$HOME_NET any -> $HOME_NET 25 (msg:"ET POLICY Inbound Frequent Emails - Possible Spambot Inbound"; \
flow:established; content:"mail from|3a|"; nocase; \
threshold: type threshold, track by_src, count 10, seconds 60; \
reference:url,doc.emergingthreats.net/2002087; classtype:misc-activity; sid:2002087; rev:10;)
This signature only generates an alert if we get 10 inbound emails or more from the same server in a time period of one minute.
If a signature sets a flowbit, flowint, etc. those actions are still performed for each of the matches.
Rule actions drop (IPS mode) and reject are applied to each packet (not only the one that meets the threshold condition).
type “limit”¶
This type can be used to make sure you’re not getting flooded with alerts. If set to limit N, it alerts at most N times.
Example:
alert http $HOME_NET any -> any $HTTP_PORTS (msg:"ET USER_AGENTS Internet Explorer 6 in use - Significant Security Risk"; \
flow:to_server,established; content:"|0d 0a|User-Agent|3a| Mozilla/4.0 (compatible|3b| MSIE 6.0|3b|"; \
threshold: type limit, track by_src, seconds 180, count 1; \
reference:url,doc.emergingthreats.net/2010706; classtype:policy-violation; sid:2010706; rev:7;)
In this example at most 1 alert is generated per host within a period of 3 minutes if MSIE 6.0 is detected.
If a signature sets a flowbit, flowint, etc. those actions are still performed for each of the matches.
Rule actions drop (IPS mode) and reject are applied to each packet (not only the one that meets the limit condition).
type “both”¶
This type is a combination of the “threshold” and “limit” types. It applies both thresholding and limiting.
Example:
alert tcp $HOME_NET 5060 -> $EXTERNAL_NET any (msg:"ET VOIP Multiple Unauthorized SIP Responses TCP"; \
flow:established,from_server; content:"SIP/2.0 401 Unauthorized"; depth:24; \
threshold: type both, track by_src, count 5, seconds 360; \
reference:url,doc.emergingthreats.net/2003194; classtype:attempted-dos; sid:2003194; rev:6;)
This alert will only generate an alert if within 6 minutes there have been 5 or more “SIP/2.0 401 Unauthorized” responses, and it will alert only once in that 6 minutes.
If a signature sets a flowbit, flowint, etc. those actions are still performed for each of the matches.
Rule actions drop (IPS mode) and reject are applied to each packet.
detection_filter¶
The detection_filter keyword can be used to alert on every match after a threshold has been reached. It differs from the threshold with type threshold in that it generates an alert for each rule match after the initial threshold has been reached, where the latter will reset it’s internal counter and alert again when the threshold has been reached again.
Syntax:
detection_filter: track <by_src|by_dst>, count <N>, seconds <T>
Example:
alert http $EXTERNAL_NET any -> $HOME_NET any \
(msg:"ET WEB_SERVER WebResource.axd access without t (time) parameter - possible ASP padding-oracle exploit"; \
flow:established,to_server; content:"GET"; http_method; content:"WebResource.axd"; http_uri; nocase; \
content:!"&t="; http_uri; nocase; content:!"&|3b|t="; http_uri; nocase; \
detection_filter:track by_src,count 15,seconds 2; \
reference:url,netifera.com/research/; reference:url,www.microsoft.com/technet/security/advisory/2416728.mspx; \
classtype:web-application-attack; sid:2011807; rev:5;)
Alerts each time after 15 or more matches have occurred within 2 seconds.
If a signature sets a flowbit, flowint, etc. those actions are still performed for each of the matches.
Rule actions drop (IPS mode) and reject are applied to each packet that generate an alert
IP Reputation Keyword¶
IP Reputation can be used in rules through a new rule keyword “iprep”.
For more information about IP Reputation see IP Reputation Config and IP Reputation Format.
iprep¶
The iprep directive matches on the IP reputation information for a host.
iprep:<side to check>,<category>,<operator>,<reputation score>
side to check: <any|src|dst|both>
category: the category short name
operator: <, >, =
reputation score: 1-127
Example:
alert ip $HOME_NET any -> any any (msg:"IPREP internal host talking to CnC server"; flow:to_server; iprep:dst,CnC,>,30; sid:1; rev:1;)
This rule will alert when a system in $HOME_NET acts as a client while communicating with any IP in the CnC category that has a reputation score set to greater than 30.
IP-only¶
The “iprep” keyword is compatible to “IP-only” rules. This means that a rule like:
alert ip any any -> any any (msg:"IPREP High Value CnC"; iprep:src,CnC,>,100; sid:1; rev:1;)
will only be checked once per flow-direction.
Lua Scripting¶
Syntax:
lua:[!]<scriptfilename>;
The script filename will be appended to your default rules location.
The script has 2 parts, an init function and a match function. First, the init.
Init function¶
function init (args)
local needs = {}
needs["http.request_line"] = tostring(true)
return needs
end
The init function registers the buffer(s) that need inspection. Currently the following are available:
- packet – entire packet, including headers
- payload – packet payload (not stream)
- http.uri
- http.uri.raw
- http.request_line
- http.request_headers
- http.request_headers.raw
- http.request_cookie
- http.request_user_agent
- http.request_body
- http.response_headers
- http.response_headers.raw
- http.response_body
- http.response_cookie
All the HTTP buffers have a limitation: only one can be inspected by a script at a time.
Match function¶
function match(args)
a = tostring(args["http.request_line"])
if #a > 0 then
if a:find("^POST%s+/.*%.php%s+HTTP/1.0$") then
return 1
end
end
return 0
end
The script can return 1 or 0. It should return 1 if the condition(s) it checks for match, 0 if not.
Entire script:
function init (args)
local needs = {}
needs["http.request_line"] = tostring(true)
return needs
end
function match(args)
a = tostring(args["http.request_line"])
if #a > 0 then
if a:find("^POST%s+/.*%.php%s+HTTP/1.0$") then
return 1
end
end
return 0
end
return 0
Differences From Snort¶
This document is intended to highlight the major differences between Suricata and Snort that apply to rules and rule writing.
Where not specified, the statements below apply to Suricata. In general, references to Snort refer to the version 2.9 branch.
Automatic Protocol Detection¶
Suricata does automatic protocol detection of the following application layer protocols:
- dcerpc
- dnp3
- dns
- http
- imap (detection only by default; no parsing)
- ftp
- modbus (disabled by default; minimalist probe parser; can lead to false positives)
- msn (detection only by default; no parsing)
- smb
- smb2 (disabled internally inside the engine)
- smtp
- ssh
- tls (SSLv2, SSLv3, TLSv1, TLSv1.1 and TLSv1.2)
In Suricata, protocol detection is port agnostic (in most cases). In Snort, in order for the
http_inspectand other preprocessors to be applied to traffic, it has to be over a configured port.- Some configurations for app-layer in the Suricata yaml can/do by default specify specific destination ports (e.g. DNS)
- You can look on ‘any’ port without worrying about the performance impact that you would have to be concerned about with Snort.
If the traffic is detected as HTTP by Suricata, the
http_*buffers are populated and can be used, regardless of port(s) specified in the rule.You don’t have to check for the http protocol (i.e.
alert http ...) to use thehttp_*buffers although it is recommended.If you are trying to detect legitimate (supported) application layer protocol traffic and don’t want to look on specific port(s), the rule should be written as
alert <protocol> ...withanyin place of the usual protocol port(s). For example, when you want to detect HTTP traffic and don’t want to limit detection to a particular port or list of ports, the rules should be written asalert http ...withanyin place of$HTTP_PORTS.- You can also use
app-layer-protocol:<protocol>;inside the rule instead.
So, instead of this Snort rule:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS ...
Do this for Suricata:
alert http $HOME_NET -> $EXTERNAL_NET any ...
Or:
alert tcp $HOME_NET any -> $EXTERNAL_NET any (app-layer-protocol:http; ...
- You can also use
urilen Keyword¶
Ranges given in the
urilenkeyword are inclusive for Snort but not inclusive for Suricata.Example:
urilen:2<>10- Snort interprets this as, “the URI length must be greater than or equal to 2, and less than or equal to 10”.
- Suricata interprets this as “the URI length must be greater than 2 and less than 10”.
- There is a request to have Suricata behave like Snort in future
versions –
https://redmine.openinfosecfoundation.org/issues/1416
- Currently on hold
By default, with Suricata,
urilenapplies to the normalized buffer- Use
,rawfor raw buffer - e.g.
urilen:>20,raw;
- Use
By default, with Snort,
urilenapplies to the raw buffer- Use
,normfor normalized buffer - e.g.
urilen:>20,norm;
- Use
http_uri Buffer¶
- In Snort, the
http_uribuffer normalizes ‘+’ characters (0x2B) to spaces (0x20).- Suricata can do this as well but you have to explicitly
set
query-plusspace-decode: yesin thelibhtpsection of Suricata’s yaml file.
- Suricata can do this as well but you have to explicitly
set
- https://redmine.openinfosecfoundation.org/issues/1035
- https://github.com/inliniac/suricata/pull/620
http_header Buffer¶
In Snort, the
http_headerbuffer includes the CRLF CRLF (0x0D 0x0A 0x0D 0x0A) that separates the end of the last HTTP header from the beginning of the HTTP body. Suricata includes a CRLF after the last header in thehttp_headerbuffer but not an extra one like Snort does. If you want to match the end of the buffer, use either thehttp_raw_headerbuffer, a relativeisdataat(e.g.isdataat:!1,relative) or a PCRE (although PCRE will be worse on performance).Suricata will include CRLF CRLF at the end of the
http_raw_headerbuffer like Snort does.Snort will include a leading CRLF in the
http_headerbuffer of server responses (but not client requests). Suricata does not have the leading CRLF in thehttp_headerbuffer of the server response or client request.In the
http_headerbuffer, Suricata will normalize HTTP header lines such that there is a single space (0x20) after the colon (‘:’) that separates the header name from the header value; this single space replaces zero or more whitespace characters (including tabs) that may be present in the raw HTTP header line immediately after the colon. If the extra whitespace (or lack thereof) is important for matching, use thehttp_raw_headerbuffer instead of thehttp_headerbuffer.Snort will also normalize superfluous whitespace between the header name and header value like Suricata does but only if there is at least one space character (0x20 only so not 0x90) immediately after the colon. This means that, unlike Suricata, if there is no space (or if there is a tab) immediately after the colon before the header value, the content of the header line will remain unchanged in the
http_headerbuffer.When there are duplicate HTTP headers (referring to the header name only, not the value), the normalized buffer (
http_header) will concatenate the values in the order seen (from top to bottom), with a comma and space (“, “) between each of them. If this hinders detection, use thehttp_raw_headerbuffer instead.Example request:
GET /test.html HTTP/1.1 Content-Length: 44 Accept: */* Content-Length: 55
The Content-Length header line becomes this in the
http_headerbuffer:Content-Length: 44, 55
The HTTP ‘Cookie’ and ‘Set-Cookie’ headers are NOT included in the
http_headerbuffer; instead they are extracted and put into their own buffer –http_cookie. See the http_cookie Buffer section.The HTTP ‘Cookie’ and ‘Set-Cookie’ headers ARE included in the
http_raw_headerbuffer so if you are trying to match on something like particular header ordering involving (or not involving) the HTTP Cookie headers, use thehttp_raw_headerbuffer.If ‘enable_cookie’ is set for Snort, the HTTP Cookie header names and trailing CRLF (i.e. “Cookie: \r\n” and “Set-Cooke \r\n”) are kept in the
http_headerbuffer. This is not the case for Suricata which removes the entire “Cookie” or “Set-Cookie” line from thehttp_headerbuffer.Other HTTP headers that have their own buffer (
http_user_agent,http_host) are not removed from thehttp_headerbuffer like the Cookie headers are.When inspecting server responses and
file_datais used, content matches inhttp_*buffers should come beforefile_dataunless you usepkt_datato reset the cursor before matching inhttp_*buffers. Snort will not complain if you usehttp_*buffers afterfile_datais set.
http_cookie Buffer¶
The
http_cookiebuffer will NOT include the header name, colon, or leading whitespace. i.e. it will not include “Cookie: ” or “Set-Cookie: “.The
http_cookiebuffer does not include a CRLF (0x0D 0x0A) at the end. If you want to match the end of the buffer, use a relativeisdataator a PCRE (although PCRE will be worse on performance).There is no
http_raw_cookiebuffer in Suricata. Usehttp_raw_headerinstead.You do not have to configure anything special to use the ‘http_cookie’ buffer in Suricata. This is different from Snort where you have to set
enable_cookiein thehttp_inspect_serverpreprocessor config in order to have thehttp_cookiebuffer treated separate from thehttp_headerbuffer.If Snort has ‘enable_cookie’ set and multiple “Cookie” or “Set-Cookie” headers are seen, it will concatenate them together (with no separator between them) in the order seen from top to bottom.
If a request contains multiple “Cookie” or “Set-Cookie” headers, the values will be concatenated in the Suricata
http_cookiebuffer, in the order seen from top to bottom, with a comma and space (“, “) between each of them.Example request:
GET /test.html HTTP/1.1 Cookie: monster Accept: */* Cookie: elmo
Suricata
http_cookiebuffer contents:monster, elmo
Snort
http_cookiebuffer contents:monsterelmoCorresponding PCRE modifier:
C(same as Snort)
New HTTP keywords¶
Suricata supports several HTTP keywords that Snort doesn’t have.
Examples are http_user_agent, http_host and http_content_type.
See HTTP关键字 for all HTTP keywords.
byte_extract Keyword¶
- Suricata supports
byte_extractfromhttp_*buffers, includinghttp_headerwhich does not always work as expected in Snort. - In Suricata, variables extracted using
byte_extractmust be used in the same buffer, otherwise they will have the value “0” (zero). Snort does allow cross-buffer byte extraction and usage. - Be sure to always positively and negatively test Suricata rules that
use
byte_extractandbyte_testto verify that they work as expected.
isdataat Keyword¶
The
rawbyteskeyword is supported in the Suricata syntax but doesn’t actually do anything.Absolute
isdataatchecks will succeed if the offset used is less than the size of the inspection buffer. This is true for Suricata and Snort.For relative
isdataatchecks, there is a 1 byte difference in the way Snort and Suricata do the comparisons.- Suricata will succeed if the relative offset is less than or
equal to the size of the inspection buffer. This is different
from absolute
isdataatchecks. - Snort will succeed if the relative offset is less than the
size of the inspection buffer, just like absolute
isdataatchecks. - Example - to check that there is no data in the inspection buffer
after the last content match:
- Snort:
isdataat:!0,relative; - Suricata:
isdataat:!1,relative;
- Snort:
- Suricata will succeed if the relative offset is less than or
equal to the size of the inspection buffer. This is different
from absolute
With Snort, the “inspection buffer” used when checking an
isdataatkeyword is generally the packet/segment with some exceptions:- With PAF enabled the PDU is examined instead of the
packet/segment. When
file_dataorbase64_datahas been set, it is those buffers (unlessrawbytesis set). - With some preprocessors - modbus, gtp, sip, dce2, and dnp3 - the
buffer can be particular portions of those protocols (unless
rawbytesis set). - With some preprocessors - rpc_decode, ftp_telnet, smtp, and dnp3
- the buffer can be particular decoded portions of those
protocols (unless
rawbytesis set).
- With PAF enabled the PDU is examined instead of the
packet/segment. When
With Suricata, the “inspection buffer” used when checking an absolute
isdataatkeyword is the packet/segment if looking at a packet (e.g.alert tcp-pkt...) or the reassembled stream segments.In Suricata, a relative
isdataatkeyword will apply to the buffer of the previous content match. So if the previous content match is ahttp_*buffer, the relativeisdataatapplies to that buffer, starting from the end of the previous content match in that buffer. Snort does not behave like this!For example, this Suricata rule looks for the string “.exe” at the end of the URI; to do the same thing in the normalized URI buffer in Snort you would have to use a PCRE –
pcre:"/\x2Eexe$/U";alert http $HOME_NET any -> $EXTERNAL_NET any (msg:".EXE File Download Request"; flow:established,to_server; content:"GET"; http_method; content:".exe"; http_uri; isdataat:!1,relative; priority:3; sid:18332111;)
If you are unclear about behavior in a particular instance, you are encouraged to positively and negatively test your rules that use an
isdataatkeyword.
Relative PCRE¶
You can do relative PCRE matches in normalized/special buffers with Suricata. Example:
content:".php?sign="; http_uri; pcre:"/^[a-zA-Z0-9]{8}$/UR";
With Snort you can’t combine the “relative” PCRE option (‘R’) with other buffer options like normalized URI (‘U’) – you get a syntax error.
tls* Keywords¶
In addition to TLS protocol identification, Suricata supports the storing of certificates to disk, verifying the validity dates on certificates, matching against the calculated SHA1 fingerprint of certificates, and matching on certain TLS/SSL certificate fields including the following:
- Negotiated TLS/SSL version.
- Certificate Subject field.
- Certificate Issuer field.
- Certificate SNI Field
For details see SSL/TLS Keywords.
dns_query Keyword¶
- Sets the detection pointer to the DNS query.
- Works like
file_datadoes (“sticky buffer”) but for a DNS request query. - Use
pkt_datato reset the detection pointer to the beginning of the packet payload. - See DNS Keywords for details.
IP Reputation and iprep Keyword¶
- Snort has the “reputation” preprocessor that can be used to define whitelist and blacklist files of IPs which are used generate GID 136 alerts as well as block/drop/pass traffic from listed IPs depending on how it is configured.
- Suricata also has the concept of files with IPs in them but provides
the ability to assign them:
- Categories
- Reputation score
- Suricata rules can leverage these IP lists with the
iprepkeyword that can be configured to match on:- Direction
- Category
- Value (reputation score)
- Reputation
- IP Reputation Config
- IP Reputation Keyword
- IP Reputation Format
- https://blog.inliniac.net/2012/11/21/ip-reputation-in-suricata/
Flowbits¶
- Suricata fully supports the setting and checking of flowbits (including the same flowbit) on the same packet/stream. Snort does not always allow for this.
- In Suricata,
flowbits:issetis checked after the fast pattern match but before othercontentmatches. In Snort,flowbits:issetis checked in the order it appears in the rule, from left to right. - If there is a chain of flowbits where multiple rules set flowbits and
they are dependent on each other, then the order of the rules or the
sidvalues can make a difference in the rules being evaluated in the proper order and generating alerts as expected. See bug 1399 - https://redmine.openinfosecfoundation.org/issues/1399. - 流关键字
flowbits:noalert;¶
A common pattern in existing rules is to use flowbits:noalert; to make
sure a rule doesn’t generate an alert if it matches.
Suricata allows using just noalert; as well. Both have an identical meaning
in Suricata.
Negated Content Match Special Case¶
For Snort, a negated content match where the starting point for searching is at or beyond the end of the inspection buffer will never return true.
- For negated matches, you want it to return true if the content is not found.
- This is believed to be a Snort bug rather than an engine difference but it was reported to Sourcefire and acknowledged many years ago indicating that perhaps it is by design.
- This is not the case for Suricata which behaves as expected.
Example HTTP request:
POST /test.php HTTP/1.1 Content-Length: 9 user=suri
This rule snippet will never return true in Snort but will in Suricata:
content:!"snort"; offset:10; http_client_body;
File Extraction¶
- Suricata has the ability to match on files from FTP, HTTP and SMTP streams and log them to disk.
- Snort has the “file” preprocessor that can do something similar but it is experimental, development of it has been stagnant for years, and it is not something that should be used in a production environment.
- Files can be matched on using a number of keywords including:
filenamefileextfilemagicfilesizefilemd5filesha1filesha256filesize- See File Keywords for a full list.
- The
filestorekeyword tells Suricata to save the file to disk. - Extracted files are logged to disk with meta data that includes things like timestamp, src/dst IP, protocol, src/dst port, HTTP URI, HTTP Host, HTTP Referer, filename, file magic, md5sum, size, etc.
- There are a number of configuration options and considerations (such as stream reassembly depth and libhtp body-limit) that should be understood if you want fully utilize file extraction in Suricata.
- File Keywords
- File Extraction
- https://blog.inliniac.net/2011/11/29/file-extraction-in-suricata/
- https://blog.inliniac.net/2014/11/11/smtp-file-extraction-in-suricata/
Lua Scripting¶
- Suricata has the
lua(orluajit) keyword which allows for a rule to reference a Lua script that can access the packet, payload, HTTP buffers, etc. - Provides powerful flexibility and capabilities that Snort does not have.
- Lua Scripting
Fast Pattern¶
- Snort’s fast pattern matcher is always case insensitive; Suricata’s is case sensitive unless ‘nocase’ is set on the content match used by the fast pattern matcher.
- Snort will truncate fast pattern matches based on the
max-pattern-lenconfig (default no limit) unlessfast_pattern:onlyis used in the rule. Suricata does not do any automatic fast pattern truncation cannot be configured to do so. - Just like in Snort, in Suricata you can specify a substring of the
content string to be use as the fast pattern match. e.g.
fast_pattern:5,20; - In Snort, leading NULL bytes (0x00) will be removed from content
matches when determining/using the longest content match unless
fast_patternis explicitly set. Suricata does not truncate anything, including NULL bytes. - Snort does not allow for all
http_*buffers to be used for the fast pattern match (e.g.http_raw_*,http_method,http_cookie, etc.). Suricata lets you use any ‘http_*’ buffer you want for the fast pattern match, includinghttp_raw_*' and ``http_cookiebuffers. - Suricata supports the
fast_pattern:onlysyntax but technically it is not really implemented; theonlyis silently ignored when encountered in a rule. It is still recommended that you usefast_pattern:onlywhere appropriate in case this gets implemented in the future and/or if the rule will be used by Snort as well. - With Snort, unless
fast_patternis explicitly set, content matches in normalized HTTP Inspect buffers (e.g. http content modifiers suchhttp_uri,http_header, etc.) take precedence over non-HTTP Inspect content matches, even if they are shorter. Suricata does the same thing and gives a higher ‘priority’ (precedence) tohttp_*buffers (except forhttp_method,http_stat_code, andhttp_stat_msg). - See Suricata快速匹配模式选择详解 for full details on how Suricata automatically determines which content to use as the fast pattern match.
- When in doubt about what is going to be use as the fast pattern match
by Suricata, set
fast_patternexplicitly in the rule and/or run Suricata with the--engine-analysisswitch and view the generated file (rules_fast_pattern.txt). - Like Snort, the fast pattern match is checked before
flowbitsin Suricata. - Using Hyperscan as the MPM matcher (
mpm-algosetting) for Suricata can greatly improve performance, especially when it comes to fast pattern matching. Hyperscan will also take in to account depth and offset when doing fast pattern matching, something the other algorithims and Snort do not do. - fast_pattern
Don’t Cross The Streams¶
Suricata will examine network traffic as individual packets and, in the
case of TCP, as part of a (reassembled) stream. However, there are
certain rule keywords that only apply to packets only (dsize,
flags, ttl) and certain ones that only apply to streams
only (http_*) and you can’t mix packet and stream keywords. Rules
that use packet keywords will inspect individual packets only and
rules that use stream keywords will inspect streams only. Snort is a
little more forgiving when you mix these – for example, in Snort you can
use dsize (a packet keyword) with http_* (stream
keywords) and Snort will allow it although, because of dsize, it
will only apply detection to individual packets (unless PAF is enabled
then it will apply it to the PDU).
If dsize is in a rule that also looks for a stream-based
application layer protocol (e.g. http), Suricata will not match on
the first application layer packet since dsize make Suricata
evaluate the packet and protocol detection doesn’t happen until after
the protocol is checked for that packet; subsequent packets in that
flow should have the application protocol set appropriately and will
match rules using dsize and a stream-based application layer
protocol.
If you need to check sizes on a stream in a rule that uses a stream
keyword, or in a rule looking for a stream-based application layer
protocol, consider using the stream_size keyword and/or
isdataat.
Suricata also supports these protocol values being used in rules and Snort does not:
tcp-pkt– example:alert tcp-pkt ...- This tells Suricata to only apply the rule to TCP packets and not the (reassembled) stream.
tcp-stream– example:alert tcp-stream ...- This tells Suricata to inspect the (reassembled) TCP stream only.
Alerts¶
- In Snort, the number of alerts generated for a packet/stream can be
limited by the
event_queueconfiguration. - Suricata has an internal hard-coded limit of 15 alerts per packet/stream (and this cannot be configured); all rules that match on the traffic being analyzed will fire up to that limit.
- Sometimes Suricata will generate what appears to be two alerts for the same TCP packet. This happens when Suricata evaluates the packet by itself and as part of a (reassembled) stream.
Buffer Reference Chart¶
| Buffer | Snort 2.9.x Support? | Suricata Support? | PCRE flag | Can be used as Fast Pattern? | Suricata Fast Pattern Priority (lower number is higher priority) |
|---|---|---|---|---|---|
| content (no modifier) | YES | YES | <none> | YES | 3 |
| http_method | YES | YES | M | Suricata only | 3 |
| http_stat_code | YES | YES | S | Suricata only | 3 |
| http_stat_msg | YES | YES | Y | Suricata only | 3 |
| uricontent | YES but deprecated, use http_uri instead | YES but deprecated, use http_uri instead | U | YES | 2 |
| http_uri | YES | YES | U | YES | 2 |
| http_raw_uri | YES | YES | I | Suricata only | 2 |
| http_header | YES | YES | H | YES | 2 |
| http_raw_header | YES | YES | D | Suricata only | 2 |
| http_cookie | YES | YES | C | Suricata only | 2 |
| http_raw_cookie | YES | NO (use http_raw_header instead) | K | NO | n/a |
| http_host | NO | YES | W | Suricata only | 2 |
| http_raw_host | NO | YES | Z | Suricata only | 2 |
| http_client_body | YES | YES | P | YES | 2 |
| http_server_body | NO | YES | Q | Suricata only | 2 |
| http_user_agent | NO | YES | V | Suricata only | 2 |
| dns_query | NO | YES | n/a* | Suricata only | 2 |
| tls_sni | NO | YES | n/a* | Suricata only | 2 |
| tls_cert_issuer | NO | YES | n/a* | Suricata only | 2 |
| tls_cert_subject | NO | YES | n/a* | Suricata only | 2 |
| file_data | YES | YES | n/a* | YES | 2 |
* Sticky buffer
Rule Management¶
Rule Management with Suricata-Update¶
While it is possible to download and install rules manually, it is recommended to use a management tool for this. Suricata-Update is the official way to update and manage rules for Suricata.
Suricata-update is bundled with Suricata and is normally installed with it. For instructions on installing manually, see http://suricata-update.readthedocs.io/en/latest/quickstart.html#install-suricata-update
注解
suricata-update is bundled with Suricata version 4.1 and
later. It can be used with older versions as well. It will
have to be installed separately in that case.
To download the Emerging Threats Open ruleset, it is enough to simply run:
sudo suricata-update
This will download the ruleset into /var/lib/suricata/rules/
Suricata’s configuration will have to be updated to have a rules config like this:
default-rule-path: /var/lib/suricata/rules
rule-files:
- suricata.rules
Now (re)start Suricata.
Updating your rules¶
To update the rules, simply run
sudo suricata-update
It is recommended to update your rules frequently.
Using other rulesets¶
Suricata-Update is capable of making other rulesets accessible as well.
To see what is available, fetch the master index from the OISF hosts:
sudo suricata-update update-sources
Then have a look at what is available:
sudo suricata-update list-sources
This will give a result similar to
Each of the rulesets has a name that has a ‘vendor’ prefix, followed by a set name. For example, OISF’s traffic id ruleset is called ‘oisf/trafficid’.
To enable ‘oisf/trafficid’, enter:
sudo suricata-update enable-source oisf/trafficid
sudo suricata-update
Now restart Suricata again and the rules from the OISF TrafficID ruleset are loaded.
To see which rulesets are currently active, use “list-enabled-sources”.
Controlling which rules are used¶
By default suricata-update will merge all rules into a single file “/var/lib/suricata/rules/suricata.rules”.
To enable rules that are disabled by default, use ‘/etc/suricata/enable.conf’
2019401 # enable this sid
group:emerging-icmp.rules # enable this rulefile
re:trojan # enable all rules with this string
Similarly, to disable rules use /etc/suricata/disable.conf:
2019401 # enable this sid
group:emerging-info.rules # disable this rulefile
re:heartbleed # disable all rules with this string
After updating these files, rerun suricata-update again:
sudo suricata-update
Finally restart Suricata.
Further reading¶
Rule Management with Oinkmaster¶
注解
suricata-update is the official and recommended way to
update and manage rules and rulesets. See Rule Management with Suricata-Update
It is possible to download and install rules manually, but there is a much easier and quicker way to do so. There are special programs which you can use for downloading and installing rules. There is for example Pulled Pork and Oinkmaster. In this documentation the use of Oinkmaster will be described.
To install Oinkmaster, enter:
sudo apt-get install oinkmaster
There are several rulesets. There is for example Emerging Threats (ET) Emerging Threats Pro and VRT. In this example we are using Emerging Threats.
Oinkmaster has to know where the rules an be found. These rules can be found at:
https://rules.emergingthreats.net/open/suricata-3.2/emerging.rules.tar.gz
open oinkmaster.conf to add this link by entering:
sudo nano /etc/oinkmaster.conf
Place a # in front of the url that is already there and add the new url like this:
(Close oinkmaster.conf by pressing ctrl x, followed by y and enter. )
The next step is to create a directory for the new rules. Enter:
sudo mkdir /etc/suricata/rules
Next enter:
cd /etc
sudo oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules
In the new rules directory a classification.config and a reference.config can be found. The directories of both have to be added in the suricata.yaml file. Do so by entering:
sudo nano /etc/suricata/suricata.yaml
And add the new file locations instead of the file locations already present, like this:
To see if everything works as pleased, run Suricata:
suricata -c /etc/suricata/suricata.yaml -i wlan0 (or eth0)
You will notice there are several rule-files Suricata tries to load, but are not available. It is possible to disable those rule-sets in suricata.yaml by deleting them or by putting a # in front of them. To stop Suricata from running, press ctrl c.
Emerging Threats contains more rules than loaded in Suricata. To see which rules are available in your rules directory, enter:
ls /etc/suricata/rules/*.rules
Find those that are not yet present in suricata.yaml and add them in yaml if desired.
You can do so by entering :
sudo nano /etc/suricata/suricata.yaml
If you disable a rule in your rule file by putting a # in front of it, it will be enabled again the next time you run Oinkmaster. You can disable it through Oinkmaster instead, by entering the following:
cd /etc/suricata/rules
and find the sid of the rule(s) you want to disable.
Subsequently enter:
sudo nano /etc/oinkmaster.conf
and go all the way to the end of the file. Type there:
disablesid 2010495
Instead of 2010495, type the sid of the rule you would like to disable. It is also possible to disable multiple rules, by entering their sids separated by a comma.
If you run Oinkmaster again, you can see the amount of rules you have disabled. You can also enable rules that are disabled by default. Do so by entering:
ls /etc/suricata/rules
In this directory you can see several rule-sets Enter for example:
sudo nano /etc/suricata/rules/emerging-malware.rules
In this file you can see which rules are enabled en which are not. You can not enable them for the long-term just by simply removing the #. Because each time you will run Oinkmaster, the rule will be disabled again. Instead, look up the sid of the rule you want to enable. Place the sid in the correct place of oinkmaster.config:
sudo nano /etc/oinkmaster.conf
do so by typing:
enablesid: 2010495
Instead of 2010495, type the sid of the rule you would like to to enable. It is also possible to enable multiple rules, by entering their sids separated by a comma.
In oinkmaster.conf you can modify rules. For example, if you use Suricata as inline/IPS and you want to modify a rule that sends an alert when it matches and you would like the rule to drop the packet instead, you can do so by entering the following:
sudo nano oinkmaster.conf
At the part where you can modify rules, type:
modifysid 2010495 “alert” | “drop”
The sid 2010495 is an example. Type the sid of the rule you desire to change, instead.
Rerun Oinkmaster to notice the change.
Updating your rules¶
If you have already downloaded a ruleset (in the way described in this file), and you would like to update the rules, enter:
sudo oinkmaster -C /etc/oinkmaster.conf -o /etc/suricata/rules
It is recommended to update your rules frequently. Emerging Threats is modified daily, VRT is updated weekly or multiple times a week.
Adding Your Own Rules¶
If you would like to create a rule yourself and use it with Suricata, this guide might be helpful.
Start creating a file for your rule. Type for example the following in your console:
sudo nano local.rules
Write your rule, see 规则格式 and save it.
Open yaml
sudo nano /etc/suricata/suricata.yaml
and make sure your local.rules file is added to the list of rules.
Now, run Suricata and see if your rule is being loaded.
suricata -c /etc/suricata/suricata.yaml -i wlan0
If your rule failed to load, check if you have made a mistake anywhere in the rule. Mind the details; look for mistakes in special characters, spaces, capital characters etc.
Next, check if your log-files are enabled in suricata.yaml.
If you had to correct your rule and/or modify yaml, you have to restart Suricata.
If you see your rule is successfully loaded, you can double check your rule by doing something that should trigger it.
Enter:
tail -f /var/log/suricata/fast.log
If you would make a rule like this:
alert http any any -> any any (msg:"Do not read gossip during work";
content:"Scarlett"; nocase; classtype:policy-violation; sid:1; rev:1;)
Your alert should look like this:
09/15/2011-16:50:27.725288 [**] [1:1:1] Do not read gossip during work [**]
[Classification: Potential Corporate Privacy Violation] [Priority: 1] {TCP} 192.168.0.32:55604 -> 68.67.185.210:80
Rule Reloads¶
Suricata can be told to reloads it’s rules without restarting.
This works by sending Suricata a signal or by using the unix socket. When Suricata is told to reload the rules these are the basic steps it takes:
- Load new config
- Load new rules
- Construct new detection engine
- Swap old and new detection engines
- Make sure all threads are updated
- Free old detection engine
Suricata will continue to process packets normally during this process. Keep in mind though, that the system should have enough memory for both detection engines.
Signal:
kill -USR2 $(pidof suricata)
Unix socket has two method for rules reload.
Blocking reload
suricatasc -c reload-rules
Non blocking reload
suricatasc -c ruleset-reload-nonblocking
It is also possible to get information about the last reload via dedicated commands. See Commands in standard running mode for more information.
Making sense out of Alerts¶
When alert happens it’s important to figure out what it means. Is it serious? Relevant? A false positive?
To find out more about the rule that fired, it’s always a good idea to look at the actual rule.
The first thing to look at in a rule is the description that follows the “msg” keyword. Lets consider an example:
msg:"ET SCAN sipscan probe";
The “ET” indicates the rule came from the Emerging Threats project. “SCAN” indicates the purpose of the rule is to match on some form of scanning. Following that a more or less detailed description is given.
Most rules contain some pointers to more information in the form of the “reference” keyword.
Consider the following example rule:
alert tcp $HOME_NET any -> $EXTERNAL_NET $HTTP_PORTS \
(msg:"ET CURRENT_EVENTS Adobe 0day Shovelware"; \
flow:established,to_server; content:"GET "; nocase; depth:4; \
content:!"|0d 0a|Referer\:"; nocase; \
uricontent:"/ppp/listdir.php?dir="; \
pcre:"/\/[a-z]{2}\/[a-z]{4}01\/ppp\/listdir\.php\?dir=/U"; \
classtype:trojan-activity; \
reference:url,isc.sans.org/diary.html?storyid=7747; \
reference:url,doc.emergingthreats.net/2010496; \
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/CURRENT_EVENTS/CURRENT_Adobe; \
sid:2010496; rev:2;)
In this rule the reference keyword indicates 3 url’s to visit for more information:
isc.sans.org/diary.html?storyid=7747
doc.emergingthreats.net/2010496
www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/CURRENT_EVENTS/CURRENT_Adobe
Some rules contain a reference like: “reference:cve,2009-3958;” should allow you to find info about the specific CVE using your favorite search engine.
It’s not always straight forward and sometimes not all of that information is available publicly. Usually asking about it on the signature support channel helps a lot then.
In Rule Management with Suricata-Update more information on the rule sources and their documentation and support methods can be found.
In many cases, looking at just the alert and the packet that triggered it won’t be enough to be conclusive. When using the default Eve settings a lot of metadata will be added to the alert.
For example, if a rule fired that indicates your web application is attacked, looking at the metadata might reveal that the web application replied with 404 not found. This will usually mean the attack failed. Usually, not always.
Not every protocol leads to metadata generation, so when running an IDS engine like Suricata, it’s often recommended to combine it with full packet capture. Using tools like Evebox, Sguil or Snorby, the full TCP session or UDP flow can be inspected.
Obviously there is a lot more to Incidence Response, but this should get you started.
Performance¶
Runmodes¶
Suricata consists of several ‘building blocks’ called threads, thread-modules and queues. A thread is like a process that runs on a computer. Suricata is multi-threaded, so multiple threads are active at once. A thread-module is a part of a functionality. One module is for example for decoding a packet, another is the detect-module and another one the output-module. A packet can be processed by more than one thread. The packet will be passed on to the next thread through a queue. Packets will be processed by one thread at a time, but there can be multiple packets being processed at a time by the engine. (see Max-pending-packets) A thread can have one or more thread-modules. If they have more modules, they can only be active on a a time. The way threads, modules and queues are arranged together is called the Runmode.
Different runmodes¶
You can choose a runmode out of several predefined runmodes. The command line option –list-runmodes shows all available runmodes. All runmodes have a name: auto, single, autofp. The heaviest task is the detection; a packet will be checked against thousands of signatures.
Example of the default runmode:
In the pfring mode, every flow follows its own fixed route in the runmode.
For more information about the command line options concerning the runmode, see 命令行选项.
Packet Capture¶
Load balancing¶
To get the best performance, Suricata will need to run in ‘workers’ mode. This effectively means that there are multiple threads, each running a full packet pipeline and each receiving packets from the capture method. This means that we rely on the capture method to distribute the packets over the various threads. One critical aspect of this is that Suricata needs to get both sides of a flow in the same thread, in the correct order.
The AF_PACKET and PF_RING capture methods both have options to select the ‘cluster-type’. These default to ‘cluster_flow’ which instructs the capture method to hash by flow (5 tuple). This hash is symmetric. Netmap does not have a cluster_flow mode built-in. It can be added separately by using the “‘lb’ tool”:https://github.com/luigirizzo/netmap/tree/master/apps/lb
> WARNING Recent AF_PACKET changes have “broken”:https://redmine.openinfosecfoundation.org/issues/1777 this symmetry. Work is under way to “address this”:https://redmine.openinfosecfoundation.org/issues/1777#note-7, but for now stay on kernel <=4.2 or update to 4.4.16+, 4.6.5+ or 4.7+.
On multi-queue NICs, which is almost any modern NIC, RSS settings need to be considered.
RSS¶
Receive Side Scaling is a technique used by network cards to distribute incoming traffic over various queues on the NIC. This is meant to improve performance but it is important to realize that it was designed for normal traffic, not for the IDS packet capture scenario. RSS using a hash algorithm to distribute the incoming traffic over the various queues. This hash is normally not symmetrical. This means that when receiving both sides of a flow, each side may end up in a different queue. Sadly, when deploying Suricata, this is the common scenario when using span ports or taps.
The problem here is that by having both sides of the traffic in different queues, the order of processing of packets becomes unpredictable. Timing differences on the NIC, the driver, the kernel and in Suricata will lead to a high chance of packets coming in at a different order than on the wire. This is specifically about a mismatch between the two traffic directions. For example, Suricata tracks the TCP 3-way handshake. Due to this timing issue, the SYN/ACK may only be received by Suricata long after the client to server side has already started sending data. Suricata would see this traffic as invalid.
None of the supported capture methods like AF_PACKET, PF_RING or NETMAP can fix this problem for us. It would require buffering and packet reordering which is expensive.
To see how many queues are configured:
$ ethtool -l ens2f1
Channel parameters for ens2f1:
Pre-set maximums:
RX: 0
TX: 0
Other: 1
Combined: 64
Current hardware settings:
RX: 0
TX: 0
Other: 1
Combined: 8
Some NIC’s allow you to set it into a symmetric mode. The Intel X(L)710 card can do this in theory, but the drivers aren’t capable of enabling this yet (work is underway to try to address this). Another way to address is by setting a special “Random Secret Key” that will make the RSS symmetrical. See http://www.ndsl.kaist.edu/~kyoungsoo/papers/TR-symRSS.pdf (PDF).
In most scenario’s however, the optimal solution is to reduce the number of RSS queues to 1:
Example:
# Intel X710 with i40e driver:
ethtool -L $DEV combined 1
Some drivers do not support setting the number of queues through ethtool. In some cases there is a module load time option. Read the driver docs for the specifics.
Offloading¶
Network cards, drivers and the kernel itself have various techniques to speed up packet handling. Generally these will all have to be disabled.
LRO/GRO lead to merging various smaller packets into big ‘super packets’. These will need to be disabled as they break the dsize keyword as well as TCP state tracking.
Checksum offloading can be left enabled on AF_PACKET and PF_RING, but needs to be disabled on PCAP, NETMAP and others.
Recommendations¶
Read your drivers documentation! E.g. for i40e the ethtool change of RSS queues may lead to kernel panics if done wrong.
Generic: set RSS queues to 1 or make sure RSS hashing is symmetric. Disable NIC offloading.
AF_PACKET: 1 RSS queue and stay on kernel <=4.2 or make sure you have >=4.4.16, >=4.6.5 or >=4.7. Exception: if RSS is symmetric cluster-type ‘cluster_qm’ can be used to bind Suricata to the RSS queues. Disable NIC offloading except the rx/tx csum.
PF_RING: 1 RSS queue and use cluster-type ‘cluster_flow’. Disable NIC offloading except the rx/tx csum.
NETMAP: 1 RSS queue. There is no flow based load balancing built-in, but the ‘lb’ tool can be helpful. Another option is to use the ‘autofp’ runmode. Exception: if RSS is symmetric, load balancing is based on the RSS hash and multiple RSS queues can be used. Disable all NIC offloading.
Tuning Considerations¶
Settings to check for optimal performance.
max-pending-packets: <number>¶
This setting controls the number simultaneous packets that the engine can handle. Setting this higher generally keeps the threads more busy, but setting it too high will lead to degradation.
Suggested setting: 1000 or higher. Max is ~65000.
mpm-algo: <ac|hs|ac-bs|ac-ks>¶
Controls the pattern matcher algorithm. AC is the default. On supported platforms, Hyperscan is the best option.
detect.profile: <low|medium|high|custom>¶
The detection engine tries to split out separate signatures into groups so that a packet is only inspected against signatures that can actually match. As in large rule set this would result in way too many groups and memory usage similar groups are merged together. The profile setting controls how aggressive this merging is done. Higher is better but results in (much) higher memory usage.
The “custom” setting allows modification of the group sizes:
custom-values:
toclient-groups: 50
toserver-groups: 50
In general, increasing will improve performance, but will lead to higher memory usage.
detect.sgh-mpm-context: <auto|single|full>¶
The multi pattern matcher can have it’s context per signature group (full) or globally (single). Auto selects between single and full based on the mpm-algo selected. ac and ac-bs use “single”. All others “full”. Setting this to “full” with AC requires a lot of memory: 32GB+ for a reasonable rule set.
Hyperscan¶
Introduction¶
“Hyperscan is a high-performance multiple regex matching library.” https://01.org/hyperscan
In Suricata it can be used to perform multi pattern matching (mpm). Support was implemented by Justin Viiret and Jim Xu from Intel: https://github.com/inliniac/suricata/pull/1965, https://redmine.openinfosecfoundation.org/issues/1704
Compilation¶
It’s possible to pass –with-libhs-includes=/usr/local/include/hs/ –with-libhs-libraries=/usr/local/lib/, although by default this shouldn’t be necessary. Suricata should pick up Hyperscan’s pkg-config file automagically.
When Suricata’s compilation succeeded, you should have:
suricata --build-info|grep Hyperscan
Hyperscan support: yes
Using Hyperscan¶
To use the hyperscan support edit your suricata.yaml. Change the mpm-algo and spm-algo values to ‘hs’.
Alternatively, use this commandline option: –set mpm-algo=hs –set spm-algo=hs
Ubuntu Hyperscan Installation¶
To use Suricata with Hyperscan support, install dependencies:
apt-get install cmake ragel
libboost headers¶
Hyperscan needs the libboost headers from 1.58+.
On Ubuntu 15.10 or 16.04+, simply do:
apt-get install libboost-dev
Trusty¶
Trusty has 1.57, so it’s too old. We can grab a newer libboost version, but we don’t install it system wide. It’s only the headers we care about during compilation of Hyperscan.
sudo apt-get python-dev libbz2-dev
wget http://downloads.sourceforge.net/project/boost/boost/1.60.0/boost_1_60_0.tar.gz
tar xvzf boost_1_60_0.tar.gz
cd boost_1_60_0
./bootstrap.sh --prefix=~/tmp/boost-1.60
./b2 install
Hyperscan¶
We’ll install version 4.2.0.
git clone https://github.com/01org/hyperscan
cd hyperscan
mkdir build
cd build
cmake -DBUILD_STATIC_AND_SHARED=1 ../
If you have your own libboost headers, use this cmake line instead:
cmake -DBUILD_STATIC_AND_SHARED=1 -DBOOST_ROOT=~/tmp/boost-1.60 ../
Finally, make and make install:
make
sudo make install
Compilation can take a long time, but it should in the end look something like this:
Install the project...
-- Install configuration: "RELWITHDEBINFO"
-- Installing: /usr/local/lib/pkgconfig/libhs.pc
-- Up-to-date: /usr/local/include/hs/hs.h
-- Up-to-date: /usr/local/include/hs/hs_common.h
-- Up-to-date: /usr/local/include/hs/hs_compile.h
-- Up-to-date: /usr/local/include/hs/hs_runtime.h
-- Installing: /usr/local/lib/libhs_runtime.a
-- Installing: /usr/local/lib/libhs_runtime.so.4.2.0
-- Installing: /usr/local/lib/libhs_runtime.so.4.2
-- Installing: /usr/local/lib/libhs_runtime.so
-- Installing: /usr/local/lib/libhs.a
-- Installing: /usr/local/lib/libhs.so.4.2.0
-- Installing: /usr/local/lib/libhs.so.4.2
-- Installing: /usr/local/lib/libhs.so
Note that you may have to add /usr/local/lib to your ld search path
echo "/usr/local/lib" | sudo tee --append /etc/ld.so.conf.d/usrlocal.conf
sudo ldconfig
High Performance Configuration¶
If you have enough RAM, consider the following options in suricata.yaml to off-load as much work from the CPU’s as possible:
detect:
profile: custom
custom-values:
toclient-groups: 200
toserver-groups: 200
sgh-mpm-context: auto
inspection-recursion-limit: 3000
Be advised, however, that this may require lots of RAM for even modestly sized rule sets. Also be aware that having additional CPU’s available provides a greater performance boost than having more RAM available. That is, it would be better to spend money on CPU’s instead of RAM when configuring a system.
It may also lead to significantly longer rule loading times.
Statistics¶
The stats.log produces statistics records on a fixed interval, by default every 8 seconds.
stats.log file¶
-------------------------------------------------------------------
Counter | TM Name | Value
-------------------------------------------------------------------
flow_mgr.closed_pruned | FlowManagerThread | 154033
flow_mgr.new_pruned | FlowManagerThread | 67800
flow_mgr.est_pruned | FlowManagerThread | 100921
flow.memuse | FlowManagerThread | 6557568
flow.spare | FlowManagerThread | 10002
flow.emerg_mode_entered | FlowManagerThread | 0
flow.emerg_mode_over | FlowManagerThread | 0
decoder.pkts | RxPcapem21 | 450001754
decoder.bytes | RxPcapem21 | 409520714250
decoder.ipv4 | RxPcapem21 | 449584047
decoder.ipv6 | RxPcapem21 | 9212
decoder.ethernet | RxPcapem21 | 450001754
decoder.raw | RxPcapem21 | 0
decoder.sll | RxPcapem21 | 0
decoder.tcp | RxPcapem21 | 448124337
decoder.udp | RxPcapem21 | 542040
decoder.sctp | RxPcapem21 | 0
decoder.icmpv4 | RxPcapem21 | 82292
decoder.icmpv6 | RxPcapem21 | 9164
decoder.ppp | RxPcapem21 | 0
decoder.pppoe | RxPcapem21 | 0
decoder.gre | RxPcapem21 | 0
decoder.vlan | RxPcapem21 | 0
decoder.avg_pkt_size | RxPcapem21 | 910
decoder.max_pkt_size | RxPcapem21 | 1514
defrag.ipv4.fragments | RxPcapem21 | 4
defrag.ipv4.reassembled | RxPcapem21 | 1
defrag.ipv4.timeouts | RxPcapem21 | 0
defrag.ipv6.fragments | RxPcapem21 | 0
defrag.ipv6.reassembled | RxPcapem21 | 0
defrag.ipv6.timeouts | RxPcapem21 | 0
tcp.sessions | Detect | 41184
tcp.ssn_memcap_drop | Detect | 0
tcp.pseudo | Detect | 2087
tcp.invalid_checksum | Detect | 8358
tcp.no_flow | Detect | 0
tcp.reused_ssn | Detect | 11
tcp.memuse | Detect | 36175872
tcp.syn | Detect | 85902
tcp.synack | Detect | 83385
tcp.rst | Detect | 84326
tcp.segment_memcap_drop | Detect | 0
tcp.stream_depth_reached | Detect | 109
tcp.reassembly_memuse | Detect | 67755264
tcp.reassembly_gap | Detect | 789
detect.alert | Detect | 14721
Detecting packet loss¶
At shut down, Suricata reports the packet loss statistics it gets from pcap, pfring or afpacket
[18088] 30/5/2012 -- 07:39:18 - (RxPcapem21) Packets 451595939, bytes 410869083410
[18088] 30/5/2012 -- 07:39:18 - (RxPcapem21) Pcap Total:451674222 Recv:451596129 Drop:78093 (0.0%).
Usually, this is not the complete story though. These are kernel drop stats, but the NIC may also have dropped packets. Use ethtool to get to those:
# ethtool -S em2
NIC statistics:
rx_packets: 35430208463
tx_packets: 216072
rx_bytes: 32454370137414
tx_bytes: 53624450
rx_broadcast: 17424355
tx_broadcast: 133508
rx_multicast: 5332175
tx_multicast: 82564
rx_errors: 47
tx_errors: 0
tx_dropped: 0
multicast: 5332175
collisions: 0
rx_length_errors: 0
rx_over_errors: 0
rx_crc_errors: 51
rx_frame_errors: 0
rx_no_buffer_count: 0
rx_missed_errors: 0
tx_aborted_errors: 0
tx_carrier_errors: 0
tx_fifo_errors: 0
tx_heartbeat_errors: 0
tx_window_errors: 0
tx_abort_late_coll: 0
tx_deferred_ok: 0
tx_single_coll_ok: 0
tx_multi_coll_ok: 0
tx_timeout_count: 0
tx_restart_queue: 0
rx_long_length_errors: 0
rx_short_length_errors: 0
rx_align_errors: 0
tx_tcp_seg_good: 0
tx_tcp_seg_failed: 0
rx_flow_control_xon: 0
rx_flow_control_xoff: 0
tx_flow_control_xon: 0
tx_flow_control_xoff: 0
rx_long_byte_count: 32454370137414
rx_csum_offload_good: 35270755306
rx_csum_offload_errors: 65076
alloc_rx_buff_failed: 0
tx_smbus: 0
rx_smbus: 0
dropped_smbus: 0
Kernel drops¶
stats.log contains interesting information in the capture.kernel_packets and capture.kernel_drops. The meaning of them is different following the capture mode.
In AF_PACKET mode:
- kernel_packets is the number of packets correctly sent to userspace
- kernel_drops is the number of packets that have been discarded instead of being sent to userspace
In PF_RING mode:
- kernel_packets is the total number of packets seen by pf_ring
- kernel_drops is the number of packets that have been discarded instead of being sent to userspace
In the Suricata stats.log the TCP data gap counter is also an indicator, as it accounts missing data packets in TCP streams:
tcp.reassembly_gap | Detect | 789
Ideally, this number is 0. Not only pkt loss affects it though, also bad checksums and stream engine running out of memory.
Tools to plot graphs¶
Some people made nice tools to plot graphs of the statistics file.
Ignoring Traffic¶
In some cases there are reasons to ignore certain traffic. Certain hosts may be trusted, or perhaps a backup stream should be ignored.
This document lists some strategies for ignoring traffic.
capture filters (BPF)¶
Through BPFs the capture methods pcap, af-packet and pf_ring can be told what to send to Suricata, and what not. For example a simple filter ‘tcp’ will only send tcp packets.
If some hosts and or nets need to be ignored, use something like “not (host IP1 or IP2 or IP3 or net NET/24)”.
Example:
not host 1.2.3.4
Capture filters are specified on the commandline after all other options:
suricata -i eth0 -v not host 1.2.3.4
suricata -i eno1 -c suricata.yaml tcp or udp
Capture filters can be set per interface in the pcap, af-packet, netmap and pf_ring sections. It can also be put in a file:
echo "not host 1.2.3.4" > capture-filter.bpf
suricata -i ens5f0 -F capture-filter.bpf
Using a capture filter limits what traffic Suricata processes. So the traffic not seen by Suricata will not be inspected, logged or otherwise recorded.
pass rules¶
Pass rules are Suricata rules that if matching, pass the packet and in case of TCP the rest of the flow. They look like normal rules, except that instead of ‘alert’ or ‘drop’ they start with ‘pass’.
Example:
pass ip 1.2.3.4 any <> any any (msg:"pass all traffic from/to 1.2.3.4"; sid:1;)
A big difference with capture filters is that logs such as Eve or http.log are still generated for this traffic.
suppress¶
Suppress rules can be used to make sure no alerts are generated for a host. This is not efficient however, as the suppression is only considered post-matching. In other words, Suricata first inspects a rule, and only then will it consider per-host suppressions.
Example:
suppress gen_id 0, sig_id 0, track by_src, ip 1.2.3.4
Encrypted traffic¶
The TLS app layer parser has the ability to stop processing encrypted traffic after the initial handshake. By setting the app-layer.protocols.tls.encryption-handling option to bypass the rest of this flow is ignored. If flow bypass is enabled, the bypass is done in the kernel or in hardware.
Packet Profiling¶
In this guide will be explained how to enable packet profiling and use it with the most recent code of Suricata on Ubuntu. It is based on the assumption that you have already installed Suricata once from the GIT repository.
Packet profiling is convenient in case you would like to know how long packets take to be processed. It is a way to figure out why certain packets are being processed quicker than others, and this way a good tool for developing Suricata.
Update Suricata by following the steps from Installation from Git. Start at the end at
cd suricata/oisf
git pull
And follow the described next steps. To enable packet profiling, make sure you enter the following during the configuring stage:
./configure --enable-profiling
Find a folder in which you have pcaps. If you do not have pcaps yet, you can get these with Wireshark. See Sniffing Packets with Wireshark.
Go to the directory of your pcaps. For example:
cd ~/Desktop
With the ls command you can see the content of the folder. Choose a folder and a pcap file
for example:
cd ~/Desktop/2011-05-05
Run Suricata with that pcap:
suricata -c /etc/suricata/suricata.yaml -r log.pcap.(followed by the number/name of your pcap)
for example:
suricata -c /etc/suricata/suricata.yaml -r log.pcap.1304589204
Rule Profiling¶
--------------------------------------------------------------------------
Date: 9/5/2013 -- 14:59:58
--------------------------------------------------------------------------
Num Rule Gid Rev Ticks % Checks Matches Max Ticks Avg Ticks Avg Match Avg No Match
-------- ------------ -------- -------- ------------ ------ -------- -------- ----------- ----------- ----------- --------------
1 2210021 1 3 12037 4.96 1 1 12037 12037.00 12037.00 0.00
2 2210054 1 1 107479 44.26 12 0 35805 8956.58 0.00 8956.58
3 2210053 1 1 4513 1.86 1 0 4513 4513.00 0.00 4513.00
4 2210023 1 1 3077 1.27 1 0 3077 3077.00 0.00 3077.00
5 2210008 1 1 3028 1.25 1 0 3028 3028.00 0.00 3028.00
6 2210009 1 1 2945 1.21 1 0 2945 2945.00 0.00 2945.00
7 2210055 1 1 2945 1.21 1 0 2945 2945.00 0.00 2945.00
8 2210007 1 1 2871 1.18 1 0 2871 2871.00 0.00 2871.00
9 2210005 1 1 2871 1.18 1 0 2871 2871.00 0.00 2871.00
10 2210024 1 1 2846 1.17 1 0 2846 2846.00 0.00 2846.00
The meaning of the individual fields:
- Ticks – total ticks spent on this rule, so a sum of all inspections
- % – share of this single sig in the total cost of inspection
- Checks – number of times a signature was inspected
- Matches – number of times it matched. This may not have resulted in an alert due to suppression and thresholding.
- Max ticks – single most expensive inspection
- Avg ticks – per inspection average, so “ticks” / “checks”.
- Avg match – avg ticks spent resulting in match
- Avg No Match – avg ticks spent resulting in no match.
The “ticks” are CPU clock ticks: http://en.wikipedia.org/wiki/CPU_time
Tcmalloc¶
‘tcmalloc’ is a library Google created as part of the google-perftools suite for improving memory handling in a threaded program. It’s very simple to use and does work fine with Suricata. It leads to minor speed ups and also reduces memory usage quite a bit.
Installation¶
On Ubuntu, install the libtcmalloc-minimal0 package:
apt-get install libtcmalloc-minimal0
On Fedora, install the gperftools-libs package:
yum install gperftools-libs
Usage¶
Use the tcmalloc by preloading it:
Ubuntu:
LD_PRELOAD=”/usr/lib/libtcmalloc_minimal.so.0" suricata -c suricata.yaml -i eth0
Fedora:
LD_PRELOAD="/usr/lib64/libtcmalloc_minimal.so.4" suricata -c suricata.yaml -i eth0
Configuration¶
Suricata.yaml¶
Suricata uses the Yaml format for configuration. The Suricata.yaml file included in the source code, is the example configuration of Suricata. This document will explain each option.
At the top of the YAML-file you will find % YAML 1.1. Suricata reads the file and identifies the file as YAML.
Max-pending-packets¶
With the max-pending-packets setting you can set the number of packets you allow Suricata to process simultaneously. This can range from one packet to tens of thousands/hundreds of thousands of packets. It is a trade of higher performance and the use of more memory (RAM), or lower performance and less use of memory. A high number of packets being processed results in a higher performance and the use of more memory. A low number of packets, results in lower performance and less use of memory. Choosing a low number of packets being processed while having many CPU’s/CPU cores, can result in not making use of the whole computer-capacity. (For instance: using one core while having three waiting for processing packets.)
max-pending-packets: 1024
Runmodes¶
By default the runmode option is disabled With the runmodes setting you can set the runmode you would like to use. For all runmodes available, enter –list-runmodes in your command line. For more information, see Runmodes.
runmode: autofp
Default-packet-size¶
For the max-pending-packets option, Suricata has to keep packets in memory. With the default-packet-size option, you can set the size of the packets on your network. It is possible that bigger packets have to be processed sometimes. The engine can still process these bigger packets, but processing it will lower the performance.
default-packet-size: 1514
User and group¶
It is possible to set the user and group to run Suricata as:
run-as:
user: suri
group: suri
PID File¶
This option sets the name of the PID file when Suricata is run in daemon mode. This file records the Suricata process ID.
pid-file: /var/run/suricata.pid
注解
This configuration file option only sets the PID file when
running in daemon mode. To force creation of a PID file when
not running in daemon mode, use the --pidfile
command line option.
Also, if running more than one Suricata process, each process will need to specify a different pid-file location.
Action-order¶
All signatures have different properties. One of those is the Action property. This one determines what will happen when a signature matches. There are four types of Action. A summary of what will happen when a signature matches and contains one of those Actions:
- Pass
If a signature matches and contains pass, Suricata stops scanning the packet and skips to the end of all rules (only for the current packet).
- Drop
This only concerns the IPS/inline mode. If the program finds a signature that matches, containing drop, it stops immediately. The packet will not be sent any further. Drawback: The receiver does not receive a message of what is going on, resulting in a time-out (certainly with TCP). Suricata generates an alert for this packet.
- Reject
This is an active rejection of the packet. Both receiver and sender receive a reject packet. There are two types of reject packets that will be automatically selected. If the offending packet concerns TCP, it will be a Reset-packet. For all other protocols it will be an ICMP-error packet. Suricata also generates an alert. When in Inline/IPS mode, the offending packet will also be dropped like with the ‘drop’ action.
- Alert
If a signature matches and contains alert, the packet will be treated like any other non-threatening packet, except for this one an alert will be generated by Suricata. Only the system administrator can notice this alert.
Inline/IPS can block network traffic in two ways. One way is by drop and the other by reject.
Rules will be loaded in the order of which they appear in files. But they will be processed in a different order. Signatures have different priorities. The most important signatures will be scanned first. There is a possibility to change the order of priority. The default order is: pass, drop, reject, alert.
action-order:
- pass
- drop
- reject
- alert
This means a pass rule is considered before a drop rule, a drop rule before a reject rule and so on.
Splitting configuration in multiple files¶
Some users might have a need or a wish to split their suricata.yaml file in to separate files, this is available vis the ‘include’ and ‘!include’ keyword. The first example is of taking the contents of the outputs section and storing them in outputs.yaml
# outputs.yaml
- fast
enabled: yes
filename: fast.log
append: yes
- unified2-alert:
enabled: yes
...
# suricata.yaml
...
outputs: !include outputs.yaml
...
The second scenario is where multiple sections are migrated to a different YAML file.
# host_1.yaml
max-pending-packets: 2048
outputs:
- fast
enabled: yes
filename: fast.log
append: yes
- unified2-alert:
enabled: yes
# suricata.yaml
include: host_1.yaml
...
If the same section, say outputs is later redefined after the include statement it will overwrite the included file. Therefor any include statement at the end of the document will overwrite the already configured sections.
Event output¶
Default logging directory¶
In the /var/log/suricata directory, all of Suricata’s output (alerts and events) will be stored.
default-log-dir: /var/log/suricata
This directory can be overridden by entering the -l command line parameter or by changing the directory directly in Yaml. To change it with the -l command line parameter, enter the following:
suricata -c suricata.yaml -i eth0 -l /var/log/suricata-logs/
Outputs¶
There are several types of output. The general structure is:
outputs:
- fast:
enabled: yes
filename: fast.log
append: yes/no
Enabling all of the logs, will result in a much lower performance and the use of more disc space, so enable only the outputs you need.
Line based alerts log (fast.log)¶
This log contains alerts consisting of a single line. Example of the appearance of a single fast.log-file line:
10/05/10-10:08:59.667372 [**] [1:2009187:4] ET WEB_CLIENT ACTIVEX iDefense
COMRaider ActiveX Control Arbitrary File Deletion [**] [Classification: Web
Application Attack] [Priority: 3] {TCP} xx.xx.232.144:80 -> 192.168.1.4:56068
-fast: #The log-name.
enabled:yes #This log is enabled. Set to 'no' to disable.
filename: fast.log #The name of the file in the default logging directory.
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
#overwritten while restarting Suricata.
Eve (Extensible Event Format)¶
This is an JSON output for alerts and events. It allows for easy integration with 3rd party tools like logstash.
outputs:
# Extensible Event Format (nicknamed EVE) event log in JSON format
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
#prefix: "@cee: " # prefix to prepend to each log entry
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
#redis:
# server: 127.0.0.1
# port: 6379
# async: true ## if redis replies are read asynchronously
# mode: list ## possible values: list|lpush (default), rpush, channel|publish
# ## lpush and rpush are using a Redis list. "list" is an alias for lpush
# ## publish is using a Redis channel. "channel" is an alias for publish
# key: suricata ## key or channel to use (default to suricata)
# Redis pipelining set up. This will enable to only do a query every
# 'batch-size' events. This should lower the latency induced by network
# connection at the cost of some memory. There is no flushing implemented
# so this setting as to be reserved to high traffic suricata.
# pipelining:
# enabled: yes ## set enable to yes to enable query pipelining
# batch-size: 10 ## number of entry to keep in buffer
# Include top level metadata. Default yes.
#metadata: no
types:
- alert:
# payload: yes # enable dumping payload in Base64
# payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
# payload-printable: yes # enable dumping payload in printable (lossy) format
# packet: yes # enable dumping of packet (without stream segments)
# http-body: yes # enable dumping of http body in Base64
# http-body-printable: yes # enable dumping of http body in printable format
# Enable the logging of tagged packets for rules using the
# "tag" keyword.
tagged-packets: yes
# Configure the metadata to be logged along with an
# alert. The following shows the default configuration
# which is used if this field is not provided or simply
# set to a truthful value. Setting of this section is only
# required if you wish to enable/disable specific fields.
#metadata:
# Include the decoded application layer (ie. http, dns)
app-layer: true
# Log the the current state of the flow record.
flow: true
rule:
# Log the metadata field from the rule in a structured
# format.
metadata: true
# Log the raw rule text.
raw: false
# HTTP X-Forwarded-For support by adding an extra field or overwriting
# the source or destination IP address (depending on flow direction)
# with the one reported in the X-Forwarded-For HTTP header. This is
# helpful when reviewing alerts for traffic that is being reverse
# or forward proxied.
xff:
enabled: no
# Two operation modes are available, "extra-data" and "overwrite".
mode: extra-data
# Two proxy deployments are supported, "reverse" and "forward". In
# a "reverse" deployment the IP address used is the last one, in a
# "forward" deployment the first IP address is used.
deployment: reverse
# Header name where the actual IP address will be reported, if more
# than one IP address is present, the last IP address will be the
# one taken into consideration.
header: X-Forwarded-For
- http:
extended: yes # enable this for extended logging information
# custom allows additional http fields to be included in eve-log
# the example below adds three additional fields when uncommented
#custom: [Accept-Encoding, Accept-Language, Authorization]
- dns:
# Use version 2 logging with the new format:
# dns answers will be logged in one single event
# rather than an event for each of the answers.
# Without setting a version the version
# will fallback to 1 for backwards compatibility.
version: 2
# Enable/disable this logger. Default: enabled.
#enabled: no
# Control logging of requests and responses:
# - requests: enable logging of DNS queries
# - responses: enable logging of DNS answers
# By default both requests and responses are logged.
#requests: no
#responses: no
# Format of answer logging:
# - detailed: array item per answer
# - grouped: answers aggregated by type
# Default: all
#answer-format: [detailed, grouped]
# Answer types to log.
# Default: all
#answer-types: [a, aaaa, cname, mx, ns, ptr, txt]
- dns:
# Version 1 (deprecated) DNS logger.
version: 1
enabled: no
# control logging of queries and answers
# default yes, no to disable
query: yes # enable logging of DNS queries
answer: yes # enable logging of DNS answers
# control which RR types are logged
# all enabled if custom not specified
#custom: [a, aaaa, cname, mx, ns, ptr, txt]
- tls:
extended: yes # enable this for extended logging information
# output TLS transaction where the session is resumed using a
# session id
#session-resumption: no
# custom allows to control which tls fields that are included
# in eve-log
#custom: [subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after, certificate, chain]
- files:
force-magic: no # force logging magic on all logged files
# force logging of checksums, available hash functions are md5,
# sha1 and sha256
#force-hash: [md5]
#- drop:
# alerts: yes # log alerts that caused drops
# flows: all # start or all: 'start' logs only a single drop
# # per flow direction. All logs each dropped pkt.
- smtp:
#extended: yes # enable this for extended logging information
# this includes: bcc, message-id, subject, x_mailer, user-agent
# custom fields logging from the list:
# reply-to, bcc, message-id, subject, x-mailer, user-agent, received,
# x-originating-ip, in-reply-to, references, importance, priority,
# sensitivity, organization, content-md5, date
#custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc]
# output md5 of fields: body, subject
# for the body you need to set app-layer.protocols.smtp.mime.body-md5
# to yes
#md5: [body, subject]
# NFS logging. Requires Rust.
- nfs
# IKEv2 logging. Requires Rust.
- ikev2
- ssh
- stats:
totals: yes # stats for all threads merged together
threads: no # per thread stats
deltas: no # include delta values
- dhcp:
# DHCP logging requires Rust.
enabled: @rust_config_enabled@
# When extended mode is on, all DHCP messages are logged
# with full detail. When extended mode is off (the
# default), just enough information to map a MAC address
# to an IP address is logged.
extended: no
# bi-directional flows
- flow
# uni-directional flows
#- netflow
# An event for logging metadata, specifically pktvars when
# they are set, but will also include the full metadata object.
#- metadata
For more advanced configuration options, see Eve JSON Output.
The format is documented in Eve JSON Format.
Alert output for use with Barnyard2 (unified2.alert)¶
This log format is a binary format compatible with the unified2 output of another popular IDS format and is designed for use with Barnyard2 or other tools that consume the unified2 log format.
By default a file with the given filename and a timestamp (unix epoch format) will be created until the file hits the configured size limit, then a new file, with a new timestamp will be created. It is the job of other tools, such as Barnyard2 to cleanup old unified2 files.
If the nostamp option is set the log file will not have a timestamp appended. The file will be re-opened on SIGHUP like other log files allowing external log rotation tools to work as expected. However, if the limit is reach the file will be deleted and re-opened.
This output supports IPv6 and IPv4 events.
- unified2-alert:
enabled: yes
# The filename to log to in the default log directory. A
# timestamp in unix epoch time will be appended to the filename
# unless nostamp is set to yes.
filename: unified2.alert
# File size limit. Can be specified in kb, mb, gb. Just a number
# is parsed as bytes.
#limit: 32mb
# By default unified2 log files have the file creation time (in
# unix epoch format) appended to the filename. Set this to yes to
# disable this behavior.
#nostamp: no
# Sensor ID field of unified2 alerts.
#sensor-id: 0
# Include payload of packets related to alerts. Defaults to true, set to
# false if payload is not required.
#payload: yes
# HTTP X-Forwarded-For support by adding the unified2 extra header or
# overwriting the source or destination IP address (depending on flow
# direction) with the one reported in the X-Forwarded-For HTTP header.
# This is helpful when reviewing alerts for traffic that is being reverse
# or forward proxied.
xff:
enabled: no
# Two operation modes are available, "extra-data" and "overwrite". Note
# that in the "overwrite" mode, if the reported IP address in the HTTP
# X-Forwarded-For header is of a different version of the packet
# received, it will fall-back to "extra-data" mode.
mode: extra-data
# Two proxy deployments are supported, "reverse" and "forward". In
# a "reverse" deployment the IP address used is the last one, in a
# "forward" deployment the first IP address is used.
deployment: reverse
# Header name where the actual IP address will be reported, if more
# than one IP address is present, the last IP address will be the
# one taken into consideration.
header: X-Forwarded-For
This alert output needs Barnyard2.
A line based log of HTTP requests (http.log)¶
This log keeps track of all HTTP-traffic events. It contains the HTTP request, hostname, URI and the User-Agent. This information will be stored in the http.log (default name, in the suricata log directory). This logging can also be performed through the use of the Eve-log capability.
Example of a HTTP-log line with non-extended logging:
07/01/2014-04:20:14.338309 vg.no [**] / [**] Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36 [**]
192.168.1.6:64685 -> 195.88.54.16:80
Example of a HTTP-log line with extended logging:
07/01/2014-04:21:06.994705 vg.no [**] / [**] Mozilla/5.0 (Macintosh; Intel Mac OS X 10_9_2)
AppleWebKit/537.36 (KHTML, like Gecko) Chrome/35.0.1916.114 Safari/537.36 [**] <no referer> [**]
GET [**] HTTP/1.1 [**] 301 => http://www.vg.no/ [**] 239 bytes [**] 192.168.1.6:64726 -> 195.88.54.16:80
- http-log: #The log-name.
enabled: yes #This log is enabled. Set 'no' to disable.
filename: http.log #The name of the file in the default logging directory.
append: yes/no #If this option is set to yes, the last filled http.log-file will not be
# overwritten while restarting Suricata.
extended: yes # If set to yes more information is written about the event.
A line based log of DNS queries and replies (dns.log)¶
This log keeps track of all DNS events (queries and replies). It contains the type of DNS activity that has been performed, the requested / replied domain name and relevant data suck as client, server, ttl, resource record data. This logging can also be performed through the use of the Eve-log capability which offers easier parsing.
Example of the appearance of a DNS log of a query with a preceding reply:
07/01/2014-04:07:08.768100 [**] Query TX 14bf [**] zeustracker.abuse.ch [**] A [**] 192.168.1.6:37681 -> 192.168.1.1:53
07/01/2014-04:07:08.768100 [**] Response TX 14bf [**] zeustracker.abuse.ch [**] A [**] TTL 60 [**] 205.188.95.206 [**] 192.168.1.1:53 -> 192.168.1.6:37681
Non-existant domains and other DNS errors are recorded by the text representation of the rcode field in the reply (see RFC1035 and RFC2136 for a list). In the example below a non-existent domain is resolved and the NXDOMAIN error logged:
02/25/2015-22:58:40.499385 [**] Query TX a3ce [**] nosuchdomainwfqwdqwdqw.com [**] A [**] 192.168.40.10:48361 -> 192.168.40.2:53
02/25/2015-22:58:40.499385 [**] Response TX a3ce [**] NXDOMAIN [**] 192.168.40.2:53 -> 192.168.40.10:48361
02/25/2015-22:58:40.499385 [**] Response TX a3ce [**] NXDOMAIN [**] 192.168.40.2:53 -> 192.168.40.10:48361
Configuration options:
- dns-log: # The log-name
enabled: yes # If this log is enabled. Set 'no' to disable
filename: dns.log # Name of this file this log is written to in the default logging directory
append: yes # If this option is set to yes, the (if any exists) dns.log file wil not be overwritten while restarting Suricata.
filetype: regular / unix_stream / unix_dgram
Packet log (pcap-log)¶
With the pcap-log option you can save all packets, that are registered by Suricata, in a log file named _log.pcap_. This way, you can take a look at all packets whenever you want. In the normal mode a pcap file is created in the default-log-dir. It can also be created elsewhere if a absolute path is set in the yaml-file.
The file that is saved in example the default -log-dir /var/log/suricata, can be be opened with every program which supports the pcap file format. This can be Wireshark, TCPdump, Suricata, Snort and many others.
The pcap-log option can be enabled and disabled.
There is a size limit for the pcap-log file that can be set. The default limit is 32 MB. If the log-file reaches this limit, the file will be rotated and a new one will be created. The pcap-log option has an extra functionality for “Sguil”:http://sguil.sourceforge.net/ that can be enabled in the ‘mode’ option. In the sguil mode the “sguil_base_dir” indicates the base directory. In this base dir the pcaps are created in a Sguil-specific directory structure that is based on the day:
$sguil_base_dir/YYYY-MM-DD/$filename.<timestamp>
If you would like to use Suricata with Sguil, do not forget to enable (and if necessary modify) the base dir in the suricata.yaml file. Remember that in the ‘normal’ mode, the file will be saved in default-log-dir or in the absolute path (if set).
The pcap files can be compressed before being written to disk by setting the compression option to lz4. This option is incompatible with sguil mode. Note: On Windows, this option increases disk I/O instead of reducing it. When using lz4 compression, you can enable checksums using the lz4-checksum option, and you can set the compression level lz4-level to a value between 0 and 16, where higher levels result in higher compression.
By default all packets are logged except:
- TCP streams beyond stream.reassembly.depth
- encrypted streams after the key exchange
- pcap-log:
enabled: yes
filename: log.pcap
# Limit in MB.
limit: 32
mode: sguil # "normal" (default) or sguil.
sguil_base_dir: /nsm_data/
Verbose Alerts Log (alert-debug.log)¶
This is a log type that gives supplementary information about an alert. It is particularly convenient for people who investigate false positives and who write signatures. However, it lowers the performance because of the amount of information it has to store.
- alert-debug: #The log-name.
enabled: no #This log is not enabled. Set 'yes' to enable.
filename: alert-debug.log #The name of the file in the default logging directory.
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
# overwritten while restarting Suricata.
Alert output to prelude (alert-prelude)¶
To be able to use this type, you have to connect with the prelude manager first.
Prelude alerts contain a lot of information and fields, including the IPfields in of the packet which triggered the alert. This information can be divided in three parts:
- The alert description (sensor name, date, ID (sid) of the rule, etc). This is always included
- The packets headers (almost all IP fields, TCP UDP etc. if relevant)
- A binary form of the entire packet.
Since the last two parts can be very big (especially since they are stored in the Prelude SQL database), they are optional and controlled by the two options ‘log_packet_header’ and ‘log_packet_content’. The default setting is to log the headers, but not the content.
The profile name is the name of the Prelude profile used to connect to the prelude manager. This profile must be registered using an external command (prelude-admin), and must match the uid/gid of the user that will run Suricata. The complete procedure is detailed in the Prelude Handbook.
- alert-prelude: #The log-name.
enabled: no #This log is not enabled. Set 'yes' to enable.
profile: suricata #The profile-name used to connect to the prelude manager.
log_packet_content: no #The log_packet_content is disabled by default.
log_packet_header: yes #The log _packet_header is enabled by default.
Stats¶
In stats you can set the options for stats.log. When enabling stats.log you can set the amount of time in seconds after which you want the output-data to be written to the log file.
- stats:
enabled: yes #By default, the stats-option is enabled
filename: stats.log #The log-name. Combined with the default logging directory
#(default-log-dir) it will result in /var/log/suricata/stats.log.
#This directory can be overruled with a absolute path. (A
#directory starting with / ).
interval: 8 #The default amount of time after which the file will be
#refreshed.
append: yes/no #If this option is set to yes, the last filled fast.log-file will not be
#overwritten while restarting Suricata.
Syslog¶
With this option it is possible to send all alert and event output to syslog.
- syslog: #This is a output-module to direct log-output to several directions.
enabled: no #The use of this output-module is not enabled.
facility: local5 #In this option you can set a syslog facility.
level: Info #In this option you can set the level of output. The possible levels are:
#Emergency, Alert, Critical, Error, Warning, Notice, Info and Debug.
Drop.log, a line based information for dropped packets¶
If Suricata works in IPS mode, it can drop packets based on rules. Packets that are being dropped are saved in the drop.log file, a Netfilter log format.
- drop:
enabled: yes #The option is enabled.
filename: drop.log #The log-name of the file for dropped packets.
append: yes #If this option is set to yes, the last filled drop.log-file will not be
#overwritten while restarting Suricata. If set to 'no' the last filled drop.log file will be overwritten.
File-store (File Extraction)¶
The file-store output enables storing of extracted files to disk and configures where they are stored.
The following shows the configuration options for version 2 of the file-store output.
- file-store:
# This configures version 2 of the file-store.
version: 2
enabled: no
# Set the directory for the filestore. If the path is not
# absolute will be be relative to the default-log-dir.
#dir: filestore
# Write out a fileinfo record for each occurrence of a
# file. Disabled by default as each occurrence is already logged
# as a fileinfo record to the main eve-log.
#write-fileinfo: yes
# Force storing of all files. Default: no.
#force-filestore: yes
# Override the global stream-depth for sessions in which we want
# to perform file extraction. Set to 0 for unlimited.
#stream-depth: 0
# Uncomment the following variable to define how many files can
# remain open for filestore by Suricata. Default value is 0 which
# means files get closed after each write
#max-open-files: 1000
# Force logging of checksums, available hash functions are md5,
# sha1 and sha256. Note that SHA256 is automatically forced by
# the use of this output module as it uses the SHA256 as the
# file naming scheme.
#force-hash: [sha1, md5]
Detection engine¶
Inspection configuration¶
The detection-engine builds internal groups of signatures. Suricata loads signatures, with which the network traffic will be compared. The fact is, that many rules certainly will not be necessary. (For instance: if there appears a packet with the UDP-protocol, all signatures for the TCP-protocol won’t be needed.) For that reason, all signatures will be divided in groups. However, a distribution containing many groups will make use of a lot of memory. Not every type of signature gets its own group. There is a possibility that different signatures with several properties in common, will be placed together in a group. The quantity of groups will determine the balance between memory and performance. A small amount of groups will lower the performance yet uses little memory. The opposite counts for a higher amount of groups. The engine allows you to manage the balance between memory and performance. To manage this, (by determining the amount of groups) there are several general options: high for good performance and more use of memory, low for low performance and little use of memory. The option medium is the balance between performance and memory usage. This is the default setting. The option custom is for advanced users. This option has values which can be managed by the user.
detect:
profile: medium
custom-values:
toclient-groups: 2
toserver-groups: 25
sgh-mpm-context: auto
inspection-recursion-limit: 3000
At all of these options, you can add (or change) a value. Most signatures have the adjustment to focus on one direction, meaning focusing exclusively on the server, or exclusively on the client.
If you take a look at example 4, the Detection-engine grouping tree, you see it has many branches. At the end of each branch, there is actually a ‘sig group head’. Within that sig group head there is a container which contains a list with signatures that are significant for that specific group/that specific end of the branch. Also within the sig group head the settings for Multi-Pattern-Matcher (MPM) can be found: the MPM-context.
As will be described again at the part ‘Pattern matching settings’, there are several MPM-algorithms of which can be chosen from. Because every sig group head has its own MPM-context, some algorithms use a lot of memory. For that reason there is the option sgh-mpm-context to set whether the groups share one MPM-context, or to set that every group has its own MPM-context.
For setting the option sgh-mpm-context, you can choose from auto, full or single. The default setting is ‘auto’, meaning Suricata selects full or single based on the algorithm you use. ‘Full’ means that every group has its own MPM-context, and ‘single’ that all groups share one MPM-context. The two algorithms ac and ac-gfbs are new in 1.03. These algorithms use a single MPM-context if the Sgh-MPM-context setting is ‘auto’. The rest of the algorithms use full in that case.
The inspection-recursion-limit option has to mitigate that possible bugs in Suricata cause big problems. Often Suricata has to deal with complicated issues. It could end up in an ‘endless loop’ due to a bug, meaning it will repeat its actions over and over again. With the option inspection-recursion-limit you can limit this action.
Example 4 Detection-engine grouping tree
src Stands for source IP-address.
dst Stands for destination IP-address.
sp Stands for source port.
dp Stands for destination port.
Example 5 Detail grouping tree
Prefilter Engines¶
The concept of prefiltering is that there are far too many rules to inspect individually. The approach prefilter takes is that from each rule one condition is added to prefilter, which is then checked in one step. The most common example is MPM (also known as fast_pattern). This takes a single pattern per rule and adds it to the MPM. Only for those rules that have at least one pattern match in the MPM stage, individual inspection is performed.
Next to MPM, other types of keywords support prefiltering. ICMP itype, icode, icmp_seq and icmp_id for example. TCP window, IP TTL are other examples.
For a full list of keywords that support prefilter, see:
suricata --list-keywords=all
Suricata can automatically select prefilter options, or it can be set manually.
detect:
prefilter:
default: mpm
By default, only MPM/fast_pattern is used.
The prefilter engines for other non-MPM keywords can then be enabled in specific rules by using the ‘prefilter’ keyword.
E.g.
alert ip any any -> any any (ttl:123; prefilter; sid:1;)
To let Suricata make these decisions set default to ‘auto’:
detect:
prefilter:
default: auto
Pattern matcher settings¶
The multi-pattern-matcher (MPM) is a part of the detection engine within Suricata that searches for multiple patterns at once. Generally, signatures have one ore more patterns. Of each signature, one pattern is used by the multi-pattern-matcher. That way Suricata can exclude many signatures from being examined, because a signature can only match when all its patterns match.
These are the proceedings:
1)A packet comes in.
- 2)The packed will be analyzed by the Multi-pattern-matcher in search
- of patterns that match.
3)All patterns that match, will be further processed by Suricata (signatures).
Example 8 Multi-pattern-matcher
Suricata offers various implementations of different multi-pattern-matcher algorithm’s. These can be found below.
To set the multi-pattern-matcher algorithm:
mpm-algo: b2gc
After ‘mpm-algo’, you can enter one of the following algorithms: b2g, b2gc, b2gm, b3g, wumanber, ac and ac-gfbs (These last two are new in 1.0.3). For more information about these last two, please read again the the end of the part ‘Detection engine’. These algorithms have no options, so the fact that below there is no option being mentioned is no omission.
Subsequently, you can set the options for the mpm-algorithm’s.
The hash_size option determines the size of the hash-table that is internal used by the pattern matcher. A low hash-size (small table) causes lower memory usage, but decreases the performance. The opposite counts for a high hash-size: higher memory usage, but (generally) higher performance. The memory settings for hash size of the algorithms can vary from lowest (2048) - low (4096) - medium (8192) - high (16384) - higher (32768) – max (65536). (Higher is ‘highest’ in YAML 1.0 -1.0.2)
The bf_size option determines the size of the bloom filter, that is used with the final step of the pattern matcher, namely the validation of the pattern. For this option the same counts as for the hash-size option: setting it to low will cause lower memory usage, but lowers the performance. The opposite counts for a high setting of the bf_size: higher memory usage, but (generally) higher performance. The bloom-filter sizes can vary from low (512) - medium (1024) - high (2048).
pattern-matcher:
- b2gc:
search_algo: B2gSearchBNDMq
hash_size: low #Determines the size of the hash-table.
bf_size: medium #Determines the size of the bloom- filter.
- b3g:
search_algo: B3gSearchBNDMq
hash_size: low #See hash-size -b2gc.
bf_size: medium #See bf-size -b2gc.
- wumanber:
hash_size: low #See hash-size -b2gc.
bf_size: medium #See bf-size -b2gc.
Threading¶
Suricata is multi-threaded. Suricata uses multiple CPU’ s/CPU cores so it can process a lot of network packets simultaneously. (In a single-core engine, the packets will be processed one at a time.)
There are four thread-modules: Packet acquisition, decode and stream application layer, detection, and outputs.
# The packet acquisition module reads packets from the network.
# The decode module decodes the packets and the stream application application layer has three tasks:
First: it performs stream-tracking, meaning it is making sure all steps will be taken to make a correct network-connection.
Second: TCP-network traffic comes in as packets. The Stream-Assembly engine reconstructs the original stream.
Finally: the application layer will be inspected. HTTP and DCERPC will be analyzed.
# The detection threads will compare signatures. There can be several detection threads so they can operate simultaneously.
# In Outputs all alerts and events will be processed.
Example 6 Threading
Packet acquisition: Reads packets from the network
Decode: Decodes packets.
Stream app. Layer: Performs stream-tracking and reassembly.
Detect: Compares signatures.
Outputs: Processes all events and alerts.
Most computers have multiple CPU’s/ CPU cores. By default the operating system determines which core works on which thread. When a core is already occupied, another one will be designated to work on the thread. So, which core works on which thread, can differ from time to time.
There is an option within threading:
set-cpu-affinity: no
With this option you can cause Suricata setting fixed cores for every thread. In that case 1, 2 and 4 are at core 0 (zero). Each core has its own detect thread. The detect thread running on core 0 has a lower priority than the other threads running on core 0. If these other cores are to occupied, the detect thread on core 0 has not much packets to process. The detect threads running on other cores will process more packets. This is only the case after setting the option to ‘yes’.
Example 7 Balancing workload
You can set the detect-thread-ratio:
detect-thread-ratio: 1.5
The detect thread-ratio will determine the amount of detect threads. By default it will be 1.5 x the amount of CPU’s/CPU cores present at your computer. This will result in having more detection threads then CPU’s/ CPU cores. Meaning you are oversubscribing the amount of cores. This may be convenient at times when there have to be waited for a detection thread. The remaining detection thread can become active.
In the option ‘cpu affinity’ you can set which CPU’s/cores work on which thread. In this option there are several sets of threads. The management-, receive-, worker- and verdict-set. These are fixed names and can not be changed. For each set there are several options: cpu, mode, and prio. In the option ‘cpu’ you can set the numbers of the CPU’s/cores which will run the threads from that set. You can set this option to ‘all’, use a range (0-3) or a comma separated list (0,1). The option ‘mode’ can be set to ‘balanced’ or ‘exclusive’. When set to ‘balanced’, the individual threads can be processed by all cores set in the option ‘cpu’. If the option ‘mode’ is set to ‘exclusive’, there will be fixed cores for each thread. As mentioned before, threads can have different priority’s. In the option ‘prio’ you can set a priority for each thread. This priority can be low, medium, high or you can set the priority to ‘default’. If you do not set a priority for a CPU, than the settings in ‘default’ will count. By default Suricata creates one ‘detect’ (worker) thread per available CPU/CPU core.
cpu-affinity:
- management-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- receive-cpu-set:
cpu: [ 0 ] # include only these cpus in affinity settings
- worker-cpu-set:
cpu: [ "all" ]
mode: "exclusive"
# Use explicitely 3 threads and don't compute number by using
# detect-thread-ratio variable:
# threads: 3
prio:
low: [ 0 ]
medium: [ "1-2" ]
high: [ 3 ]
default: "medium"
- verdict-cpu-set:
cpu: [ 0 ]
prio:
default: "high"
Relevant cpu-affinity settings for IDS/IPS modes¶
IDS mode¶
Runmode AutoFp:
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
receive-cpu-set - used for receive and decode
worker-cpu-set - used for streamtcp,detect,output(logging),reject
Rumode Workers:
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
worker-cpu-set - used for receive,streamtcp,decode,detect,output(logging),respond/reject
IPS mode¶
Runmode AutoFp:
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
receive-cpu-set - used for receive and decode
worker-cpu-set - used for streamtcp,detect,output(logging)
verdict-cpu-set - used for verdict and respond/reject
Runmode Workers:
management-cpu-set - used for management (example - flow.managers, flow.recyclers)
worker-cpu-set - used for receive,streamtcp,decode,detect,output(logging),respond/reject, verdict
IP Defrag¶
Occasionally network packets appear fragmented. On some networks it occurs more often than on others. Fragmented packets exist of many parts. Before Suricata is able to inspect these kind of packets accurately, the packets have to be reconstructed. This will be done by a component of Suricata; the defragment-engine. After a fragmented packet is reconstructed by the defragment-engine, the engine sends on the reassembled packet to rest of Suricata.
There are three options within defrag: max-frags, prealloc and timeout. At the moment Suricata receives a fragment of a packet, it keeps in memory that other fragments of that packet will appear soon to complete the packet. However, there is a possibility that one of the fragments does not appear. To prevent Suricata for keeping waiting for that packet (thereby using memory) there is a timespan after which Suricata discards the fragments. This occurs by default after 60 seconds.
defrag:
max-frags: 65535
prealloc: yes
timeout: 60
Flow and Stream handling¶
Flow Settings¶
Within Suricata, Flows are very important. They play a big part in the way Suricata organizes data internally. A flow is a bit similar to a connection, except a flow is more general. All packets having the same Tuple (protocol, source IP, destination IP, source-port, destination-port), belong to the same flow. Packets belonging to a flow are connected to it internally.
Example 9 Flow
Example 10 Tuple
Keeping track of all these flows, uses memory. The more flows, the more memory it will cost.
To keep control over memory usage, there are several options:
The option memcap for setting the maximum amount of bytes the flow-engine will use, hash-size for setting the size of the hash-table and prealloc for the following:
For packets not yet belonging to a flow, Suricata creates a new flow. This is a relative expensive action. The risk coming with it, is that attackers /hackers can a attack the engine system at this part. When they make sure a computer gets a lot of packets with different tuples, the engine has to make a lot of new flows. This way, an attacker could flood the system. To mitigate the engine from being overloaded, this option instructs Suricata to keep a number of flows ready in memory. This way Suricata is less vulnerable to these kind of attacks.
The flow-engine has a management thread that operates independent from the packet processing. This thread is called the flow-manager. This thread ensures that wherever possible and within the memcap. there will be 10000 flows prepared.
flow:
memcap: 33554432 #The maximum amount of bytes the flow-engine will make use of.
hash_size: 65536 #Flows will be organized in a hash-table. With this option you can set the
#size of the hash-table.
Prealloc: 10000 #The amount of flows Suricata has to keep ready in memory.
At the point the memcap will still be reached, despite prealloc, the flow-engine goes into the emergency-mode. In this mode, the engine will make use of shorter time-outs. It lets flows expire in a more aggressive manner so there will be more space for new Flows.
There are two options: emergency_recovery and prune_flows. The emergency recovery is set on 30. This is the percentage of prealloc’d flows after which the flow-engine will be back to normal (when 30 percent of the 10000 flows is completed).
If during the emergency-mode, the aggressive time-outs do not have the desired result, this option is the final resort. It ends some flows even if they have not reached their time-outs yet. The prune-flows option shows how many flows there will be terminated at each time a new flow is set up.
emergency_recovery: 30 #Percentage of 1000 prealloc'd flows.
prune_flows: 5 #Amount of flows being terminated during the emergency mode.
Flow Time-Outs¶
The amount of time Suricata keeps a flow in memory is determined by the Flow time-out.
There are different states in which a flow can be. Suricata distinguishes three flow-states for TCP and two for UDP. For TCP, these are: New, Established and Closed,for UDP only new and established. For each of these states Suricata can employ different timeouts.
The state new in a TCP-flow, means the period during the three way handshake. The state established is the state when the three way handshake is completed. The state closed in the TCP-flow: there a several ways to end a flow. This is by means of Reset or the Four-way FIN handshake.
New in a UDP-flow: the state in which packets are send from only one direction.
Established in a UDP-flow: packets are send from both directions.
In the example configuration the are settings for each protocol. TCP, UDP, ICMP and default (all other protocols).
flow-timeouts:
default:
new: 30 #Time-out in seconds after the last activity in this flow in a New state.
established: 300 #Time-out in seconds after the last activity in this flow in a Established
#state.
emergency_new: 10 #Time-out in seconds after the last activity in this flow in a New state
#during the emergency mode.
emergency_established: 100 #Time-out in seconds after the last activity in this flow in a Established
#state in the emergency mode.
tcp:
new: 60
established: 3600
closed: 120
emergency_new: 10
emergency_established: 300
emergency_closed: 20
udp:
new: 30
established: 300
emergency_new: 10
emergency_established: 100
icmp:
new: 30
established: 300
emergency_new: 10
emergency_established: 100
Stream-engine¶
The Stream-engine keeps track of the TCP-connections. The engine exists of two parts: The stream tracking- and the reassembly-engine.
The stream-tracking engine monitors the state of a connection. The reassembly-engine reconstructs the flow as it used to be, so it will be recognized by Suricata.
The stream-engine has two memcaps that can be set. One for the stream-tracking-engine and one for the reassembly-engine.
The stream-tracking-engine keeps information of the flow in memory. Information about the state, TCP-sequence-numbers and the TCP window. For keeping this information, it can make use of the capacity the memcap allows.
TCP packets have a so-called checksum. This is an internal code which makes it possible to see if a packet has arrived in a good state. The stream-engine will not process packets with a wrong checksum. This option can be set off by entering ‘no’ instead of ‘yes’.
stream:
memcap: 64mb # Max memory usage (in bytes) for TCP session tracking
checksum_validation: yes # Validate packet checksum, reject packets with invalid checksums.
To mitigate Suricata from being overloaded by fast session creation, the option prealloc_sessions instructs Suricata to keep a number of sessions ready in memory.
A TCP-session starts with the three-way-handshake. After that, data can be send en received. A session can last a long time. It can happen that Suricata will be started after a few TCP sessions have already been started. This way, Suricata misses the original setup of those sessions. This setup always includes a lot of information. If you want Suricata to check the stream from that time on, you can do so by setting the option ‘midstream’ to ‘true’. The default setting is ‘false’. Normally Suricata is able to see all packets of a connection. Some networks make it more complicated though. Some of the network-traffic follows a different route than the other part, in other words: the traffic goes asynchronous. To make sure Suricata will check the one part it does see, instead of getting confused, the option ‘async-oneside’ is brought to life. By default the option is set to ‘false’.
Suricata inspects content in the normal/IDS mode in chunks. In the inline/IPS mode it does that on the sliding window way (see example ..) In the case Suricata is set in inline mode, it has to inspect packets immediately before sending it to the receiver. This way Suricata is able to drop a packet directly if needed.(see example …) It is important for Suricata to note which operating system it is dealing with, because operating systems differ in the way they process anomalies in streams. See Host-os-policy.
prealloc_sessions: 32768 # 32k sessions prealloc'd
midstream: false # do not allow midstream session pickups
async_oneside: false # do not enable async stream handling
inline: no # stream inline mode
drop-invalid: yes # drop invalid packets
The ‘drop-invalid’ option can be set to no to avoid blocking packets that are seen invalid by the streaming engine. This can be useful to cover some weird cases seen in some layer 2 IPS setup.
Example 11 Normal/IDS mode
Suricata inspects traffic in chunks.
Example 12 Inline/IPS Sliding Window
Suricata inspects traffic in a sliding window manner.
Example 13 Normal/IDS (reasembly on ACK’D data)
Example 14 Inline/IPS (reassembly on UNACK’D data)
The reassembly-engine has to keep data segments in memory in order to be able to reconstruct a stream. To avoid resource starvation a memcap is used to limit the memory used.
Reassembling a stream is an expensive operation. With the option depth you can control how far into a stream reassembly is done. By default this is 1MB. This setting can be overridden per stream by the protocol parsers that do file extraction.
Inspection of reassembled data is done in chunks. The size of these
chunks is set with toserver_chunk_size and toclient_chunk_size.
To avoid making the borders predictable, the sizes van be varied by
adding in a random factor.
reassembly:
memcap: 256mb # Memory reserved for stream data reconstruction (in bytes)
depth: 1mb # The depth of the reassembling.
toserver_chunk_size: 2560 # inspect raw stream in chunks of at least this size
toclient_chunk_size: 2560 # inspect raw stream in chunks of at least
randomize-chunk-size: yes
#randomize-chunk-range: 10
‘Raw’ reassembly is done for inspection by simple content, pcre
keywords use and other payload inspection not done on specific protocol
buffers like http_uri. This type of reassembly can be turned off:
reassembly:
raw: no
Incoming segments are stored in a list in the stream. To avoid constant memory allocations a per-thread pool is used.
reassembly:
segment-prealloc: 2048 # pre-alloc 2k segments per thread
Resending different data on the same sequence number is a way to confuse network inspection.
reassembly:
check-overlap-different-data: true
Example 15 Stream reassembly
Application Layer Parsers¶
Asn1_max_frames (new in 1.0.3 and 1.1)¶
Asn1 (Abstract Syntax One) is a standard notation to structure and describe data.
Within Asn1_max_frames there are several frames. To protect itself, Suricata will inspect a maximum of 256. You can set this amount differently if wanted.
Application layer protocols such as X.400 electronic mail, X.500 and LDAP directory services, H.323 (VoIP), BACnet and SNMP, use ASN.1 to describe the protocol data units (PDUs) they exchange. It is also extensively used in the Access and Non-Access Strata of UMTS.
Limit for the maximum number of asn1 frames to decode (default 256):
asn1_max_frames: 256
Configure HTTP (libhtp)¶
The library Libhtp is being used by Suricata to parse HTTP-sessions.
While processing HTTP-traffic, Suricata has to deal with different kind of servers which each process anomalies in HTTP-traffic differently. The most common web-server is Apache. This is a open source web -server program.
Beside Apache, IIS (Internet Information Services/Server)a web-server program of Microsoft is also well-known.
Like with host-os-policy, it is important for Suricata to which IP-address/network-address is used by which server. In Libhtp this assigning of web-servers to IP-and network addresses is called personality.
Currently Available Personalities:
- Minimal
- Generic
- IDS (default)
- IIS_4_0
- IIS_5_0
- IIS_5_1
- IIS_6_0
- IIS_7_0
- IIS_7_5
- Apache
- Apache_2_2
You can assign names to each block of settings. Which in this case is -apache and -iis7. Under these names you can set IP-addresses, network-addresses the personality and the request-body-limit.
The version-specific personalities know exactly how web servers behave, and emulate that. The IDS personality (will be GENERIC in the future) would try to implement a best-effort approach that would work reasonably well in the cases where you do not know the specifics.
The default configuration also applies to every IP-address for which no specific setting is available.
HTTP request body’s are often big, so they take a lot of time to process which has a significant impact on the performance. With the option ‘request-body-limit’ you can set the limit (in bytes) of the client-body that will be inspected. Setting it to 0 will inspect all of the body.
HTTP response body’s are often big, so they take a lot of time to process which has a significant impact on the performance. With the option ‘response-body-limit’ you can set the limit (in bytes) of the server-body that will be inspected. Setting it to 0 will inspect all of the body.
libhtp:
default-config:
personality: IDS
request-body-limit: 3072
response-body-limit: 3072
server-config:
- apache:
address: [192.168.1.0/24, 127.0.0.0/8, "::1"]
personality: Apache_2_2
request-body-limit: 0
response-body-limit: 0
- iis7:
address:
- 192.168.0.0/24
- 192.168.10.0/24
personality: IIS_7_0
request-body-limit: 4096
response-body-limit: 8192
As of 1.4, Suricata makes available the whole set of libhtp customisations for its users.
You can now use these parameters in the conf to customise suricata’s use of libhtp.
# Configures whether backslash characters are treated as path segment
# separators. They are not on Unix systems, but are on Windows systems.
# If this setting is enabled, a path such as "/one\two/three" will be
# converted to "/one/two/three". Accepted values - yes, no.
#path-backslash-separators: yes
# Configures whether consecutive path segment separators will be
# compressed. When enabled, a path such as "/one//two" will be normalized
# to "/one/two". The backslash_separators and decode_separators
# parameters are used before compression takes place. For example, if
# backslash_separators and decode_separators are both enabled, the path
# "/one\\/two\/%5cthree/%2f//four" will be converted to
# "/one/two/three/four". Accepted values - yes, no.
#path-compress-separators: yes
# This parameter is used to predict how a server will react when control
# characters are present in a request path, but does not affect path
# normalization. Accepted values - none or status_400 */
#path-control-char-handling: none
# Controls the UTF-8 treatment of request paths. One option is to only
# validate path as UTF-8. In this case, the UTF-8 flags will be raised
# as appropriate, and the path will remain in UTF-8 (if it was UTF-8 in
# the first place). The other option is to convert a UTF-8 path into a
# single byte stream using best-fit mapping. Accepted values - yes, no.
#path-convert-utf8: yes
# Configures whether encoded path segment separators will be decoded.
# Apache does not do this, but IIS does. If enabled, a path such as
# "/one%2ftwo" will be normalized to "/one/two". If the
# backslash_separators option is also enabled, encoded backslash
# characters will be converted too (and subsequently normalized to
# forward slashes). Accepted values - yes, no.
#path-decode-separators: yes
# Configures whether %u-encoded sequences in path will be decoded. Such
# sequences will be treated as invalid URL encoding if decoding is not
# desireable. Accepted values - yes, no.
#path-decode-u-encoding: yes
# Configures how server reacts to invalid encoding in path. Accepted
# values - preserve_percent, remove_percent, decode_invalid, status_400
#path-invalid-encoding-handling: preserve_percent
# Configures how server reacts to invalid UTF-8 characters in path.
# This setting will not affect path normalization; it only controls what
# response status we expect for a request that contains invalid UTF-8
# characters. Accepted values - none, status_400.
#path-invalid-utf8-handling: none
# Configures how server reacts to encoded NUL bytes. Some servers will
# terminate path at NUL, while some will respond with 400 or 404. When
# the termination option is not used, the NUL byte will remain in the
# path. Accepted values - none, terminate, status_400, status_404.
# path-nul-encoded-handling: none
# Configures how server reacts to raw NUL bytes. Some servers will
# terminate path at NUL, while some will respond with 400 or 404. When
# the termination option is not used, the NUL byte will remain in the
# path. Accepted values - none, terminate, status_400, status_404.
path-nul-raw-handling: none
# Sets the replacement character that will be used to in the lossy
# best-fit mapping from Unicode characters into single-byte streams.
# The question mark is the default replacement character.
#set-path-replacement-char: ?
# Controls what the library does when it encounters an Unicode character
# where only a single-byte would do (e.g., the %u-encoded characters).
# Conversion always takes place; this parameter is used to correctly
# predict the status code used in response. In the future there will
# probably be an option to convert such characters to UCS-2 or UTF-8.
# Accepted values - bestfit, status_400 and status_404.
#set-path-unicode-mapping: bestfit
Configure SMB (Rust)¶
注解
for full SMB support compile Suricata with Rust support
The SMB parser will parse version 1, 2 and 3 of the SMB protocol over TCP.
To enable the parser add the following to the app-layer section of the YAML.
smb:
enabled: yes
detection-ports:
dp: 139, 445
The parser uses pattern based protocol detection and will fallback to probing parsers
if the pattern based detection fails. As usual, the pattern based detection is port
independent. The probing parsers will only run on the detection-ports.
SMB is commonly used to transfer the DCERPC protocol. This traffic is also handled by this parser.
Engine output¶
Logging configuration¶
The logging subsystem can display all output except alerts and events. It gives information at runtime about what the engine is doing. This information can be displayed during the engine startup, at runtime and while shutting the engine down. For informational messages, errors, debugging, etc.
The log-subsystem has several log levels:
Error, warning, informational and debug. Note that debug level logging will only be emitted if Suricata was compiled with the –enable-debug configure option.
The first option within the logging configuration is the default-log-level. This option determines the severity/importance level of information that will be displayed. Messages of lower levels than the one set here, will not be shown. The default setting is Info. This means that error, warning and info will be shown and the other levels won’t be.
There are more levels: emergency, alert, critical and notice, but those are not used by Suricata yet. This option can be changed in the configuration, but can also be overridden in the command line by the environment variable: SC_LOG_LEVEL .
logging:
default-log-level: info
Default log format¶
A logging line exists of two parts. First it displays meta information (thread id, date etc.), and finally the actual log message. Example:
[27708] 15/10/2010 -- 11:40:07 - (suricata.c:425) <Info> (main) – This is Suricata version 1.0.2
(Here the part until the – is the meta info, “This is Suricata 1.0.2” is the actual message.)
It is possible to determine which information will be displayed in this line and (the manner how it will be displayed) in which format it will be displayed. This option is the so called format string:
default-log-format: "[%i] %t - (%f:%l) <%d> (%n) -- "
The % followed by a character, has a special meaning. There are eight specified signs:
t: Time, timestamp, time and date
example: 15/10/2010 - -11:40:07
p: Process ID. Suricata's whole processing consists of multiple threads.
i: Thread ID. ID of individual threads.
m: Thread module name. (Outputs, Detect etc.)
d: Log-level of specific log-event. (Error, info, debug etc.)
f: Filename. Name of C-file (source code) where log-event is generated.
l: Line-number within the filename, where the log-event is generated in the source-code.
n: Function-name in the C-code (source code).
The last three, f, l and n are mainly convenient for developers.
The log-format can be overridden in the command line by the environment variable: SC_LOG_FORMAT
Output-filter¶
Within logging you can set an output-filter. With this output-filter you can set which part of the event-logs should be displayed. You can supply a regular expression (Regex). A line will be shown if the regex matches.
default-output-filter: #In this option the regular expression can be entered.
This value is overridden by the environment var: SC_LOG_OP_FILTER
Outputs¶
There are different ways of displaying output. The output can appear directly on your screen, it can be placed in a file or via syslog. The last mentioned is an advanced tool for log-management. The tool can be used to direct log-output to different locations (files, other computers etc.)
outputs:
- console: #Output on your screen.
enabled: yes #This option is enabled.
- file: #Output stored in a file.
enabled: no #This option is not enabled.
filename: /var/log/suricata.log #Filename and location on disc.
- syslog: #This is a program to direct log-output to several directions.
enabled: no #The use of this program is not enabled.
facility: local5 #In this option you can set a syslog facility.
format: "[%i] <%d> -- " #The option to set your own format.
Packet Acquisition¶
Pf-ring¶
The Pf_ring is a library that aims to improve packet capture performance over libcap. It performs packet acquisition. There are three options within Pf_ring: interface, cluster-id and cluster-type.
pfring:
interface: eth0 # In this option you can set the network-interface
# on which you want the packets of the network to be read.
Pf_ring will load balance packets based on flow. All packet acquisition threads that will participate in the load balancing need to have the same cluster-id. It is important to make sure this ID is unique for this cluster of threads, so that no other engine / program is making use of clusters with the same id.
cluster-id: 99
Pf_ring can load balance traffic using pf_ring-clusters. All traffic for pf_ring can be load balanced in one of two ways, in a round robin manner or a per flow manner that are part of the same cluster. All traffic for pf_ring will be load balanced across acquisition threads of the same cluster id.
The cluster_round_robin manner is a way of distributing packets one at a time to each thread (like distributing playing cards to fellow players). The cluster_flow manner is a way of distributing all packets of the same flow to the same thread. The flows itself will be distributed to the threads in a round-robin manner.
cluster-type: cluster_round_robin
NFQ¶
Using NFQUEUE in iptables rules, will send packets to Suricata. If the mode is set to ‘accept’, the packet that has been send to Suricata by a rule using NFQ, will by default not be inspected by the rest of the iptables rules after being processed by Suricata. There are a few more options to NFQ to change this if desired.
If the mode is set to ‘repeat’, the packets will be marked by Suricata and be re-injected at the first rule of iptables. To mitigate the packet from being going round in circles, the rule using NFQ will be skipped because of the mark.
If the mode is set to ‘route’, you can make sure the packet will be send to another tool after being processed by Suricata. It is possible to assign this tool at the mandatory option ‘route_queue’. Every engine/tool is linked to a queue-number. This number you can add to the NFQ rule and to the route_queue option.
Add the numbers of the options repeat_mark and route_queue to the NFQ-rule:
iptables -I FORWARD -m mark ! --mark $MARK/$MASK -j NFQUEUE
nfq:
mode: accept #By default the packet will be accepted or dropped by Suricata
repeat_mark: 1 #If the mode is set to 'repeat', the packets will be marked after being
#processed by Suricata.
repeat_mask: 1
route_queue: 2 #Here you can assign the queue-number of the tool that Suricata has to
#send the packets to after processing them.
Example 1 NFQ1
mode: accept
Example 2 NFQ
mode: repeat
Example 3 NFQ
mode: route
Ipfw¶
Suricata does not only support Linux, it supports the FreeBSD operating system (this is an open source Unix operating system) and Mac OS X as well. The in-line mode on FreeBSD uses ipfw (IP-firewall).
Certain rules in ipfw send network-traffic to Suricata. Rules have numbers. In this option you can set the rule to which the network-traffic will be placed back. Make sure this rule comes after the one that sends the traffic to Suricata, otherwise it will go around in circles.
The following tells the engine to re-inject packets back into the ipfw firewall at rule number 5500:
ipfw:
ipfw-reinjection-rule-number: 5500
Example 16 Ipfw-reinjection.
Rules¶
Rule-files¶
For different categories of risk there are different rule-files available containing one or more rules. There is a possibility to instruct Suricata where to find these rules and which rules you want to be load for use. You can set the directory where the files can be found.
default-rule-path: /etc/suricata/rules/
rule-files:
- backdoor.rules
- bad-traffic.rules
- chat.rules
- ddos.rules
- ....
The above mentioned is an example of rule-files of which can be chosen from. There are much more rule-files available.
If wanted, you can set a full path for a specific rule or rule-file. In that case, the above directory (/etc/suricata/rules/) will be ignored for that specific file. This is convenient in case you write your own rules and want to store them separate from other rules like that of VRT, ET or ET pro.
If you set a file-name that appears to be not existing, Suricata will ignore that entry and display a error-message during the engine startup. It will continue with the startup as usual.
Threshold-file¶
Within this option, you can state the directory in which the threshold-file will be stored. The default directory is: /etc/suricata/threshold.config
Classifications¶
The Classification-file is a file which makes the purpose of rules clear.
Some rules are just for providing information. Some of them are to warn you for serious risks like when you are being hacked etc.
In this classification-file, there is a part submitted to the rule to make it possible for the system-administrator to distinguish events.
A rule in this file exists of three parts: the short name, a description and the priority of the rule (in which 1 has the highest priority and 4 the lowest).
You can notice these descriptions returning in the rule and events / alerts.
Example:
configuration classification: misc-activity,Misc activity,3
Rule:
alert tcp $HOME_NET 21 -> $EXTERNAL_NET any (msg:"ET POLICY FTP Login Successful (non-anonymous)";
flow:from_server,established;flowbits:isset,ET.ftp.user.login; flowbits:isnotset,ftp.user.logged_in;
flowbits:set,ftp.user.logged_in; content:"230 ";pcre:!"/^230(\s+USER)?\s+(anonymous|ftp)/smi";
classtype:misc-activity; reference:urldoc.emergingthreats.net/2003410,;
reference:url,www.emergingthreats.net/cgi-bin/cvsweb.cgi/sigs/POLICY/POLICY_FTP_Login; sid:2003410; rev:7;)
Event/Alert:
10/26/10-10:13:42.904785 [**] [1:2003410:7] ET POLICY FTP Login Successful (non-anonymous) [**]
[Classification: Misc activity[Priority: 3] {TCP} 192.168.0.109:21 -> x.x.x.x:34117
You can set the direction of the classification configuration.
classification-file: /etc/suricata/classification.config
Rule-vars¶
There are variables which can be used in rules.
Within rules, there is a possibility to set for which IP-address the rule should be checked and for which IP-address it should not.
This way, only relevant rules will be used. To prevent you from having to set this rule by rule, there is an option in which you can set the relevant IP-address for several rules. This option contains the address group vars that will be passed in a rule. So, after HOME_NET you can enter your home IP-address.
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]" #By using [], it is possible to set
#complicated variables.
EXTERNAL_NET: any
HTTP_SERVERS: "$HOME_NET" #The $-sign tells that what follows is
#a variable.
SMTP_SERVERS: "$HOME_NET"
SQL_SERVERS: "$HOME_NET"
DNS_SERVERS: "$HOME_NET"
TELNET_SERVERS: "$HOME_NET"
AIM_SERVERS: any
It is a convention to use upper-case characters.
There are two kinds of variables: Address groups and Port-groups. They both have the same function: change the rule so it will be relevant to your needs.
In a rule there is a part assigned to the address and one to the port. Both have their variable.
All options have to be set. If it is not necessary to set a specific address, you should enter ‘any’.
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
ORACLE_PORTS: 1521
SSH_PORTS: 22
Host-os-policy¶
Operating systems differ in the way they process fragmented packets and streams. Suricata performs differently with anomalies for different operating systems. It is important to set of which operating system your IP-address makes use of, so Suricata knows how to process fragmented packets and streams. For example in stream-reassembly there can be packets with overlapping payloads.
Example 17 Overlapping payloads
In the configuration-file, the operating-systems are listed. You can add your IP-address behind the name of the operating system you make use of.
host-os-policy:
windows: [0.0.0.0/0]
bsd: []
bsd_right: []
old_linux: []
linux: [10.0.0.0/8, 192.168.1.100, "8762:2352:6241:7245:E000:0000:0000:0000"]
old_solaris: []
solaris: ["::1"]
hpux10: []
hpux11: []
irix: []
macos: []
vista: []
windows2k3: []
Engine analysis and profiling¶
Suricata offers several ways of analyzing performance of rules and the engine itself.
Engine-analysis¶
The option engine-analysis provides information for signature writers about how Suricata organizes signatures internally.
Like mentioned before, signatures have zero or more patterns on which they can match. Only one of these patterns will be used by the multi pattern matcher (MPM). Suricata determines which patterns will be used unless the fast-pattern rule option is used.
The option engine-analysis creates a new log file in the default log dir. In this file all information about signatures and patterns can be found so signature writers are able to see which pattern is used and change it if desired.
To create this log file, you have to run Suricata with ./src/suricata -c suricata.yaml –engine-analysis.
engine-analysis:
rules-fast-pattern: yes
Example:
[10703] 26/11/2010 -- 11:41:15 - (detect.c:560) <Info> (SigLoadSignatures)
-- Engine-Analyis for fast_pattern printed to file - /var/log/suricata/rules_fast_pattern.txt
== Sid: 1292 ==
Fast pattern matcher: content
Fast pattern set: no
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: Volume Serial Number
Final content: Volume Serial Number
---
alert tcp any any -> any any (content:"abc"; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: no
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: defghi
Final content: defghi
---
alert tcp any any -> any any (content:"abc"; fast_pattern:only; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: yes
Fast pattern chop set: no
Content negated: no
Original content: abc
Final content: abc
---
alert tcp any any -> any any (content:"abc"; fast_pattern; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: no
Content negated: no
Original content: abc
Final content: abc
---
alert tcp any any -> any any (content:"abc"; fast_pattern:1,2; content:"defghi"; sid:1;)
== Sid: 1 ==
Fast pattern matcher: content
Fast pattern set: yes
Fast pattern only set: no
Fast pattern chop set: yes
Fast pattern offset, length: 1, 2
Content negated: no
Original content: abc
Final content: bc
Rule and Packet Profiling settings¶
Rule profiling is a part of Suricata to determine how expensive rules are. Some rules are very expensive while inspecting traffic. Rule profiling is convenient for people trying to track performance problems and resolving them. Also for people writing signatures.
Compiling Suricata with rule-profiling will have an impact on performance, even if the option is disabled in the configuration file.
To observe the rule-performance, there are several options.
profiling:
rules:
enabled: yes
This engine is not used by default. It can only be used if Suricata is compiled with:
-- enable-profiling
At the end of each session, Suricata will display the profiling statistics. The list will be displayed sorted.
This order can be changed as pleased. The choice is between ticks, avgticks, checks, maxticks and matches. The setting of your choice will be displayed from high to low.
The amount of time it takes to check the signatures, will be administrated by Suricata. This will be counted in ticks. One tick is one CPU computation. 3 GHz will be 3 billion ticks.
Beside the amount of checks, ticks and matches it will also display the average and the maximum of a rule per session at the end of the line.
The option Limit determines the amount of signatures of which the statistics will be shown, based on the sorting.
sort: avgticks
limit: 100
Example of how the rule statistics can look like;
Rule Ticks % Checks Matches Max Tick Avg
Ticks
7560 107766621 0.02 138 37 105155334 780917.54
11963 1605394413 0.29 2623 1 144418923 612045.14
7040 1431034011 0.26 2500 0 106018209 572413.60
5726 1437574662 0.26 2623 1 115632900 548065.06
7037 1355312799 0.24 2562 0 116048286 529005.78
11964 1276449255 0.23 2623 1 96412347 486637.15
7042 1272562974 0.23 2623 1 96405993 485155.54
5719 1233969192 0.22 2562 0 106439661 481642.93
5720 1204053246 0.21 2562 0 125155431 469966.14
Packet Profiling¶
packets:
# Profiling can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: yes #this option is enabled by default
filename: packet_stats.log #name of the file in which packet profiling information will be
#stored.
append: yes #If set to yes, new packet profiling information will be added to the
#information that was saved last in the file.
# per packet csv output
csv:
# Output can be disabled here, but it will still have a
# performance impact if compiled in.
enabled: no #the sending of packet output to a csv-file is by default disabled.
filename: packet_stats.csv #name of the file in which csv packet profiling information will be
#stored
Packet profiling is enabled by default in suricata.yaml but it will only do its job if you compiled Suricata with –enable profiling.
The filename in which packet profiling information will be stored, is packet-stats.log. Information in this file can be added to the last information that was saved there, or if the append option is set to no, the existing file will be overwritten.
Per packet, you can send the output to a csv-file. This file contains one line for each packet with all profiling information of that packet. This option can be used only if Suricata is build with –enable-profiling and if the packet profiling option is enabled in yaml.
It is best to use runmode ‘single’ if you would like to profile the speed of the code. When using a single thread, there is no situation in which two threads have to wait for each other. When using two threads, the time threads might have to wait for each other will be taken in account when/during profiling packets. For more information see Packet Profiling.
Application layers¶
SSL/TLS¶
SSL/TLS parsers track encrypted SSLv2, SSLv3, TLSv1, TLSv1.1 and TLSv1.2 sessions.
Protocol detection is done using patterns and a probing parser running on only TCP/443 by default. The pattern based protocol detection is port independent.
tls:
enabled: yes
detection-ports:
dp: 443
# What to do when the encrypted communications start:
# - default: keep tracking TLS session, check for protocol anomalies,
# inspect tls_* keywords. Disables inspection of unmodified
# 'content' signatures.
# - bypass: stop processing this flow as much as possible. No further
# TLS parsing and inspection. Offload flow bypass to kernel
# or hardware if possible.
# - full: keep tracking and inspection as normal. Unmodified content
# keyword signatures are inspected as well.
#
# For best performance, select 'bypass'.
#
#encrypt-handling: default
Encrypted traffic¶
There is no decryption of encrypted traffic, so once the handshake is complete
continued tracking of the session is of limited use. The encrypt-handling
option controls the behavior after the handshake.
If encrypt-handling is set to default (or if the option is not set),
Suricata will continue to track the SSL/TLS session. Inspection will be limited,
as raw content inspection will still be disabled. There is no point in doing
pattern matching on traffic known to be encrypted. Inspection for (encrypted)
Heartbleed and other protocol anomalies still happens.
When encrypt-handling is set to bypass, all processing of this session is
stopped. No further parsing and inspection happens. If stream.bypass is enabled
this will lead to the flow being bypassed, either inside Suricata or by the
capture method if it supports it and is configured for it.
Finally, if encrypt-handling is set to full, Suricata will process the
flow as normal, without inspection limitations or bypass.
The option has replaced the no-reassemble option. If no-reassemble is
present, and encrypt-handling is not, false is intepreted as
encrypt-handling: default and true is interpreted as
encrypt-handling: bypass.
Modbus¶
According to MODBUS Messaging on TCP/IP Implementation Guide V1.0b, it is recommended to keep the TCP connection opened with a remote device and not to open and close it for each MODBUS/TCP transaction. In that case, it is important to set the stream-depth of the modbus as unlimited.
modbus:
# Stream reassembly size for modbus, default is 0
stream-depth: 0
Decoder¶
Teredo¶
The Teredo decoder can be disabled. It is enabled by default.
decoder:
# Teredo decoder is known to not be completely accurate
# it will sometimes detect non-teredo as teredo.
teredo:
enabled: true
Advanced Options¶
luajit¶
states¶
Luajit has a strange memory requirement, it’s ‘states’ need to be in the first 2G of the process’ memory. For this reason when luajit is used the states are allocated at the process startup. This option controls how many states are preallocated.
If the pool is depleted a warning is generated. Suricata will still try to continue, but may fail if other parts of the engine take too much memory. If the pool was depleted a hint will be printed at the engines exit.
States are allocated as follows: for each detect script a state is used per detect thread. For each output script, a single state is used. Keep in mind that a rule reload temporary doubles the states requirement.
Global-Thresholds¶
Thresholds can be configured in the rules themselves, see Thresholding Keywords. They are often set by rule writers based on their intelligence for creating a rule combined with a judgement on how often a rule will alert.
Threshold Config¶
Next to rule thresholding more thresholding can be configured on the sensor using the threshold.config.
threshold/event_filter¶
Syntax:
threshold gen_id <gid>, sig_id <sid>, type <threshold|limit|both>, \
track <by_src|by_dst>, count <N>, seconds <T>
rate_filter¶
Rate filters allow changing of a rule action when a rule matches.
Syntax:
rate_filter: rate_filter gen_id <gid>, sig_id <sid>, track <tracker>, \
count <c>, seconds <s>, new_action <action>, timeout <timeout>
Example:
rate_filter gen_id 1, sig_id 1000, track by_rule, count 100, seconds 60, \
new_action alert, timeout 30
gen_id¶
Generator id. Normally 1, but if a rule uses the gid keyword to set
another value it has to be matched in the gen_id.
sig_id¶
Rule/signature id as set by the rule sid keyword.
track¶
Where to track the rule matches. When using by_src/by_dst the tracking is done per IP-address. The Host table is used for storage. When using by_rule it’s done globally for the rule. Option by_both used to track per IP pair of source and destination. Packets going to opposite directions between same addresses tracked as the same pair.
count¶
Number of rule hits before the rate_filter is activated.
seconds¶
Time period within which the count needs to be reached to activate
the rate_filter
new_action¶
New action that is applied to matching traffic when the rate_filter
is in place.
Values:
<alert|drop|pass|reject>
Note: ‘sdrop’ and ‘log’ are supported by the parser but not implemented otherwise.
timeout¶
Time in seconds during which the rate_filter will remain active.
Example¶
Lets say we want to limit incoming connections to our SSH server. The rule
888 below simply alerts on SYN packets to the SSH port of our SSH server.
If an IP-address triggers this more than 10 or more with a minute, the
drop rate_filter is set with a timeout of 5 minutes.
Rule:
alert tcp any any -> $MY_SSH_SERVER 22 (msg:"Connection to SSH server"; \
flow:to_server; flags:S,12; sid:888;)
Rate filter:
rate_filter gen_id 1, sig_id 888, track by_src, count 10, seconds 60, \
new_action drop, timeout 300
suppress¶
Suppressions can be used to suppress alerts for a rule or a host/network. Actions performed when a rule matches, such as setting a flowbit, are still performed.
Syntax:
suppress gen_id <gid>, sig_id <sid>
suppress gen_id <gid>, sig_id <sid>, track <by_src|by_dst|by_either>, ip <ip|subnet|addressvar>
Examples:
suppress gen_id 1, sig_id 2002087, track by_src, ip 209.132.180.67
This will make sure the signature 2002087 will never match for src host 209.132.180.67.
Other possibilities/examples:
suppress gen_id 1, sig_id 2003614, track by_src, ip 217.110.97.128/25
suppress gen_id 1, sig_id 2003614, track by_src, ip [192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]
suppress gen_id 1, sig_id 2003614, track by_src, ip $HOME_NET
suppress gen_id 1, sig_id 2003614, track by_either, ip 217.110.97.128/25
In the last example above, the by_either tracking means that if either
the source ip or destination ip matches 217.110.97.128/25 the
rule with sid 2003614 is suppressed.
Global thresholds vs rule thresholds¶
Note: this section applies to 1.4+ In 1.3 and before mixing rule and global thresholds is not supported.
When a rule has a threshold/detection_filter set a rule can still be affected by the global threshold file.
The rule below will only fire if 10 or more emails are being delivered/sent from a host within 60 seconds.
alert tcp any any -> any 25 (msg:"ET POLICY Inbound Frequent Emails - Possible Spambot Inbound"; \
flow:established; content:"mail from|3a|"; nocase; \
threshold: type threshold, track by_src, count 10, seconds 60; \
reference:url,doc.emergingthreats.net/2002087; classtype:misc-activity; sid:2002087; rev:10;)
Next, we’ll see how global settings affect this rule.
Suppress¶
Suppressions can be combined with rules with thresholds/detection_filters with no exceptions.
suppress gen_id 1, sig_id 2002087, track by_src, ip 209.132.180.67
suppress gen_id 0, sig_id 0, track by_src, ip 209.132.180.67
suppress gen_id 1, sig_id 0, track by_src, ip 209.132.180.67
Each of the rules above will make sure 2002087 doesn’t alert when the source of the emails is 209.132.180.67. It will alert for all other hosts.
suppress gen_id 1, sig_id 2002087
This suppression will simply convert the rule to “noalert”, meaning it will never alert in any case. If the rule sets a flowbit, that will still happen.
Threshold/event_filter¶
When applied to a specific signature, thresholds and event_filters (threshold from now on) will override the signature setting. This can be useful for when the default in a signature doesn’t suit your evironment.
threshold gen_id 1, sig_id 2002087, type both, track by_src, count 3, seconds 5
threshold gen_id 1, sig_id 2002087, type threshold, track by_src, count 10, seconds 60
threshold gen_id 1, sig_id 2002087, type limit, track by_src, count 1, seconds 15
Each of these will replace the threshold setting for 2002087 by the new threshold setting.
Note: overriding all gids or sids (by using gen_id 0 or sig_id 0) is not supported. Bug https://redmine.openinfosecfoundation.org/issues/425.
Rate_filter¶
Snort.conf to Suricata.yaml¶
This guide is meant for those who are familiar with Snort and the snort.conf configuration format. This guide will provide a 1:1 mapping between Snort and Suricata configuration wherever possible.
Variables¶
snort.conf
ipvar HOME_NET any
ipvar EXTERNAL_NET any
...
portvar HTTP_PORTS [80,81,311,591,593,901,1220,1414,1741,1830,2301,2381,2809,3128,3702,4343,4848,5250,7001,7145,7510,7777,7779,8000,8008,8014,8028,8080,8088,8090,8118,8123,8180,8181,8243,8280,8800,8888,8899,9000,9080,9090,9091,9443,9999,11371,55555]
portvar SHELLCODE_PORTS !80
...
suricata.yaml
vars:
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
EXTERNAL_NET: "!$HOME_NET"
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
Note that Suricata can automatically detect HTTP traffic regardless of the port it uses. So the HTTP_PORTS variable is not nearly as important as it is with Snort, if you use a Suricata enabled ruleset.
Decoder alerts¶
snort.conf
# Stop generic decode events:
config disable_decode_alerts
# Stop Alerts on experimental TCP options
config disable_tcpopt_experimental_alerts
# Stop Alerts on obsolete TCP options
config disable_tcpopt_obsolete_alerts
# Stop Alerts on T/TCP alerts
config disable_tcpopt_ttcp_alerts
# Stop Alerts on all other TCPOption type events:
config disable_tcpopt_alerts
# Stop Alerts on invalid ip options
config disable_ipopt_alerts
suricata.yaml
Suricata has no specific decoder options. All decoder related alerts are controlled by rules. See #Rules below.
Checksum handling¶
snort.conf
config checksum_mode: all
suricata.yaml
Suricata’s checksum handling works on-demand. The stream engine checks TCP and IP checksum by default:
stream:
checksum-validation: yes # reject wrong csums
Alerting on bad checksums can be done with normal rules. See #Rules, decoder-events.rules specifically.
Various configs¶
Active response¶
snort.conf
# Configure active response for non inline operation. For more information, see REAMDE.active
# config response: eth0 attempts 2
suricata.yaml
Active responses are handled automatically w/o config if rules with the “reject” action are used.
Dropping privileges¶
snort.conf
# Configure specific UID and GID to run snort as after dropping privs. For more information see snort -h command line options
#
# config set_gid:
# config set_uid:
Suricata
To set the user and group use the –user <username> and –group <groupname> commandline options.
Snaplen¶
snort.conf
# Configure default snaplen. Snort defaults to MTU of in use interface. For more information see README
#
# config snaplen:
#
Suricata always works at full snap length to provide full traffic visibility.
Bpf¶
snort.conf
# Configure default bpf_file to use for filtering what traffic reaches snort. For more information see snort -h command line options (-F)
#
# config bpf_file:
#
suricata.yaml
BPF filters can be set per packet acquisition method, with the “bpf-filter: <file>” yaml option and in a file using the -F command line option.
For example:
pcap:
- interface: eth0
#buffer-size: 16777216
#bpf-filter: "tcp and port 25"
#checksum-checks: auto
#threads: 16
#promisc: no
#snaplen: 1518
Log directory¶
snort.conf
# Configure default log directory for snort to log to. For more information see snort -h command line options (-l)
#
# config logdir:
suricata.yaml
default-log-dir: /var/log/suricata/
This value is overridden by the -l commandline option.
Packet acquisition¶
snort.conf
# Configure DAQ related options for inline operation. For more information, see README.daq
#
# config daq: <type>
# config daq_dir: <dir>
# config daq_mode: <mode>
# config daq_var: <var>
#
# <type> ::= pcap | afpacket | dump | nfq | ipq | ipfw
# <mode> ::= read-file | passive | inline
# <var> ::= arbitrary <name>=<value passed to DAQ
# <dir> ::= path as to where to look for DAQ module so's
suricata.yaml
Suricata has all packet acquisition support built-in. It’s configuration format is very verbose.
pcap:
- interface: eth0
#buffer-size: 16777216
#bpf-filter: "tcp and port 25"
#checksum-checks: auto
#threads: 16
#promisc: no
#snaplen: 1518
pfring:
afpacket:
nfq:
ipfw:
Passive vs inline vs reading files is determined by how Suricata is invoked on the command line.
Rules¶
snort.conf:
In snort.conf a RULE_PATH variable is set, as well as variables for shared object (SO) rules and preprocessor rules.
var RULE_PATH ../rules
var SO_RULE_PATH ../so_rules
var PREPROC_RULE_PATH ../preproc_rules
include $RULE_PATH/local.rules
include $RULE_PATH/emerging-activex.rules
...
suricata.yaml:
In the suricata.yaml the default rule path is set followed by a list of rule files. Suricata does not have a concept of shared object rules or preprocessor rules. Instead of preprocessor rules, Suricata has several rule files for events set by the decoders, stream engine, http parser etc.
default-rule-path: /etc/suricata/rules
rule-files:
- local.rules
- emerging-activex.rules
The equivalent of preprocessor rules are loaded like normal rule files:
rule-files:
- decoder-events.rules
- stream-events.rules
- http-events.rules
- smtp-events.rules
Multi Tenancy¶
Introduction¶
Multi tenancy support allows for different rule sets with different rule vars.
YAML¶
In the main (“master”) YAML, the suricata.yaml, a new section called “multi-detect” should be added.
Settings:
- enabled: yes/no -> is multi-tenancy support enable
- default: yes/no -> is the normal detect config a default ‘fall back’ tenant?
- selector: direct (for unix socket pcap processing, see below) or vlan
- loaders: number of ‘loader’ threads, for parallel tenant loading at startup
- tenants: list of tenants
- id: tenant id
- yaml: separate yaml file with the tenant specific settings
- mappings:
- vlan id
- tenant id: tenant to associate with the vlan id
multi-detect:
enabled: yes
#selector: direct # direct or vlan
selector: vlan
loaders: 3
tenants:
- id: 1
yaml: tenant-1.yaml
- id: 2
yaml: tenant-2.yaml
- id: 3
yaml: tenant-3.yaml
mappings:
- vlan-id: 1000
tenant-id: 1
- vlan-id: 2000
tenant-id: 2
- vlan-id: 1112
tenant-id: 3
The tenant-1.yaml, tenant-2.yaml, tenant-3.yaml each contain a partial configuration:
# Set the default rule path here to search for the files.
# if not set, it will look at the current working dir
default-rule-path: /etc/suricata/rules
rule-files:
- rules1
# You can specify a threshold config file by setting "threshold-file"
# to the path of the threshold config file:
# threshold-file: /etc/suricata/threshold.config
classification-file: /etc/suricata/classification.config
reference-config-file: /etc/suricata/reference.config
# Holds variables that would be used by the engine.
vars:
# Holds the address group vars that would be passed in a Signature.
# These would be retrieved during the Signature address parsing stage.
address-groups:
HOME_NET: "[192.168.0.0/16,10.0.0.0/8,172.16.0.0/12]"
EXTERNAL_NET: "!$HOME_NET"
...
port-groups:
HTTP_PORTS: "80"
SHELLCODE_PORTS: "!80"
...
Unix Socket¶
Registration¶
register-tenant <id> <yaml>
Examples:
register-tenant 1 tenant-1.yaml
register-tenant 2 tenant-2.yaml
register-tenant 3 tenant-3.yaml
register-tenant 5 tenant-5.yaml
register-tenant 7 tenant-7.yaml
unregister-tenant <id>
unregister-tenant 2
unregister-tenant 1
Unix socket runmode (pcap processing)¶
The Unix Socket “pcap-file” command can be used to select the tenant to inspect the pcap against:
pcap-file traffic1.pcap /logs1/ 1
pcap-file traffic2.pcap /logs2/ 2
pcap-file traffic3.pcap /logs3/ 3
pcap-file traffic4.pcap /logs5/ 5
pcap-file traffic5.pcap /logs7/ 7
This runs the traffic1.pcap against tenant 1 and it logs into /logs1/, traffic2.pcap against tenant 2 and logs to /logs2/ and so on.
Live traffic mode¶
For live traffic currently only a vlan based multi-tenancy is supported.
The master yaml needs to have the selector set to “vlan”.
Registration¶
Tenants can be mapped to vlan id’s.
register-tenant-handler <tenant id> vlan <vlan id>
register-tenant-handler 1 vlan 1000
unregister-tenant-handler <tenant id> vlan <vlan id>
unregister-tenant-handler 4 vlan 1111
unregister-tenant-handler 1 vlan 1000
The registration of tenant and tenant handlers can be done on a running engine.
Dropping Privileges After Startup¶
Currently, libcap-ng is needed for dropping privileges on Suricata after startup. For libcap, see status of feature request number #276 – Libcap support for dropping privileges.
Most distributions have libcap-ng in their repositories.
To download the current version of libcap-ng from upstream, see also http://people.redhat.com/sgrubb/libcap-ng/ChangeLog
wget http://people.redhat.com/sgrubb/libcap-ng/libcap-ng-0.7.8.tar.gz
tar -xzvf libcap-ng-0.7.8.tar.gz
cd libcap-ng-0.7.8
./configure
make
make install
Download, configure, compile and install Suricata for your particular setup. See 安装. Depending on your environment, you may need to add the –with-libpcap_ng-libraries and –with-libpcap_ng-includes options during the configure step. e.g:
./configure --with-libcap_ng-libraries=/usr/local/lib \
--with-libcap_ng-includes=/usr/local/include
Now, when you run Suricata, tell it what user and/or group you want it to run as after startup with the –user and –group options. e.g. (this assumes a ‘suri’ user and group):
suricata -D -i eth0 --user=suri --group=suri
You will also want to make sure your user/group permissions are set so suricata can still write to its log files which are usually located in /var/log/suricata.
mkdir -p /var/log/suricata
chown -R root:suri /var/log/suricata
chmod -R 775 /var/log/suricata
Reputation¶
IP Reputation¶
IP Reputation Config¶
IP reputation has a few configuration directives, all disabled by default.
# IP Reputation
#reputation-categories-file: /etc/suricata/iprep/categories.txt
#default-reputation-path: /etc/suricata/iprep
#reputation-files:
# - reputation.list
reputation-categories-file¶
The categories file mapping numbered category values to short names.
reputation-categories-file: /etc/suricata/iprep/categories.txt
default-reputation-path¶
Path where reputation files from the “reputation-files” directive are loaded from by default.
default-reputation-path: /etc/suricata/iprep
reputation-files¶
YAML list of file names to load. In case of a absolute path the file is loaded directly, otherwise the path from “default-reputation-path” is pre-pended to form the final path.
reputation-files:
- badhosts.list
- knowngood.list
- sharedhosting.list
Hosts¶
IP reputation information is stored in the host table, so the settings of the host table affect it.
Depending on the number of hosts reputation information is available for, the memcap and hash size may have to be increased.
Reloads¶
If the “rule-reloads” option is enabled, sending Suricata a USR2 signal will reload the IP reputation data, along with the normal rules reload.
During the reload the host table will be updated to contain the new data. The iprep information is versioned. When the reload is complete, Suricata will automatically clean up the old iprep information.
Only the reputation files will be reloaded, the categories file won’t be. If categories change, Suricata should be restarted.
File format¶
The format of the reputation files is described in the IP Reputation Format page.
IP Reputation Format¶
Description of IP Reputation file formats. For the configuration see IP Reputation Config and IP Reputation Keyword for the rule format.
Categories file¶
The categories file provides a mapping between a category number, short name, and long description. It’s a simple CSV file:
<id>,<short name>,<description>
Example:
1,BadHosts,Known bad hosts
2,Google,Known google host
The maximum value for the category id is hard coded at 60 currently.
Reputation file¶
The reputation file lists a reputation score for hosts in the categories. It’s a simple CSV file:
<ip>,<category>,<reputation score>
The IP is an IPv4 address in the quad-dotted notation. The category is the number as defined in the categories file. The reputation score is the confidence that this IP is in the specified category, represented by a number between 1 and 127 (0 means no data).
Example:
1.2.3.4,1,101
1.1.1.1,6,88
If an IP address has a score in multiple categories it should be listed in the file multiple times.
Example:
1.1.1.1,1,10
1.1.1.1,2,10
This lists 1.1.1.1 in categories 1 and 2, each with a score of 10.
The purpose of the IP reputation component is the ranking of IP Addresses within the Suricata Engine. It will collect, store, update and distribute reputation intelligence on IP Addresses. The hub and spoke architecture will allows the central database (The Hub) to collect, store and compile updated IP reputation details that are then distributed to user-side sensor databases (Spokes) for inclusion in user security systems. The reputation data update frequency and security action taken, is defined in the user security configuration.
The intent of IP Reputation is to allow sharing of intelligence regarding a vast number of IP addresses. This can be positive or negative intelligence classified into a number of categories. The technical implementation requires three major efforts; engine integration, the hub that redistributes reputation, and the communication protocol between hubs and sensors. The hub will have a number of responsibilities. This will be a separate module running on a separate system as any sensor. Most often it would run on a central database that all sensors already have communication with. It will be able to subscribe to one or more external feeds. The local admin should be able to define the feeds to be subscribed to, provide authentication credentials if required, and give a weight to that feed. The weight can be an overall number or a by category weight. This will allow the admin to minimize the influence a feed has on their overall reputation if they distrust a particular category or feed, or trust another implicitly. Feeds can be configured to accept feedback or not and will report so on connect. The admin can override and choose not to give any feedback, but the sensor should report these to the Hub upstream on connect. The hub will take all of these feeds and aggregate them into an average single score for each IP or IP Block, and then redistribute this data to all local sensors as configured. It should receive connections from sensors. The sensor will have to provide authentication and will provide feedback. The hub should redistribute that feedback from sensors to all other sensors as well as up to any feeds that accept feedback. The hub should also have an API to allow outside statistical analysis to be done to the database and fed back in to the stream. For instance a local site may choose to change the reputation on all Russian IP blocks, etc.
For more information about IP Reputation see IP Reputation Config, IP Reputation Keyword and IP Reputation Format.
Init Scripts¶
For Ubuntu with Upstart, the following can be used in /etc/init/suricata.conf:
# suricata
description "Intruder Detection System Daemon"
start on runlevel [2345]
stop on runlevel [!2345]
expect fork
exec suricata -D --pidfile /var/run/suricata.pid -c /etc/suricata/suricata.yaml -i eth1
Setting up IPS/inline for Linux¶
In this guide will be explained how to work with Suricata in layer3 inline mode and how to set iptables for that purpose.
First start with compiling Suricata with NFQ support. For instructions see Ubuntu Installation. For more information about NFQ and iptables, see NFQ.
To check if you have NFQ enabled in your Suricata, enter the following command:
suricata --build-info
and examine if you have NFQ between the features.
To run suricata with the NFQ mode, you have to make use of the -q option. This option tells Suricata which of the queue numbers it should use.
sudo suricata -c /etc/suricata/suricata.yaml -q 0
Iptables configuration¶
First of all it is important to know which traffic you would like to send to Suricata. Traffic that passes your computer or traffic that is generated by your computer.
If Suricata is running on a gateway and is meant to protect the computers behind that gateway you are dealing with the first scenario: forward_ing . If Suricata has to protect the computer it is running on, you are dealing with the second scenario: host (see drawing 2). These two ways of using Suricata can also be combined.
The easiest rule in case of the gateway-scenario to send traffic to Suricata is:
sudo iptables -I FORWARD -j NFQUEUE
In this case, all forwarded traffic goes to Suricata.
In case of the host situation, these are the two most simple iptable rules;
sudo iptables -I INPUT -j NFQUEUE
sudo iptables -I OUTPUT -j NFQUEUE
It is possible to set a queue number. If you do not, the queue number will be 0 by default.
Imagine you want Suricata to check for example just TCP-traffic, or all incoming traffic on port 80, or all traffic on destination-port 80, you can do so like this:
sudo iptables -I INPUT -p tcp -j NFQUEUE
sudo iptables -I OUTPUT -p tcp -j NFQUEUE
In this case, Suricata checks just TCP traffic.
sudo iptables -I INPUT -p tcp --sport 80 -j NFQUEUE
sudo iptables -I OUTPUT -p tcp --dport 80 -j NFQUEUE
In this example, Suricata checks all input and output on port 80.
To see if you have set your iptables rules correct make sure Suricata is running and enter:
sudo iptables -vnL
In the example you can see if packets are being logged.
This description of the use of iptables is the way to use it with IPv4. To use it with IPv6 all previous mentioned commands have to start with ‘ip6tables’. It is also possible to let Suricata check both kinds of traffic.
There is also a way to use iptables with multiple networks (and interface cards). Example:
sudo iptables -I FORWARD -i eth0 -o eth1 -j NFQUEUE
sudo iptables -I FORWARD -i eth1 -o eth0 -j NFQUEUE
The options -i (input) -o (output) can be combined with all previous mentioned options
If you would stop Suricata and use internet, the traffic will not come through. To make internet work correctly, you have to erase all iptable rules.
To erase all iptable rules, enter:
sudo iptables -F
Setting up IPS/inline for Windows¶
This guide explains how to work with Suricata in layer 4 inline mode using WinDivert on Windows.
First start by compiling Suricata with WinDivert support. For instructions, see Windows Installation. This documentation has not yet been updated with WinDivert information, so make sure to add the following flags to configure:
--enable-windivert=yes --with-windivert-include=<include-dir> --with-windivert-libraries=<libraries-dir>
WinDivert.dll and WinDivert.sys must be in the same directory as the Suricata executable. WinDivert automatically installs the driver when it is run. For more information about WinDivert, see https://www.reqrypt.org/windivert-doc.html.
To check if you have WinDivert enabled in your Suricata, enter the following command in an elevated command prompt or terminal:
suricata -c suricata.yaml --windivert [filter string]
For information on the WinDivert filter language, see https://www.reqrypt.org/windivert-doc.html#filter_language
If Suricata is running on a gateway and is meant to protect the network behind that gateway, you need to run WinDivert at the NETWORK_FORWARD layer. This can be achieved using the following command:
suricata -c suricata.yaml --windivert-forward [filter string]
The filter is automatically stopped and normal traffic resumes when Suricata is stopped.
A quick start is to examine all traffic, in which case you can use the following command:
suricata -c suricata.yaml --windivert[-forward] true
A few additional examples:
Only TCP traffic:
suricata -c suricata.yaml --windivert tcp
Only TCP traffic on port 80:
suricata -c suricata.yaml --windivert "tcp.DstPort == 80"
TCP and ICMP traffic:
suricata -c suricata.yaml --windivert "tcp or icmp"
输出¶
EVE¶
Eve JSON输出¶
Suricata能够使用JSON输出告警，HTTP事件，DNS事件,TLS事件和文件信息。
The most common way to use this is through ‘EVE’, which is a firehose approach where all these logs go into a single file. 最常用的方法是通过’EVE’把所有这些日志都放在一个文件中。
outputs:
# Extensible Event Format (nicknamed EVE) event log in JSON format
- eve-log:
enabled: yes
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
#prefix: "@cee: " # prefix to prepend to each log entry
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
#redis:
# server: 127.0.0.1
# port: 6379
# async: true ## if redis replies are read asynchronously
# mode: list ## possible values: list|lpush (default), rpush, channel|publish
# ## lpush and rpush are using a Redis list. "list" is an alias for lpush
# ## publish is using a Redis channel. "channel" is an alias for publish
# key: suricata ## key or channel to use (default to suricata)
# Redis pipelining set up. This will enable to only do a query every
# 'batch-size' events. This should lower the latency induced by network
# connection at the cost of some memory. There is no flushing implemented
# so this setting as to be reserved to high traffic suricata.
# pipelining:
# enabled: yes ## set enable to yes to enable query pipelining
# batch-size: 10 ## number of entry to keep in buffer
# Include top level metadata. Default yes.
#metadata: no
types:
- alert:
# payload: yes # enable dumping payload in Base64
# payload-buffer-size: 4kb # max size of payload buffer to output in eve-log
# payload-printable: yes # enable dumping payload in printable (lossy) format
# packet: yes # enable dumping of packet (without stream segments)
# http-body: yes # enable dumping of http body in Base64
# http-body-printable: yes # enable dumping of http body in printable format
# Enable the logging of tagged packets for rules using the
# "tag" keyword.
tagged-packets: yes
# Configure the metadata to be logged along with an
# alert. The following shows the default configuration
# which is used if this field is not provided or simply
# set to a truthful value. Setting of this section is only
# required if you wish to enable/disable specific fields.
#metadata:
# Include the decoded application layer (ie. http, dns)
app-layer: true
# Log the the current state of the flow record.
flow: true
rule:
# Log the metadata field from the rule in a structured
# format.
metadata: true
# Log the raw rule text.
raw: false
# HTTP X-Forwarded-For support by adding an extra field or overwriting
# the source or destination IP address (depending on flow direction)
# with the one reported in the X-Forwarded-For HTTP header. This is
# helpful when reviewing alerts for traffic that is being reverse
# or forward proxied.
xff:
enabled: no
# Two operation modes are available, "extra-data" and "overwrite".
mode: extra-data
# Two proxy deployments are supported, "reverse" and "forward". In
# a "reverse" deployment the IP address used is the last one, in a
# "forward" deployment the first IP address is used.
deployment: reverse
# Header name where the actual IP address will be reported, if more
# than one IP address is present, the last IP address will be the
# one taken into consideration.
header: X-Forwarded-For
- http:
extended: yes # enable this for extended logging information
# custom allows additional http fields to be included in eve-log
# the example below adds three additional fields when uncommented
#custom: [Accept-Encoding, Accept-Language, Authorization]
- dns:
# Use version 2 logging with the new format:
# dns answers will be logged in one single event
# rather than an event for each of the answers.
# Without setting a version the version
# will fallback to 1 for backwards compatibility.
version: 2
# Enable/disable this logger. Default: enabled.
#enabled: no
# Control logging of requests and responses:
# - requests: enable logging of DNS queries
# - responses: enable logging of DNS answers
# By default both requests and responses are logged.
#requests: no
#responses: no
# Format of answer logging:
# - detailed: array item per answer
# - grouped: answers aggregated by type
# Default: all
#answer-format: [detailed, grouped]
# Answer types to log.
# Default: all
#answer-types: [a, aaaa, cname, mx, ns, ptr, txt]
- dns:
# Version 1 (deprecated) DNS logger.
version: 1
enabled: no
# control logging of queries and answers
# default yes, no to disable
query: yes # enable logging of DNS queries
answer: yes # enable logging of DNS answers
# control which RR types are logged
# all enabled if custom not specified
#custom: [a, aaaa, cname, mx, ns, ptr, txt]
- tls:
extended: yes # enable this for extended logging information
# output TLS transaction where the session is resumed using a
# session id
#session-resumption: no
# custom allows to control which tls fields that are included
# in eve-log
#custom: [subject, issuer, session_resumed, serial, fingerprint, sni, version, not_before, not_after, certificate, chain]
- files:
force-magic: no # force logging magic on all logged files
# force logging of checksums, available hash functions are md5,
# sha1 and sha256
#force-hash: [md5]
#- drop:
# alerts: yes # log alerts that caused drops
# flows: all # start or all: 'start' logs only a single drop
# # per flow direction. All logs each dropped pkt.
- smtp:
#extended: yes # enable this for extended logging information
# this includes: bcc, message-id, subject, x_mailer, user-agent
# custom fields logging from the list:
# reply-to, bcc, message-id, subject, x-mailer, user-agent, received,
# x-originating-ip, in-reply-to, references, importance, priority,
# sensitivity, organization, content-md5, date
#custom: [received, x-mailer, x-originating-ip, relays, reply-to, bcc]
# output md5 of fields: body, subject
# for the body you need to set app-layer.protocols.smtp.mime.body-md5
# to yes
#md5: [body, subject]
# NFS logging. Requires Rust.
- nfs
# IKEv2 logging. Requires Rust.
- ikev2
- ssh
- stats:
totals: yes # stats for all threads merged together
threads: no # per thread stats
deltas: no # include delta values
- dhcp:
# DHCP logging requires Rust.
enabled: @rust_config_enabled@
# When extended mode is on, all DHCP messages are logged
# with full detail. When extended mode is off (the
# default), just enough information to map a MAC address
# to an IP address is logged.
extended: no
# bi-directional flows
- flow
# uni-directional flows
#- netflow
# An event for logging metadata, specifically pktvars when
# they are set, but will also include the full metadata object.
#- metadata
任意一条告警，HTTP日志等将会输出到 ‘eve.json’ 这一个文件中，这个文件能够被第三方工具进行处理，如Logstash或者jq.
输出类型¶
EVE可以通过多种方式输出. regular 表示输出到文件. 其它选项有 syslog, unix_dgram, unix_stream 和 redis.
输出类型
filetype: regular #regular|syslog|unix_dgram|unix_stream|redis
filename: eve.json
#prefix: "@cee: " # prefix to prepend to each log entry
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
#redis:
# server: 127.0.0.1
# port: 6379
# async: true ## if redis replies are read asynchronously
# mode: list ## possible values: list|lpush (default), rpush, channel|publish
# ## lpush and rpush are using a Redis list. "list" is an alias for lpush
# ## publish is using a Redis channel. "channel" is an alias for publish
# key: suricata ## key or channel to use (default to suricata)
# Redis pipelining set up. This will enable to only do a query every
# 'batch-size' events. This should lower the latency induced by network
# connection at the cost of some memory. There is no flushing implemented
# so this setting as to be reserved to high traffic suricata.
# pipelining:
# enabled: yes ## set enable to yes to enable query pipelining
# batch-size: 10 ## number of entry to keep in buffer
告警¶
告警是记录规则匹配的事件，它可以被元数据修改，例如应用层（HTTP，DNS等）根据规则及其元素生成一条告警。
Metadata:
- alert:
#payload: yes # 使payload以base64编码输出
#payload-buffer-size: 4kb # 向eve-log输出payload缓冲区的最大size
#payload-printable: yes # 使payload以可打印(lossy)格式输出
#packet: yes # enable dumping of packet (without stream segments)
#http-body: yes # enable dumping of http body in Base64
#http-body-printable: yes # enable dumping of http body in printable format
# metadata:
# Include the decoded application layer (ie. http, dns)
#app-layer: true
# Log the the current state of the flow record.
#flow: true
#rule:
# Log the metadata field from the rule in a structured
# format.
#metadata: true
# Log the raw rule text.
#raw: false
DNS¶
DNS records are logged one log record per query/answer record.
YAML:
- dns:
# control logging of queries and answers
# default yes, no to disable
query: yes # enable logging of DNS queries
answer: yes # enable logging of DNS answers
# control which RR types are logged
# all enabled if custom not specified
#custom: [a, aaaa, cname, mx, ns, ptr, txt]
To reduce verbosity the output can be filtered by supplying the record types
to be logged under custom.
TLS¶
TLS records are logged one record per session.
YAML:
- tls:
extended: yes # enable this for extended logging information
# custom allows to control which tls fields that are included
# in eve-log
#custom: [subject, issuer, serial, fingerprint, sni, version, not_before, not_after, certificate, chain, ja3]
The default is to log certificate subject and issuer. If extended is
enabled, then the log gets more verbose.
By using custom it is possible to select which TLS fields to log.
Date modifiers in filename¶
It is possible to use date modifiers in the eve-log filename.
outputs:
- eve-log:
filename: eve-%s.json
The example above adds epoch time to the filename. All the date modifiers from the
C library should be supported. See the man page for strftime for all supported
modifiers.
Rotate log file¶
Eve-log can be configured to rotate based on time.
outputs:
- eve-log:
filename: eve-%Y-%m-%d-%H:%M.json
rotate-interval: minute
The example above creates a new log file each minute, where the filename contains
a timestamp. Other supported rotate-interval values are hour and day.
In addition to this, it is also possible to specify the rotate-interval as a
relative value. One example is to rotate the log file each X seconds.
outputs:
- eve-log:
filename: eve-%Y-%m-%d-%H:%M:%S.json
rotate-interval: 30s
The example above rotates eve-log each 30 seconds. This could be replaced with
30m to rotate every 30 minutes, 30h to rotate every 30 hours, 30d
to rotate every 30 days, or 30w to rotate every 30 weeks.
Multiple Logger Instances¶
It is possible to have multiple ‘EVE’ instances, for example the following is valid:
outputs:
- eve-log:
enabled: yes
type: file
filename: eve-ips.json
types:
- alert
- drop
- eve-log:
enabled: yes
type: file
filename: eve-nsm.json
types:
- http
- dns
- tls
So here the alerts and drops go into ‘eve-ips.json’, while http, dns and tls go into ‘eve-nsm.json’.
In addition to this, each log can be handled completely separately:
outputs:
- alert-json-log:
enabled: yes
filename: alert-json.log
- dns-json-log:
enabled: yes
filename: dns-json.log
- drop-json-log:
enabled: yes
filename: drop-json.log
- http-json-log:
enabled: yes
filename: http-json.log
- ssh-json-log:
enabled: yes
filename: ssh-json.log
- tls-json-log:
enabled: yes
filename: tls-json.log
For most output types, you can add multiple:
outputs:
- alert-json-log:
enabled: yes
filename: alert-json1.log
- alert-json-log:
enabled: yes
filename: alert-json2.log
Except for drop for which only a single logger instance is supported.
File permissions¶
Log file permissions can be set individually for each logger. filemode can be used to
control the permissions of a log file, e.g.:
outputs:
- eve-log:
enabled: yes
filename: eve.json
filemode: 600
The example above sets the file permissions on eve.json to 600, which means that it is
only readable and writable by the owner of the file.
JSON flags¶
Several flags can be specified to control the JSON output in EVE:
outputs:
- eve-log:
json:
# Sort object keys in the same order as they were inserted
preserve-order: yes
# Make the output more compact
compact: yes
# Escape all unicode characters outside the ASCII range
ensure-ascii: yes
# Escape the '/' characters in string with '\/'
escape-slash: yes
All these flags are enabled by default, and can be modified per EVE instance.
Eve JSON Format¶
Example:
{
"timestamp": "2009-11-24T21:27:09.534255",
"event_type": "alert",
"src_ip": "192.168.2.7",
"src_port": 1041,
"dest_ip": "x.x.250.50",
"dest_port": 80,
"proto": "TCP",
"alert": {
"action": "allowed",
"gid": 1,
"signature_id" :2001999,
"rev": 9,
"signature": "ET MALWARE BTGrab.com Spyware Downloading Ads",
"category": "A Network Trojan was detected",
"severity": 1
}
}
Common Section¶
All the JSON log types share a common structure:
{"timestamp":"2009-11-24T21:27:09.534255","event_type":"TYPE", ...tuple... ,"TYPE":{ ... type specific content ... }}
PCAP fields¶
If Suricata is processing a pcap file, additional fields are added:
"pcap_cnt": 123
pcap_cnt contains the packet number in the pcap. This can be used to look
up a packet in Wireshark for example.
"pcap_filename":"/path/to/file.pcap"
pcap_filename contains the file name and location of the pcap that
generated the event.
注解
the pcap fields are only available on “real” packets, and are omitted from internal “pseudo” packets such as flow timeout packets.
Event type: Alert¶
Field action¶
Possible values: “allowed” and “blocked”
Example:
"action":"allowed"
Action is set to “allowed” unless a rule used the “drop” action and Suricata is in IPS mode, or when the rule used the “reject” action.
It can also contain information about Source and Target of the attack in the alert.source and alert.target field it target keyword is used in the signature.
"alert": {
"action": "allowed",
"gid": 1,
"signature_id": 1,
"rev": 1,
"app_proto": "http",
"signature": "HTTP body talking about corruption",
"severity": 3,
"source": {
"ip": "192.168.43.32",
"port": 36292
},
"target": {
"ip": "179.60.192.3",
"port": 80
},
Event type: HTTP¶
Fields¶
- “hostname”: The hostname this HTTP event is attributed to
- “url”: URL at the hostname that was accessed
- “http_user_agent”: The user-agent of the software that was used
- “http_content_type”: The type of data returned (ex: application/x-gzip)
- “cookie”
In addition to these fields, if the extended logging is enabled in the suricata.yaml file the following fields are (can) also included:
- “length”: The content size of the HTTP body
- “status”: HTTP status code
- “protocol”: Protocol / Version of HTTP (ex: HTTP/1.1)
- “http_method”: The HTTP method (ex: GET, POST, HEAD)
- “http_refer”: The referer for this action
In addition to the extended logging fields one can also choose to enable/add from more than 50 additional custom logging HTTP fields enabled in the suricata.yaml file. The additional fields can be enabled as following:
- eve-log:
enabled: yes
type: file #file|syslog|unix_dgram|unix_stream
filename: eve.json
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
types:
- alert
- http:
extended: yes # enable this for extended logging information
# custom allows additional http fields to be included in eve-log
# the example below adds three additional fields when uncommented
#custom: [Accept-Encoding, Accept-Language, Authorization]
custom: [accept, accept-charset, accept-encoding, accept-language,
accept-datetime, authorization, cache-control, cookie, from,
max-forwards, origin, pragma, proxy-authorization, range, te, via,
x-requested-with, dnt, x-forwarded-proto, accept-range, age,
allow, connection, content-encoding, content-language,
content-length, content-location, content-md5, content-range,
content-type, date, etags, expires, last-modified, link, location,
proxy-authenticate, referrer, refresh, retry-after, server,
set-cookie, trailer, transfer-encoding, upgrade, vary, warning,
www-authenticate, x-flash-version, x-authenticated-user]
The benefits here of using the extended logging is to see if this action for example was a POST or perhaps if a download of an executable actually returned any bytes.
Examples¶
Event with non-extended logging:
"http": {
"hostname": "www.digip.org",
"url" :"\/jansson\/releases\/jansson-2.6.tar.gz",
"http_user_agent": "<User-Agent>",
"http_content_type": "application\/x-gzip"
}
Event with extended logging:
"http": {
"hostname": "direkte.vg.no",
"url":".....",
"http_user_agent": "<User-Agent>",
"http_content_type": "application\/json",
"http_refer": "http:\/\/www.vg.no\/",
"http_method": "GET",
"protocol": "HTTP\/1.1",
"status":"200",
"length":310
}
Event type: DNS¶
A new version of dns logging has been introduced to improve how dns answers are logged.
With that new version, dns answers are logged in one event rather than an event for each answer.
It’s possible to customize how a dns answer will be logged with the following formats:
- “detailed”: “rrname”, “rrtype”, “rdata” and “ttl” fields are logged for each answer
- “grouped”: answers logged are aggregated by their type (A, AAAA, NS, …)
It will be still possible to use the old DNS logging format, you can control it with “version” option in dns configuration section.
Fields¶
Outline of fields seen in the different kinds of DNS events:
- “type”: Indicating DNS message type, can be “answer” or “query”.
- “id”: Identifier field
- “version”: Indicating DNS logging version in use
- “flags”: Indicating DNS answer flag, in hexadecimal (ex: 8180 , please note 0x is not output)
- “qr”: Indicating in case of DNS answer flag, Query/Response flag (ex: true if set)
- “aa”: Indicating in case of DNS answer flag, Authoritative Answer flag (ex: true if set)
- “tc”: Indicating in case of DNS answer flag, Truncation flag (ex: true if set)
- “rd”: Indicating in case of DNS answer flag, Recursion Desired flag (ex: true if set)
- “ra”: Indicating in case of DNS answer flag, Recursion Available flag (ex: true if set)
- “rcode”: (ex: NOERROR)
- “rrname”: Resource Record Name (ex: a domain name)
- “rrtype”: Resource Record Type (ex: A, AAAA, NS, PTR)
- “rdata”: Resource Data (ex. IP that domain name resolves to)
- “ttl”: Time-To-Live for this resource record
One can also control which RR types are logged explicitly from additional custom field enabled in the suricata.yaml file. If custom field is not specified, all RR types are logged. More than 50 values can be specified with the custom field and can be used as following:
- eve-log:
enabled: yes
type: file #file|syslog|unix_dgram|unix_stream
filename: eve.json
# the following are valid when type: syslog above
#identity: "suricata"
#facility: local5
#level: Info ## possible levels: Emergency, Alert, Critical,
## Error, Warning, Notice, Info, Debug
types:
- alert
- dns:
# control logging of queries and answers
# default yes, no to disable
query: yes # enable logging of DNS queries
answer: yes # enable logging of DNS answers
# control which RR types are logged
# all enabled if custom not specified
#custom: [a, aaaa, cname, mx, ns, ptr, txt]
custom: [a, ns, md, mf, cname, soa, mb, mg, mr, null,
wks, ptr, hinfo, minfo, mx, txt, rp, afsdb, x25, isdn,
rt, nsap, nsapptr, sig, key, px, gpos, aaaa, loc, nxt,
srv, atma, naptr, kx, cert, a6, dname, opt, apl, ds,
sshfp, ipseckey, rrsig, nsec, dnskey, dhcid, nsec3,
nsec3param, tlsa, hip, cds, cdnskey, spf, tkey,
tsig, maila, any, uri]
Examples¶
Example of a DNS query for the IPv4 address of “twitter.com” (resource record type ‘A’):
"dns": {
"type": "query",
"id": 16000,
"rrname": "twitter.com",
"rrtype":"A"
}
Example of a DNS answer with “detailed” format:
"dns": {
"version": 2,
"type": "answer",
"id": 45444,
"flags": "8180",
"qr": true,
"rd": true,
"ra": true,
"rcode": "NOERROR",
"answers": [
{
"rrname": "www.suricata-ids.org",
"rrtype": "CNAME",
"ttl": 3324,
"rdata": "suricata-ids.org"
},
{
"rrname": "suricata-ids.org",
"rrtype": "A",
"ttl": 10,
"rdata": "192.0.78.24"
},
{
"rrname": "suricata-ids.org",
"rrtype": "A",
"ttl": 10,
"rdata": "192.0.78.25"
}
]
}
Example of a DNS answer with “grouped” format:
"dns": {
"version": 2,
"type": "answer",
"id": 18523,
"flags": "8180",
"qr": true,
"rd": true,
"ra": true,
"rcode": "NOERROR",
"grouped": {
"A": [
"192.0.78.24",
"192.0.78.25"
],
"CNAME": [
"suricata-ids.org"
]
}
}
Example of a old DNS answer with an IPv4 (resource record type ‘A’) return:
"dns": {
"type": "answer",
"id":16000,
"flags":"8180",
"qr":true,
"rd":true,
"ra":true,
"rcode":"NOERROR"
"rrname": "twitter.com",
"rrtype":"A",
"ttl":8,
"rdata": "199.16.156.6"
}
Event type: TLS¶
Fields¶
- “subject”: The subject field from the TLS certificate
- “issuer”: The issuer field from the TLS certificate
- “session_resumed”: This field has the value of “true” if the TLS session was resumed via a session id. If this field appears, “subject” and “issuer” do not appear, since a TLS certificate is not seen.
If extended logging is enabled the following fields are also included:
- “serial”: The serial number of the TLS certificate
- “fingerprint”: The (SHA1) fingerprint of the TLS certificate
- “sni”: The Server Name Indication (SNI) extension sent by the client
- “version”: The SSL/TLS version used
- “not_before”: The NotBefore field from the TLS certificate
- “not_after”: The NotAfter field from the TLS certificate
- “ja3”: The JA3 fingerprint consisting of both a JA3 hash and a JA3 string
JA3 must be enabled in the Suricata config file (set ‘app-layer.protocols.tls.ja3-fingerprints’ to ‘yes’).
In addition to this, custom logging also allows the following fields:
- “certificate”: The TLS certificate base64 encoded
- “chain”: The entire TLS certificate chain base64 encoded
Examples¶
Example of regular TLS logging:
"tls": {
"subject": "C=US, ST=California, L=Mountain View, O=Google Inc, CN=*.google.com",
"issuerdn": "C=US, O=Google Inc, CN=Google Internet Authority G2"
}
Example of regular TLS logging for resumed sessions:
"tls": {
"session_resumed": true
}
Example of extended TLS logging:
"tls": {
"subject": "C=US, ST=California, L=Mountain View, O=Google Inc, CN=*.google.com",
"issuerdn": "C=US, O=Google Inc, CN=Google Internet Authority G2",
"serial": "0C:00:99:B7:D7:54:C9:F6:77:26:31:7E:BA:EA:7C:1C",
"fingerprint": "8f:51:12:06:a0:cc:4e:cd:e8:a3:8b:38:f8:87:59:e5:af:95:ca:cd",
"sni": "calendar.google.com",
"version": "TLS 1.2",
"notbefore": "2017-01-04T10:48:43",
"notafter": "2017-03-29T10:18:00"
}
Example of certificate logging using TLS custom logging (subject, sni, certificate):
"tls": {
"subject": "C=US, ST=California, L=Mountain View, O=Google Inc, CN=*.googleapis.com
"sni": "www.googleapis.com",
"certificate": "MIIE3TCCA8WgAwIBAgIIQPsvobRZN0gwDQYJKoZIhvcNAQELBQAwSTELMA [...]"
}
Event type: TFTP¶
Fields¶
- “packet”: The operation code, can be “read” or “write” or “error”
- “file”: The filename transported with the tftp protocol
- “mode”: The mode field, can be “octet” or “mail” or “netascii” (or any combination of upper and lower case)
Example of TFTP logging:
"tftp": {
"packet": "write",
"file": "rfc1350.txt",
"mode": "octet"
}
Event type: SMB¶
SMB Fields¶
- “id” (integer): internal transaction id
- “dialect” (string): the negotiated protocol dialect, or “unknown” if missing
- “command” (string): command name. E.g. SMB2_COMMAND_CREATE or SMB1_COMMAND_WRITE_ANDX
- “status” (string): status string. Can be both NT_STATUS or DOS_ERR and other variants
- “status_code” (string): status code as hex string
- “session_id” (integer): SMB2+ session_id. SMB1 user id.
- “tree_id” (integer): Tree ID
- “filename” (string): filename for CREATE and other commands.
- “disposition” (string): requested disposition. E.g. FILE_OPEN, FILE_CREATE and FILE_OVERWRITE. See https://msdn.microsoft.com/en-us/library/ee442175.aspx#Appendix_A_Target_119
- “access” (string): indication of how the file was opened. “normal” or “delete on close” (field is subject to change)
- “created”, “accessed”, “modified”, “changed” (integer): timestamps in seconds since unix epoch
- “size” (integer): size of the requested file
- “fuid” (string): SMB2+ file GUID. SMB1 FID as hex.
- “share” (string): share name.
- “share_type” (string): FILE, PIPE, PRINT or unknown.
- “client_dialects” (array of strings): list of SMB dialects the client speaks.
- “client_guid” (string): client GUID
- “server_guid” (string): server GUID
- “request.native_os” (string): SMB1 native OS string
- “request.native_lm” (string): SMB1 native Lan Manager string
- “response.native_os” (string): SMB1 native OS string
- “response.native_lm” (string): SMB1 native Lan Manager string
Examples of SMB logging:
Pipe open:
"smb": {
"id": 1,
"dialect": "unknown",
"command": "SMB2_COMMAND_CREATE",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 4398046511201,
"tree_id": 1,
"filename": "atsvc",
"disposition": "FILE_OPEN",
"access": "normal",
"created": 0,
"accessed": 0,
"modified": 0,
"changed": 0,
"size": 0,
"fuid": "0000004d-0000-0000-0005-0000ffffffff"
}
File/pipe close:
"smb": {
"id": 15,
"dialect": "2.10",
"command": "SMB2_COMMAND_CLOSE",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 4398046511121,
"tree_id": 1,
}
Tree connect (share open):
"smb": {
"id": 3,
"dialect": "2.10",
"command": "SMB2_COMMAND_TREE_CONNECT",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 4398046511121,
"tree_id": 1,
"share": "\\\\admin-pc\\c$",
"share_type": "FILE"
}
Dialect negotiation from SMB1 to SMB2 dialect 2.10:
"smb": {
"id": 1,
"dialect": "2.??",
"command": "SMB1_COMMAND_NEGOTIATE_PROTOCOL",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 0,
"tree_id": 0,
"client_dialects": [
"PC NETWORK PROGRAM 1.0",
"LANMAN1.0",
"Windows for Workgroups 3.1a",
"LM1.2X002",
"LANMAN2.1",
"NT LM 0.12",
"SMB 2.002",
"SMB 2.???"
],
"server_guid": "aec6e793-2b11-4019-2d95-55453a0ad2f1"
}
"smb": {
"id": 2,
"dialect": "2.10",
"command": "SMB2_COMMAND_NEGOTIATE_PROTOCOL",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 0,
"tree_id": 0,
"client_dialects": [
"2.02",
"2.10"
],
"client_guid": "601985d2-aad9-11e7-8494-00088bb57f27",
"server_guid": "aec6e793-2b11-4019-2d95-55453a0ad2f1"
}
SMB1 partial SMB1_COMMAND_SESSION_SETUP_ANDX:
"request": {
"native_os": "Unix",
"native_lm": "Samba 3.9.0-SVN-build-11572"
},
"response": {
"native_os": "Windows (TM) Code Name \"Longhorn\" Ultimate 5231",
"native_lm": "Windows (TM) Code Name \"Longhorn\" Ultimate 6.0"
}
DCERPC fields¶
- “request” (string): command. E.g. REQUEST, BIND.
- “response” (string): reply. E.g. RESPONSE, BINDACK or FAULT.
- “opnum” (integer): the opnum
- “call_id” (integer): the call id
- “frag_cnt” (integer): the number of fragments for the stub data
- “stub_data_size”: total stub data size
- “interfaces” (array): list of interfaces
- “interfaces.uuid” (string): string representation of the UUID
- “interfaces.version” (string): interface version
- “interfaces.ack_result” (integer): ack result
- “interfaces.ack_reason” (integer): ack reason
DCERPC REQUEST/RESPONSE:
"smb": {
"id": 4,
"dialect": "unknown",
"command": "SMB2_COMMAND_IOCTL",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 4398046511201,
"tree_id": 0,
"dcerpc": {
"request": "REQUEST",
"response": "RESPONSE",
"opnum": 0,
"req": {
"frag_cnt": 1,
"stub_data_size": 136
},
"res": {
"frag_cnt": 1,
"stub_data_size": 8
},
"call_id": 2
}
}
DCERPC BIND/BINDACK:
"smb": {
"id": 53,
"dialect": "2.10",
"command": "SMB2_COMMAND_WRITE",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 35184439197745,
"tree_id": 1,
"dcerpc": {
"request": "BIND",
"response": "BINDACK",
"interfaces": [
{
"uuid": "12345778-1234-abcd-ef00-0123456789ac",
"version": "1.0",
"ack_result": 2,
"ack_reason": 0
},
{
"uuid": "12345778-1234-abcd-ef00-0123456789ac",
"version": "1.0",
"ack_result": 0,
"ack_reason": 0
},
{
"uuid": "12345778-1234-abcd-ef00-0123456789ac",
"version": "1.0",
"ack_result": 3,
"ack_reason": 0
}
],
"call_id": 2
}
NTLMSSP fields¶
- “domain” (string): the Windows domain.
- “user” (string): the user.
- “host” (string): the host.
Example:
"ntlmssp": {
"domain": "VNET3",
"user": "administrator",
"host": "BLU"
}
More complete example:
"smb": {
"id": 3,
"dialect": "NT LM 0.12",
"command": "SMB1_COMMAND_SESSION_SETUP_ANDX",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 2048,
"tree_id": 0,
"ntlmssp": {
"domain": "VNET3",
"user": "administrator",
"host": "BLU"
},
"request": {
"native_os": "Unix",
"native_lm": "Samba 3.9.0-SVN-build-11572"
},
"response": {
"native_os": "Windows (TM) Code Name \"Longhorn\" Ultimate 5231",
"native_lm": "Windows (TM) Code Name \"Longhorn\" Ultimate 6.0"
}
}
Kerberos fields¶
- “kerberos.realm” (string): the Kerberos Realm.
- “kerberos.snames (array of strings): snames.
Example:
"smb": {
"dialect": "2.10",
"command": "SMB2_COMMAND_SESSION_SETUP",
"status": "STATUS_SUCCESS",
"status_code": "0x0",
"session_id": 35184439197745,
"tree_id": 0,
"kerberos": {
"realm": "CONTOSO.LOCAL",
"snames": [
"cifs",
"DC1.contoso.local"
]
}
}
Eve JSON ‘jq’ Examples¶
The jq tool is very useful for quickly parsing and filtering JSON files. This page is contains various examples of how it can be used with Suricata’s Eve.json.
The basics are discussed here:
Colorize output¶
tail -f eve.json | jq -c '.'
DNS NXDOMAIN¶
tail -f eve.json|jq -c 'select(.dns.rcode=="NXDOMAIN")'
Unique HTTP User Agents¶
cat eve.json | jq -s '[.[]|.http.http_user_agent]|group_by(.)|map({key:.[0],value:(.|length)})|from_entries'
Source: https://twitter.com/mattarnao/status/601807374647750657
Data use for a host¶
tail -n500000 eve.json | jq -s 'map(select(.event_type=="netflow" and .dest_ip=="192.168.1.3").netflow.bytes)|add'|numfmt --to=iec
1.3G
Note: can use a lot of memory. Source: https://twitter.com/pkt_inspector/status/605524218722148352
Monitor part of the stats¶
$ tail -f eve.json | jq -c 'select(.event_type=="stats")|.stats.decoder'
Inspect Alert Data¶
cat eve.json | jq -r -c 'select(.event_type=="alert")|.payload'|base64 --decode
Top 10 Destination Ports¶
cat eve.json | jq -c 'select(.event_type=="flow")|[.proto, .dest_port]'|sort |uniq -c|sort -nr|head -n10
Lua Output¶
Suricata offers the possibility to get more detailed output on specific kinds of network traffic via pluggable lua scripts. You can write these scripts yourself and only need to define four hook functions.
For lua output scripts suricata offers a wide range of lua functions. They all return information on specific engine internals and aspects of the network traffic. They are described in the following sections, grouped by the event/traffic type. But let’s start with a example explaining the four hook functions, and how to make suricata load a lua output script.
Script structure¶
A lua output script needs to define 4 hook functions: init(), setup(), log(), deinit()
- init() – registers where the script hooks into the output engine
- setup() – does per output thread setup
- log() – logging function
- deinit() – clean up function
Example:
function init (args)
local needs = {}
needs["protocol"] = "http"
return needs
end
function setup (args)
filename = SCLogPath() .. "/" .. name
file = assert(io.open(filename, "a"))
SCLogInfo("HTTP Log Filename " .. filename)
http = 0
end
function log(args)
http_uri = HttpGetRequestUriRaw()
if http_uri == nil then
http_uri = "<unknown>"
end
http_uri = string.gsub(http_uri, "%c", ".")
http_host = HttpGetRequestHost()
if http_host == nil then
http_host = "<hostname unknown>"
end
http_host = string.gsub(http_host, "%c", ".")
http_ua = HttpGetRequestHeader("User-Agent")
if http_ua == nil then
http_ua = "<useragent unknown>"
end
http_ua = string.gsub(http_ua, "%g", ".")
timestring = SCPacketTimeString()
ip_version, src_ip, dst_ip, protocol, src_port, dst_port = SCFlowTuple()
file:write (timestring .. " " .. http_host .. " [**] " .. http_uri .. " [**] " ..
http_ua .. " [**] " .. src_ip .. ":" .. src_port .. " -> " ..
dst_ip .. ":" .. dst_port .. "\n")
file:flush()
http = http + 1
end
function deinit (args)
SCLogInfo ("HTTP transactions logged: " .. http);
file:close(file)
end
YAML¶
To enable the lua output, add the ‘lua’ output and add one or more scripts like so:
outputs:
- lua:
enabled: yes
scripts-dir: /etc/suricata/lua-output/
scripts:
- tcp-data.lua
- flow.lua
The scripts-dir option is optional. It makes Suricata load the scripts from this directory. Otherwise scripts will be loaded from the current workdir.
Developing lua output script¶
You can use functions described in Lua Functions
Syslog Alerting Compatibility¶
Suricata can alert via sylog which is a very handy feature for central log collection, compliance, and reporting to a SIEM. Instructions on setting this up can be found in the .yaml file in the section where you can configure what type of alert (and other) logging you would like.
However, there are different syslog daemons and there can be parsing issues with the syslog format a SIEM expects and what syslog format Suricata sends. The syslog format from Suricata is dependent on the syslog daemon running on the Suricata sensor but often the format it sends is not the format the SIEM expects and cannot parse it properly.
Popular syslog daemons¶
- syslogd - logs system messages
- syslog-ng - logs system messages but also supports TCP, TLS, and other enhanced enterprise features
- rsyslogd - logs system messages but also support TCP, TLS, multi-threading, and other enhanced features
- klogd - logs kernel messages
- sysklogd - basically a bundle of syslogd and klogd
If the syslog format the Suricata sensor is sending is not compatible with what your SIEM or syslog collector expects, you will need to fix this. You can do this on your SIEM if it is capable of being able to be configured to interpret the message, or by configuring the syslog daemon on the Suricata sensor itself to send in a format you SIEM can parse. The latter can be done by applying a template to your syslog config file.
Finding what syslog daemon you are using¶
There are many ways to find out what syslog daemon you are using but here is one way:
cd /etc/init.d
ls | grep syslog
You should see a file with the word syslog in it, e.g. “syslog”, “rsyslogd”, etc. Obviously if the name is “rsyslogd” you can be fairly confident you are running rsyslogd. If unsure or the filename is just “syslog”, take a look at that file. For example, if it was “rsyslogd”, run:
less rsyslogd
At the top you should see a comment line that looks something like this:
# rsyslog Starts rsyslogd/rklogd.
Locate those files and look at them to give you clues as to what syslog daemon you are running. Also look in the start() section of the file you ran “less” on and see what binaries get started because that can give you clues as well.
Example¶
Here is an example where the Suricata sensor is sending syslog messages in rsyslogd format but the SIEM is expecting and parsing them in a sysklogd format. In the syslog configuration file (usually in /etc with a filename like rsyslog.conf or syslog.conf), first add the template:
$template sysklogd, "<%PRI%>%syslogtag:1:32%%msg:::sp-if-no-1st-sp%%msg%"
Then send it to the syslog server with the template applied:
user.alert @10.8.75.24:514;sysklogd
Of course this is just one example and it will probably be different in your environment depending on what syslog daemons and SIEM you use but hopefully this will point you in the right direction.
Custom http logging¶
As of Suricata 1.3.1 you can enable a custom http logging option.
In your Suricata.yaml, find the http-log section and edit as follows:
- http-log:
enabled: yes
filename: http.log
custom: yes # enable the custom logging format (defined by custom format)
customformat: "%{%D-%H:%M:%S}t.%z %{X-Forwarded-For}i %{User-agent}i %H %m %h %u %s %B %a:%p -> %A:%P"
append: no
#extended: yes # enable this for extended logging information
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
And in your http.log file you would get the following, for example:
8/28/12-22:14:21.101619 - Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0 HTTP/1.1 GET us.cnn.com /video/data/3.0/video/world/2012/08/28/hancocks-korea-typhoon-bolavan.cnn/index.xml 200 16856 192.168.1.91:45111 -> 157.166.255.18:80
08/28/12-22:14:30.693856 - Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0 HTTP/1.1 GET us.cnn.com /video/data/3.0/video/showbiz/2012/08/28/conan-reports-from-rnc-convention.teamcoco/index.xml 200 15789 192.168.1.91:45108 -> 157.166.255.18:80
The list of supported format strings is the following:
- %h - Host HTTP Header (remote host name). ie: google.com
- %H - Request Protocol. ie: HTTP/1.1
- %m - Request Method. ie: GET
- %u - URL including query string. ie: /search?q=suricata
- %{header_name}i - contents of the defined HTTP Request Header name. ie:
- %{User-agent}i: Mozilla/5.0 (X11; Ubuntu; Linux i686; rv:11.0) Gecko/20100101 Firefox/11.0
- %{X-Forwarded-For}i: outputs the IP address contained in the X-Forwarded-For HTTP header (inserted by a reverse proxy)
- %s - return status code. In the case of 301 and 302 it will print the url in brackets. ie: 200
- %B - response size in bytes. ie: 15789
- %{header_name}o - contents of the defined HTTP Response Header name
- %{strftime_format]t - timestamp of the HTTP transaction in the selected strftime format. ie: 08/28/12-22:14:30
- %z - precision time in useconds. ie: 693856
- %a - client IP address
- %p - client port number
- %A - server IP address
- %P - server port number
Any non printable character will be represented by its byte value in hexadecimal format (|XX|, where XX is the hex code)
Custom tls logging¶
In your Suricata.yaml, find the tls-log section and edit as follows:
- tls-log:
enabled: yes # Log TLS connections.
filename: tls.log # File to store TLS logs.
append: yes
custom: yes # enabled the custom logging format (defined by customformat)
customformat: "%{%D-%H:%M:%S}t.%z %a:%p -> %A:%P %n %n %d %D"
And in your tls.log file you would get the following, for example:
12/03/16-19:20:14.85859 10.10.10.4:58274 -> 192.0.78.24:443 VERSION='TLS 1.2' suricata-ids.org NOTBEFORE='2016-10-27T20:36:00' NOTAFTER='2017-01-25T20:36:00'
- ::
- 12/03/16-19:20:20.36849 10.10.10.4:39472 -> 192.30.253.113:443 VERSION=’TLS 1.2’ github.com NOTBEFORE=‘2016-03-10T00:00:00’ NOTAFTER=‘2018-05-17T12:00:00’
The list of supported format strings is the following:
- %n - client SNI
- %v - TLS/SSL version
- %d - certificate date not before
- %D - certificate date not after
- %f - certificate fingerprint SHA1
- %s - certificate subject
- %i - certificate issuer dn
- %E - extended format
- %{strftime_format}t - timestamp of the TLS transaction in the selected strftime format. ie: 08/28/12-22:14:30
- %z - precision time in useconds. ie: 693856
- %a - client IP address
- %p - client port number
- %A - server IP address
- %P - server port number
Any non printable character will be represented by its byte value in hexadecimal format (|XX|, where XX is the hex code)
Log Rotation¶
All outputs in the outputs section of the configuration file can be subject to log rotation.
For most outputs an external tool like logrotate is required to rotate the log files in combination with sending a SIGHUP to Suricata to notify it that the log files have been rotated.
On receipt of a SIGHUP, Suricata simply closes all open log files and then re-opens them in append mode. If the external tool has renamed any of the log files, new files will be created, otherwise the files will be re-opened and new data will be appended to them with no noticeable affect.
The following is an example logrotate configuration file that will rotate Suricata log files then send Suricata a SIGHUP triggering Suricata to open new files:
/var/log/suricata/*.log /var/log/suricata/*.json
{
rotate 3
missingok
nocompress
create
sharedscripts
postrotate
/bin/kill -HUP `cat /var/run/suricata.pid 2>/dev/null` 2>/dev/null || true
endscript
}
注解
The above logrotate configuration file depends on the
existence of a Suricata PID file. If running in daemon mode
a PID file will be created by default, otherwise the
--pidfile option should be used to create a PID file.
In addition to the SIGHUP style rotation discussed above, some outputs support their own time and date based rotation, however removal of old log files is still the responsibility of external tools. These outputs include:
Lua support¶
Lua usage in Suricata¶
Lua scripting can be used in two components of Suricata. The first is in output an the second one in rules in the detection engine.
Both features are using a list of functions to access to data extracted by Suricata. You can get the list of functions in the Lua functions page.
Lua output¶
Lua can be used to write arbitrary output. See Lua Output for more information.
Lua detection¶
Lua script can be used as a filter condition in signatures. See Lua Scripting for more information.
Lua functions¶
packet¶
Initialize with:
function init (args)
local needs = {}
needs["type"] = "packet"
return needs
end
SCPacketTimestamp¶
Get packets timestamp as 2 numbers: seconds & microseconds elapsed since 1970-01-01 00:00:00 UTC.
function log(args)
local sec, usec = SCPacketTimestamp()
end
SCPacketTimeString¶
Add SCPacketTimeString to get the packets time string in the format: 11/24/2009-18:57:25.179869
function log(args)
ts = SCPacketTimeString()
SCPacketTuple¶
ipver, srcip, dstip, proto, sp, dp = SCPacketTuple()
SCPacketPayload¶
p = SCPacketPayload()
flow¶
function init (args)
local needs = {}
needs["type"] = "flow"
return needs
end
SCFlowTimestamps¶
Get timestamps (seconds and microseconds) of the first and the last packet from the flow.
startts, lastts = SCFlowTimestamps()
startts_s, lastts_s, startts_us, lastts_us = SCFlowTimestamps()
SCFlowTimeString¶
startts = SCFlowTimeString()
SCFlowTuple¶
ipver, srcip, dstip, proto, sp, dp = SCFlowTuple()
SCFlowAppLayerProto¶
Get alprotos as string from the flow. If a alproto is not (yet) known, it returns “unknown”.
Example:
function log(args)
alproto = SCFlowAppLayerProto()
if alproto ~= nil then
print (alproto)
end
end
Returns 5 values: <alproto> <alproto_ts> <alproto_tc> <alproto_orig> <alproto_expect>
Orig and expect are used when changing and upgrading protocols. In a SMTP STARTTLS case, orig would normally be set to “smtp” and expect to “tls”.
SCFlowHasAlerts¶
Returns true if flow has alerts.
Example:
function log(args)
has_alerts = SCFlowHasAlerts()
if has_alerts then
-- do something
end
end
SCFlowStats¶
Gets the packet and byte counts per flow.
tscnt, tsbytes, tccnt, tcbytes = SCFlowStats()
SCFlowId¶
Gets the flow id.
id = SCFlowId()
Note that simply printing ‘id’ will likely result in printing a scientific notation. To avoid that, simply do:
id = SCFlowId()
idstr = string.format("%.0f",id)
print ("Flow ID: " .. idstr .. "\n")
http¶
Init with:
function init (args)
local needs = {}
needs["protocol"] = "http"
return needs
end
HttpGetRequestBody and HttpGetResponseBody.¶
Make normalized body data available to the script through HttpGetRequestBody and HttpGetResponseBody.
There no guarantees that all of the body will be availble.
Example:
function log(args)
a, o, e = HttpGetResponseBody();
--print("offset " .. o .. " end " .. e)
for n, v in ipairs(a) do
print(v)
end
end
HttpGetRequestHost¶
Get the host from libhtp’s tx->request_hostname, which can either be the host portion of the url or the host portion of the Host header.
Example:
http_host = HttpGetRequestHost()
if http_host == nil then
http_host = "<hostname unknown>"
end
HttpGetRequestHeader¶
http_ua = HttpGetRequestHeader("User-Agent")
if http_ua == nil then
http_ua = "<useragent unknown>"
end
HttpGetResponseHeader¶
server = HttpGetResponseHeader("Server");
print ("Server: " .. server);
HttpGetRequestLine¶
rl = HttpGetRequestLine();
print ("Request Line: " .. rl);
HttpGetResponseLine¶
rsl = HttpGetResponseLine();
print ("Response Line: " .. rsl);
HttpGetRawRequestHeaders¶
rh = HttpGetRawRequestHeaders();
print ("Raw Request Headers: " .. rh);
HttpGetRawResponseHeaders¶
rh = HttpGetRawResponseHeaders();
print ("Raw Response Headers: " .. rh);
HttpGetRequestUriRaw¶
http_uri = HttpGetRequestUriRaw()
if http_uri == nil then
http_uri = "<unknown>"
end
HttpGetRequestUriNormalized¶
http_uri = HttpGetRequestUriNormalized()
if http_uri == nil then
http_uri = "<unknown>"
end
HttpGetRequestHeaders¶
a = HttpGetRequestHeaders();
for n, v in pairs(a) do
print(n,v)
end
HttpGetResponseHeaders¶
a = HttpGetResponseHeaders();
for n, v in pairs(a) do
print(n,v)
end
DNS¶
DnsGetQueries¶
dns_query = DnsGetQueries();
if dns_query ~= nil then
for n, t in pairs(dns_query) do
rrname = t["rrname"]
rrtype = t["type"]
print ("QUERY: " .. ts .. " " .. rrname .. " [**] " .. rrtype .. " [**] " ..
"TODO" .. " [**] " .. srcip .. ":" .. sp .. " -> " ..
dstip .. ":" .. dp)
end
end
returns a table of tables
DnsGetAnswers¶
dns_answers = DnsGetAnswers();
if dns_answers ~= nil then
for n, t in pairs(dns_answers) do
rrname = t["rrname"]
rrtype = t["type"]
ttl = t["ttl"]
print ("ANSWER: " .. ts .. " " .. rrname .. " [**] " .. rrtype .. " [**] " ..
ttl .. " [**] " .. srcip .. ":" .. sp .. " -> " ..
dstip .. ":" .. dp)
end
end
returns a table of tables
DnsGetAuthorities¶
dns_auth = DnsGetAuthorities();
if dns_auth ~= nil then
for n, t in pairs(dns_auth) do
rrname = t["rrname"]
rrtype = t["type"]
ttl = t["ttl"]
print ("AUTHORITY: " .. ts .. " " .. rrname .. " [**] " .. rrtype .. " [**] " ..
ttl .. " [**] " .. srcip .. ":" .. sp .. " -> " ..
dstip .. ":" .. dp)
end
end
returns a table of tables
DnsGetRcode¶
rcode = DnsGetRcode();
if rcode == nil then
return 0
end
print (rcode)
returns a lua string with the error message, or nil
DnsGetRecursionDesired¶
if DnsGetRecursionDesired() == true then
print ("RECURSION DESIRED")
end
returns a bool
TLS¶
Initialize with:
function init (args)
local needs = {}
needs["protocol"] = "tls"
return needs
end
TlsGetCertInfo¶
Make certificate information available to the script through TlsGetCertInfo.
Example:
function log (args)
version, subject, issuer, fingerprint = TlsGetCertInfo()
if version == nil then
return 0
end
end
TlsGetCertChain¶
Make certificate chain available to the script through TlsGetCertChain.
The output is an array of certificate with each certificate being an hash with data and length keys.
Example:
-- Use debian lua-luaossl coming from https://github.com/wahern/luaossl
local x509 = require"openssl.x509"
chain = TlsGetCertChain()
for k, v in pairs(chain) do
-- v.length is length of data
-- v.data is raw binary data of certificate
cert = x509.new(v["data"], "DER")
print(cert:text() .. "\n")
end
TlsGetCertNotAfter¶
Get the Unix timestamp of end of validity of certificate.
Example:
function log (args)
notafter = TlsGetCertNotAfter()
if notafter < os.time() then
-- expired certificate
end
end
TlsGetCertNotBefore¶
Get the Unix timestamp of beginning of validity of certificate.
Example:
function log (args)
notbefore = TlsGetCertNotBefore()
if notbefore > os.time() then
-- not yet valid certificate
end
end
TlsGetCertSerial¶
Get TLS certificate serial number through TlsGetCertSerial.
Example:
function log (args)
serial = TlsGetCertSerial()
if serial then
-- do something
end
end
TlsGetSNI¶
Get the Server name Indication from a TLS connection.
Example:
function log (args)
asked_domain = TlsGetSNI()
if string.find(asked_domain, "badguys") then
-- ok connection to bad guys let's do someting
end
end
JA3¶
JA3 must be enabled in the Suricata config file (set ‘app-layer.protocols.tls.ja3-fingerprints’ to ‘yes’).
Initialize with:
function init (args)
local needs = {}
needs["protocol"] = "tls"
return needs
end
Ja3GetHash¶
Get the JA3 hash (md5sum of JA3 string) through Ja3GetHash.
Example:
function log (args)
hash = Ja3GetHash()
if hash == nil then
return
end
end
Ja3GetString¶
Get the JA3 string through Ja3GetString.
Example:
function log (args)
str = Ja3GetString()
if str == nil then
return
end
end
SSH¶
Initialize with:
function init (args)
local needs = {}
needs["protocol"] = "ssh"
return needs
end
SshGetServerProtoVersion¶
Get SSH protocol version used by the server through SshGetServerProtoVersion.
Example:
function log (args)
version = SshGetServerProtoVersion()
if version == nil then
return 0
end
end
SshGetServerSoftwareVersion¶
Get SSH software used by the server through SshGetServerSoftwareVersion.
Example:
function log (args)
software = SshGetServerSoftwareVersion()
if software == nil then
return 0
end
end
SshGetClientProtoVersion¶
Get SSH protocol version used by the client through SshGetClientProtoVersion.
Example:
function log (args)
version = SshGetClientProtoVersion()
if version == nil then
return 0
end
end
SshGetClientSoftwareVersion¶
Get SSH software used by the client through SshGetClientSoftwareVersion.
Example:
function log (args)
software = SshGetClientSoftwareVersion()
if software == nil then
return 0
end
end
Files¶
To use the file logging API, the script’s init() function needs to look like:
function init (args)
local needs = {}
needs['type'] = 'file'
return needs
end
SCFileInfo¶
fileid, txid, name, size, magic, md5 = SCFileInfo()
returns fileid (number), txid (number), name (string), size (number), magic (string), md5 in hex (string)
Alerts¶
Alerts are a subset of the ‘packet’ logger:
function init (args)
local needs = {}
needs["type"] = "packet"
needs["filter"] = "alerts"
return needs
end
SCRuleIds¶
sid, rev, gid = SCRuleIds()
SCRuleMsg¶
msg = SCRuleMsg()
SCRuleClass¶
class, prio = SCRuleClass()
Streaming Data¶
Streaming data can currently log out reassembled TCP data and normalized HTTP data. The script will be invoked for each consecutive data chunk.
In case of TCP reassembled data, all possible overlaps are removed according to the host OS settings.
function init (args)
local needs = {}
needs["type"] = "streaming"
needs["filter"] = "tcp"
return needs
end
In case of HTTP body data, the bodies are unzipped and dechunked if applicable.
function init (args)
local needs = {}
needs["type"] = "streaming"
needs["protocol"] = "http"
return needs
end
SCStreamingBuffer¶
function log(args)
data = SCStreamingBuffer()
hex_dump(data)
end
Flow variables¶
It is possible to access, define and modify Flow variables from Lua. To do so, you must use the functions described in this section and declare the counter in init function:
function init(args)
local needs = {}
needs["tls"] tostring(true)
needs["flowint"] = {"tls-cnt"}
return needs
end
Here we define a tls-cnt Flowint that can now be used in output or in a signature via dedicted functions. The access to the Flow variable is done by index so in our case we need to use 0.
function match(args)
a = ScFlowintGet(0);
if a then
ScFlowintSet(0, a + 1)
else
ScFlowintSet(0, 1)
end
ScFlowintGet¶
Get the Flowint at index given by the parameter.
ScFlowintSet¶
Set the Flowint at index given by the first parameter. The second parameter is the value.
ScFlowintIncr¶
Increment Flowint at index given by the first parameter.
ScFlowintDecr¶
Decrement Flowint at index given by the first parameter.
ScFlowvarGet¶
Get the Flowvar at index given by the parameter.
ScFlowvarSet¶
Set a Flowvar. First parameter is the index, second is the data and third is the length of data.
You can use it to set string
function init (args)
local needs = {}
needs["http.request_headers"] = tostring(true)
needs["flowvar"] = {"cnt"}
return needs
end
function match(args)
a = ScFlowvarGet(0);
if a then
a = tostring(tonumber(a)+1)
ScFlowvarSet(0, a, #a)
else
a = tostring(1)
ScFlowvarSet(0, a, #a)
end
Misc¶
SCThreadInfo¶
tid, tname, tgroup = SCThreadInfo()
It gives: tid (integer), tname (string), tgroup (string)
SCLogError, SCLogWarning, SCLogNotice, SCLogInfo, SCLogDebug¶
Print a message. It will go into the outputs defined in the yaml. Whether it will be printed depends on the log level.
Example:
SCLogError("some error message")
SCLogPath¶
Expose the log path.
name = "fast_lua.log"
function setup (args)
filename = SCLogPath() .. "/" .. name
file = assert(io.open(filename, "a"))
end
File Extraction¶
Architecture¶
The file extraction code works on top of the some protocols parsers. The application layer parsers runs on top of the stream reassembly engine and the UDP flow tracking.
In case of HTTP, the parser takes care of dechunking and unzipping the request and/or response data if necessary.
This means that settings in the stream engine, reassembly engine and the application layer parsers all affect the workings of the file extraction.
What files are actually extracted and stored to disk is controlled by the rule language.
Supported protocols are:
- HTTP
- SMTP
- FTP
- NFS
- SMB
Settings¶
stream.checksum_validation controls whether or not the stream engine rejects packets with invalid checksums. A good idea normally, but the network interface performs checksum offloading a lot of packets may seem to be broken. This setting is enabled by default, and can be disabled by setting to “no”. Note that the checksum handling can be controlled per interface, see “checksum_checks” in example configuration.
file-store.stream-depth controls how far into a stream reassembly is done. Beyond this value no reassembly will be done. This means that after this value the HTTP session will no longer be tracked. By default a settings of 1 Megabyte is used. 0 sets it to unlimited. If set to no, it is disabled and stream.reassembly.depth is considered.
libhtp.default-config.request-body-limit / libhtp.server-config.<config>.request-body-limit controls how much of the HTTP request body is tracked for inspection by the http_client_body keyword, but also used to limit file inspection. A value of 0 means unlimited.
libhtp.default-config.response-body-limit / libhtp.server-config.<config>.response-body-limit is like the request body limit, only it applies to the HTTP response body.
Output¶
File-Store and Eve Fileinfo¶
There are two output modules for logging information about files
extracted. The first is eve.files which is an eve sub-logger
that logs fileinfo records. These fileinfo records provide
metadata about the file, but not the actual file contents.
This must be enabled in the eve output:
- outputs:
- eve-log:
types:
- files:
force-magic: no
force-hash: [md5,sha256]
See Eve (Extensible Event Format) for more details on working with the eve output.
The other output module, file-store stores the actual files to
disk.
The file-store uses its own log directory (default: filestore in
the default logging directory) and logs files using the SHA256 of the
contents as the filename. Each file is then placed in a directory
named 00 to ff where the directory shares the first 2 characters
of the filename. For example, if the SHA256 hex string of an extracted
file starts with “f9bc6d…” the file we be placed in the directory
filestore/f9.
Using the SHA256 for file names allows for automatic de-duplication of extracted files. However, the timestamp of a pre-existing file will be updated if the same files is extracted again, similar to the touch command.
Optionally a fileinfo record can be written to its own file
sharing the same SHA256 as the file it references. To handle recording
the metadata of each occurrence of an extracted file, these filenames
include some extra fields to ensure uniqueness. Currently the format
is:
<SHA256>.<SECONDS>.<ID>.json
where <SECONDS> is the seconds from the packet that triggered the
stored file to be closed and <ID> is a unique ID for the runtime
of the Suricata instance. These values should not be depended on, and
are simply used to ensure uniqueness.
These fileinfo records are idential to the fileinfo records
logged to the eve output.
See File-store (File Extraction) for more information on configuring the file-store output.
注解
This section documents version 2 of the file-store.
File-Store (Version 1)¶
File-store version 1 has been replaced by version 2 and is no longer recommended.
- file-store:
enabled: yes # set to yes to enable
log-dir: files # directory to store the files
force-magic: no # force logging magic on all stored files
force-md5: no # force logging of md5 checksums
stream-depth: 1mb # reassemble 1mb into a stream, set to no to disable
waldo: file.waldo # waldo file to store the file_id across runs
max-open-files: 0 # how many files to keep open (O means none)
write-meta: yes # write a .meta file if set to yes
include-pid: yes # include the pid in filenames if set to yes.
Each file that is stored will have a name “file.<id>”. The id will be reset and files will be overwritten unless the waldo option is used. A “file.<id>.meta” file is generated containing file metadata if write-meta is set to yes (default). If the include-pid option is set, the files will instead have a name “file.<pid>.<id>”, and metafiles will be “file.<pid>.<id>.meta”. Files will additionally have the suffix “.tmp” while they are open, which is only removed when they are finalized.
File-Log (Deprecated)¶
The file-log is a standalone logger that logs information about files
extracted. It is now deprecated and should not be used. It has been
replaced by eve.files in the Eve (Extensible Event Format)
output.
- file-log:
enabled: yes
filename: files-json.log
append: yes
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
force-magic: no # force logging magic on all logged files
force-md5: no # force logging of md5 checksums
Rules¶
Without rules in place no extraction will happen. The simplest rule would be:
alert http any any -> any any (msg:"FILE store all"; filestore; sid:1; rev:1;)
This will simply store all files to disk.
Want to store all files with a pdf extension?
alert http any any -> any any (msg:"FILE PDF file claimed"; fileext:"pdf"; filestore; sid:2; rev:1;)
Or rather all actual pdf files?
alert http any any -> any any (msg:"FILE pdf detected"; filemagic:"PDF document"; filestore; sid:3; rev:1;)
Or rather only store files from black list checksum md5 ?
alert http any any -> any any (msg:"Black list checksum match and extract MD5"; filemd5:fileextraction-chksum.list; filestore; sid:4; rev:1;)
Or only store files from black list checksum sha1 ?
alert http any any -> any any (msg:"Black list checksum match and extract SHA1"; filesha1:fileextraction-chksum.list; filestore; sid:5; rev:1;)
Or finally store files from black list checksum sha256 ?
alert http any any -> any any (msg:"Black list checksum match and extract SHA256"; filesha256:fileextraction-chksum.list; filestore; sid:6; rev:1;)
Bundled with the Suricata download is a file with more example rules. In the archive, go to the rules/ directory and check the files.rules file.
MD5¶
Suricata can calculate MD5 checksums of files on the fly and log them. See Storing MD5s checksums for an explanation on how to enable this.
Storing MD5s checksums¶
Configuration¶
In the suricata yaml:
- file-store:
enabled: yes # set to yes to enable
log-dir: files # directory to store the files
force-magic: yes # force logging magic on all stored files
force-md5: yes # force logging of md5 checksums
#waldo: file.waldo # waldo file to store the file_id across runs
Optionally, for JSON output:
- file-log:
enabled: yes
filename: files-json.log
append: no
Other settings affecting File Extraction
stream:
memcap: 64mb
checksum-validation: yes # reject wrong csums
inline: no # no inline mode
reassembly:
memcap: 32mb
depth: 0 # reassemble all of a stream
toserver-chunk-size: 2560
toclient-chunk-size: 2560
Make sure we have depth: 0 so all files can be tracked fully.
libhtp:
default-config:
personality: IDS
# Can be specified in kb, mb, gb. Just a number indicates
# it's in bytes.
request-body-limit: 0
response-body-limit: 0
Make sure we have request-body-limit: 0 and response-body-limit: 0
Testing¶
For the purpose of testing we use this rule only in a file.rules (a test/example file):
alert http any any -> any any (msg:"FILE store all"; filestore; sid:1; rev:1;)
This rule above will save all the file data for files that are opened/downloaded through HTTP
Start Suricata (-S option loads ONLY the specified rule file, with disregard if any other rules that are enabled in suricata.yaml):
suricata -c /etc/suricata/suricata.yaml -S file.rules -i eth0
Meta data:
TIME: 05/01/2012-11:09:52.425751
SRC IP: 2.23.144.170
DST IP: 192.168.1.91
PROTO: 6
SRC PORT: 80
DST PORT: 51598
HTTP URI: /en/US/prod/collateral/routers/ps5855/prod_brochure0900aecd8019dc1f.pdf
HTTP HOST: www.cisco.com
HTTP REFERER: http://www.cisco.com/c/en/us/products/routers/3800-series-integrated-services-routers-isr/index.html
FILENAME: /en/US/prod/collateral/routers/ps5855/prod_brochure0900aecd8019dc1f.pdf
MAGIC: PDF document, version 1.6
STATE: CLOSED
MD5: 59eba188e52467adc11bf2442ee5bf57
SIZE: 9485123
and in files-json.log (or eve.json) :
{ "id": 1, "timestamp": "05\/01\/2012-11:10:27.693583", "ipver": 4, "srcip": "2.23.144.170", "dstip": "192.168.1.91", "protocol": 6, "sp": 80, "dp": 51598, "http_uri": "\/en\/US\/prod\/collateral\/routers\/ps5855\/prod_brochure0900aecd8019dc1f.pdf", "http_host": "www.cisco.com", "http_referer": "http:\/\/www.google.com\/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.cisco.com%2Fen%2FUS%2Fprod%2Fcollateral%2Frouters%2Fps5855%2Fprod_brochure0900aecd8019dc1f.pdf&ei=OqyfT9eoJubi4QTyiamhAw&usg=AFQjCNGdjDBpBDfQv2r3VogSH41V6T5x9Q", "filename": "\/en\/US\/prod\/collateral\/routers\/ps5855\/prod_brochure0900aecd8019dc1f.pdf", "magic": "PDF document, version 1.6", "state": "CLOSED", "md5": "59eba188e52467adc11bf2442ee5bf57", "stored": true, "size": 9485123 }
{ "id": 12, "timestamp": "05\/01\/2012-11:12:57.421420", "ipver": 4, "srcip": "2.23.144.170", "dstip": "192.168.1.91", "protocol": 6, "sp": 80, "dp": 51598, "http_uri": "\/en\/US\/prod\/collateral\/routers\/ps5855\/prod_brochure0900aecd8019dc1f.pdf", "http_host": "www.cisco.com", "http_referer": "http:\/\/www.google.com\/url?sa=t&rct=j&q=&esrc=s&source=web&cd=1&ved=0CDAQFjAA&url=http%3A%2F%2Fwww.cisco.com%2Fen%2FUS%2Fprod%2Fcollateral%2Frouters%2Fps5855%2Fprod_brochure0900aecd8019dc1f.pdf&ei=OqyfT9eoJubi4QTyiamhAw&usg=AFQjCNGdjDBpBDfQv2r3VogSH41V6T5x9Q", "filename": "\/en\/US\/prod\/collateral\/routers\/ps5855\/prod_brochure0900aecd8019dc1f.pdf", "magic": "PDF document, version 1.6", "state": "CLOSED", "md5": "59eba188e52467adc11bf2442ee5bf57", "stored": true, "size": 9485123 }
Log all MD5s without any rules¶
If you would like to log MD5s for everything and anything that passes through the traffic that you are inspecting with Suricata, but not log the files themselves, all you have to do is disable file-store and enable only the JSON output with forced MD5s - in suricata.yaml like so:
- file-store:
enabled: no # set to yes to enable
log-dir: files # directory to store the files
force-magic: yes # force logging magic on all stored files
force-md5: yes # force logging of md5 checksums
#waldo: file.waldo # waldo file to store the file_id across runs
- file-log:
enabled: yes
filename: files-json.log
append: no
#filetype: regular # 'regular', 'unix_stream' or 'unix_dgram'
force-magic: yes # force logging magic on all logged files
force-md5: yes # force logging of md5 checksums
Public SHA1 MD5 data sets¶
National Software Reference Library - http://www.nsrl.nist.gov/Downloads.htm
Public Data Sets¶
Collections of pcaps for testing, profiling.
DARPA sets: http://www.ll.mit.edu/mission/communications/cyber/CSTcorpora/ideval/data/
MAWI sets (pkt headers only, no payloads): http://mawi.wide.ad.jp/mawi/samplepoint-F/2012/
MACCDC: http://www.netresec.com/?page=MACCDC
Netresec: http://www.netresec.com/?page=PcapFiles
Wireshark: https://wiki.wireshark.org/SampleCaptures
Security Onion collection: https://github.com/security-onion-solutions/security-onion/wiki/Pcaps
Stratosphere IPS. Malware Capture Facility Project: https://stratosphereips.org/category/dataset.html
Using Capture Hardware¶
Endace DAG¶
Suricata comes with native Endace DAG card support. This means Suricata can use the libdag interface directly, instead of a libpcap wrapper (which should also work).
Steps:
Configure with DAG support:
./configure --enable-dag --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
sudo make install
Results in:
Suricata Configuration:
AF_PACKET support: no
PF_RING support: no
NFQueue support: no
IPFW support: no
DAG enabled: yes
Napatech enabled: no
Start with:
suricata -c suricata.yaml --dag 0:0
Started up!
[5570] 10/7/2012 -- 13:52:30 - (source-erf-dag.c:262) <Info> (ReceiveErfDagThreadInit) -- Attached and started stream: 0 on DAG: /dev/dag0
[5570] 10/7/2012 -- 13:52:30 - (source-erf-dag.c:288) <Info> (ReceiveErfDagThreadInit) -- Starting processing packets from stream: 0 on DAG: /dev/dag0
Napatech Suricata Installation Guide¶
Contents¶
- Introduction
- Package Installation
- Basic Configuration
- Advanced Multithreaded Configuration
Introduction¶
Napatech packet capture accelerator cards can greatly improve the performance of your Suricata deployment using these hardware based features:
- On board burst buffering (up to 12GB)
- Zero-copy kernel bypass DMA
- Non-blocking PCIe performance
- Port merging
- Load distribution to up 128 host buffers
- Precise timestamping
- Accurate time synchronization
The Napatech Software Suite (driver package) comes in two varieties, NAC and OEM. The NAC package distributes deb and rpm packages to ease the installation. The OEM package uses a proprietary shell script to handle the installation process. In either case, gcc, make and the kernel header files are required to compile the kernel module and install the software.
Package Installation¶
Note that make, gcc, and the kernel headers are required for installation
Root privileges are also required
Napatech NAC Package¶
Red Hat Based Distros:
$ yum install kernel-devel-$(uname -r) gcc make ncurses-libs
$ yum install nac-pcap-<release>.x86_64.rpm
Some distributions will require you to use the –nogpgcheck option with yum for the NAC Software Suite package file:
$ yum --nogpgcheck install nac-pcap-<release>.x86_64.rpm
Debian Based Distros:
$ apt-get install linux-headers-$(uname .r) gcc make libncurses5
$ dpkg .i nac-pcap_<release>_amd64.deb
To complete installation for all distros stop ntservice:
$ /opt/napatech3/bin/ntstop.sh -m
Remove these existing setup files:
$ cd /opt/napatech3/config
$ rm ntservice.ini setup.ini
Restart ntservice (a new ntservice.ini configuration file will be generated automatically):
$ /opt/napatech3/bin/ntstart.sh -m
Napatech OEM Package¶
Note that you will be prompted to install the Napatech libpcap library. Answer “yes” if you would like to use the Napatech card to capture packets in WIreshark, tcpdump, or another pcap based application. Libpcap is not needed for Suricata as native Napatech API support is included
Red Hat Based Distros:
$ yum install kernel-devel-$(uname -r) gcc make
$ ./package_install_3gd.sh
Debian Based Distros:
$ apt-get install linux-headers-$(uname .r) gcc make
$ ./package_install_3gd.sh
To complete installation for all distros ntservice:
$ /opt/napatech3/bin/ntstart.sh -m
Suricata Installation¶
After downloading and extracting the Suricata tarball, you need to run configure to enable Napatech support and prepare for compilation:
$ ./configure --enable-napatech --with-napatech-includes=/opt/napatech3/include --with-napatech-libraries=/opt/napatech3/lib
$ make
$ make install-full
Now edit the suricata.yaml file to configure the maximum number of streams to use. If you plan on using the load distribution (RSS - like) feature in the Napatech accelerator, then the list should contain the same number of streams as host buffers defined in ntservice.ini:
Napatech:
# The Host Buffer Allowance for all streams
# (-1 = OFF, 1 - 100 = percentage of the host buffer that can be held back)
hba: -1
# use_all_streams set to "yes" will query the Napatech service for all configured
# streams and listen on all of them. When set to "no" the streams config array
# will be used.
use-all-streams: yes
# The streams to listen on
streams: [0, 1, 2, 3, 4, 5, 6, 7]
Note: hba is useful only when a stream is shared with another application. When hba is enabled packets will be dropped (i.e. not delivered to suricata) when the host-buffer utilization reaches the high-water mark indicated by the hba value. This insures that, should suricata get behind in it’s packet processing, the other application will still receive all of the packets. If this is enabled without another application sharing the stream it will result in sub-optimal packet buffering.
Basic Configuration¶
For the basic installation we will setup the Napatech capture accelerator to merge all physical ports into single stream that Suricata can read from. for this configuration, Suricata will handle the packet distribution to multiple threads.
Here are the lines that need changing in /opt/napatech3/bin/ntservice.ini for best single buffer performance:
TimeSyncReferencePriority = OSTime # Timestamp clock synchronized to the OS
HostBuffersRx = [1,16,0] # [number of host buffers, Size(MB), NUMA node]
Stop and restart ntservice after making changes to ntservice:
$ /opt/napatech3/bin/ntstop.sh -m
$ /opt/napatech3/bin/ntstart.sh -m
Now we need to execute a few NTPL (Napatech Programming Language) commands to complete the setup. Create a file will the following commands:
Delete=All # Delete any existing filters
Setup[numaNode=0] = streamid==0 # Set stream ID 0 to NUMA 0
Assign[priority=0; streamid=0]= all # Assign all phisical ports to stream ID 0
Next execute those command using the ntpl tool:
$ /opt/napatech3/bin/ntpl -f <my_ntpl_file>
Now you are ready to start suricata:
$ suricata -c /usr/local/etc/suricata/suricata.yaml --napatech --runmode workers
Advanced Multithreaded Configuration¶
Now let’s do a more advanced configuration where we will use the load distribution (RSS - like) capability in the accelerator. We will create 8 streams and setup the accelerator to distribute the load based on a 5 tuple hash. Increasing buffer size will minimize packet loss only if your CPU cores are fully saturated. Setting the minimum buffer size (16MB) will gave the best performance (minimize L3 cache hits) if your CPU cores are keeping up.
Note that it is extremely important that the NUMA node the host buffers are define in is the same physical CPU socket that the Napatech accelerator is plugged into
First let’s modify the ntservice.ini file to increase the number and size of the host buffers:
HostBuffersRx = [8,256,0] # [number of host buffers, Size (MB), NUMA node]
Stop and restart ntservice after making changes to ntservice:
$ /opt/napatech3/bin/ntstop.sh -m
$ /opt/napatech3/bin/ntstart.sh -m
Now let’s assign the streams to host buffers and configure the load distribution. The load distribution will be setup to support both tunneled and non-tunneled traffic. Create a file that contains the ntpl commands below:
Delete=All # Delete any existing filters
Setup[numaNode=0] = streamid==0
Setup[numaNode=0] = streamid==1
Setup[numaNode=0] = streamid==2
Setup[numaNode=0] = streamid==3
Setup[numaNode=0] = streamid==4
Setup[numaNode=0] = streamid==5
Setup[numaNode=0] = streamid==6
Setup[numaNode=0] = streamid==7
HashMode[priority=4]=Hash5TupleSorted
Assign[priority=0; streamid=(0..7)]= all
Next execute those command using the ntpl tool:
$ /opt/napatech3/bin/ntpl -f <my_ntpl_file>
Now you are ready to start Suricata:
$ suricata -c /usr/local/etc/suricata/suricata.yaml --napatech --runmode workers
Counters¶
For each stream that is being processed the following counters will be output in stats.log:
- nt<streamid>.pkts - The number of packets recieved by the stream.
- nt<streamid>.bytes - The total bytes received by the stream.
- nt<streamid>.drop - The number of packets that were dropped from this stream due to buffer overflow conditions.
If hba is enabled the following counter will also be provided:
- nt<streamid>.hba_drop - the number of packets dropped because the host buffer allowance high-water mark was reached.
In addition to counters host buffer utilization is tracked and logged. This is also useful for debugging. Log messages are output for both Host and On-Board buffers when reach 25, 50, 75 percent of utilization. Corresponding messages are output when utilization decreases.
Support¶
Contact a support engineer at: ntsupport@napatech.com
Myricom¶
From: https://blog.inliniac.net/2012/07/10/suricata-on-myricom-capture-cards/
In this guide I’ll describe using the Myricom libpcap support. I’m going to assume you installed the card properly, installed the Sniffer driver and made sure that all works. Make sure that in your dmesg you see that the card is in sniffer mode:
[ 2102.860241] myri_snf INFO: eth4: Link0 is UP
[ 2101.341965] myri_snf INFO: eth5: Link0 is UP
I have installed the Myricom runtime and libraries in /opt/snf
Compile Suricata against Myricom’s libpcap:
./configure --with-libpcap-includes=/opt/snf/include/ --with-libpcap-libraries=/opt/snf/lib/ --prefix=/usr --sysconfdir=/etc --localstatedir=/var
make
sudo make install
Next, configure the amount of ringbuffers. I’m going to work with 8 here, as my quad core + hyper threading has 8 logical CPU’s. See below for additional information about the buffer-size parameter.
pcap:
- interface: eth5
threads: 8
buffer-size: 512kb
checksum-checks: no
The 8 threads setting makes Suricata create 8 reader threads for eth5. The Myricom driver makes sure each of those is attached to it’s own ringbuffer.
Then start Suricata as follows:
SNF_NUM_RINGS=8 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
If you want 16 ringbuffers, update the “threads” variable in your yaml to 16 and start Suricata:
SNF_NUM_RINGS=16 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
Note that the pcap.buffer-size yaml setting shown above is currently ignored when using Myricom cards. The value is passed through to the pcap_set_buffer_size libpcap API within the Suricata source code. From Myricom support:
“The libpcap interface to Sniffer10G ignores the pcap_set_buffer_size() value. The call to snf_open() uses zero as the dataring_size which informs the Sniffer library to use a default value or the value from the SNF_DATARING_SIZE environment variable."
The following pull request opened by Myricom in the libpcap project indicates that a future SNF software release could provide support for setting the SNF_DATARING_SIZE via the pcap.buffer-size yaml setting:
Until then, the data ring and descriptor ring values can be explicitly set using the SNF_DATARING_SIZE and SNF_DESCRING_SIZE environment variables, respectively.
The SNF_DATARING_SIZE is the total amount of memory to be used for storing incoming packet data. This size is shared across all rings. The SNF_DESCRING_SIZE is the total amount of memory to be used for storing meta information about the packets (packet lengths, offsets, timestamps). This size is also shared across all rings.
Myricom recommends that the descriptor ring be 1/4 the size of the data ring, but the ratio can be modified based on your traffic profile. If not set explicitly, Myricom uses the following default values: SNF_DATARING_SIZE = 256MB, and SNF_DESCRING_SIZE = 64MB
Expanding on the 16 thread example above, you can start Suricata with a 16GB Data Ring and a 4GB Descriptor Ring using the following command:
SNF_NUM_RINGS=16 SNF_DATARING_SIZE=17179869184 SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 suricata -c suricata.yaml -i eth5 --runmode=workers
Debug Info¶
Myricom also provides a means for obtaining debug information. This can be useful for verifying your configuration and gathering additional information. Setting SNF_DEBUG_MASK=3 enables debug information, and optionally setting the SNF_DEBUG_FILENAME allows you to specify the location of the output file.
Following through with the example:
SNF_NUM_RINGS=16 SNF_DATARING_SIZE=17179869184 SNF_DESCRING_SIZE=4294967296 SNF_FLAGS=0x1 SNF_DEBUG_MASK=3 SNF_DEBUG_FILENAME="/tmp/snf.out" suricata -c suricata.yaml -i eth5 --runmode=workers
eBPF and XDP¶
Introduction¶
eBPF stands for extended BPF. This is an extended version of Berkeley Packet Filter available in recent Linux kernel versions.
It provides more advanced features with eBPF programs developed in C and capability to use structured data shared between kernel and userspace.
eBPF is used for three things in Suricata:
- eBPF filter: any BPF like filter can be developed. An example of filter accepting only packet for some VLANs is provided. A bypass implementation is also provided.
- eBPF load balancing: provide programmable load balancing. A simple ippair load balancing is provided.
- XDP programs: suricata can load XDP programs. A bypass program is provided.
Bypass can be implemented in eBPF and XDP. The advantage of XDP is that the packets are dropped at the earliest stage possible. So performance is better. But bypassed packets don’t reach the network so you can’t use this on regular traffic but only on duplicated/sniffed traffic.
XDP¶
XDP provides another Linux native way of optimising Suricata’s performance on sniffing high speed networks.
XDP or eXpress Data Path provides a high performance, programmable network data path in the Linux kernel as part of the IO Visor Project. XDP provides bare metal packet processing at the lowest point in the software stack which makes it ideal for speed without compromising programmability. Furthermore, new functions can be implemented dynamically with the integrated fast path without kernel modification.
Requirements¶
You will need a kernel that supports XDP and, for real performance improvement, a network card that support XDP in the driver.
Suricata XDP code has been tested with 4.13.10 but 4.15 or later is necessary to use all features like the CPU redirect map.
If you are using an Intel netword card, you will need to stay with in tree kernel NIC drivers. The out of tree drivers do not contain the XDP support.
Having a network card with support for RSS symmetric hashing is a good point or you will have to use the XDP CPU redirect map feature.
Prerequisites¶
This guide has been confirmed on Debian/Ubuntu “LTS” Linux.
Disable irqbalance¶
systemctl stop irqbalance
systemctl disable irqbalance
Kernel¶
You need to run a kernel 4.13 or newer.
libbpf¶
Suricata uses libbpf to interact with eBPF and XDP. This library is available in the Linux tree. Before Linux 4.16, a patched libbpf library is also needed:
git clone -b libbpf-release https://github.com/regit/linux.git
If you have a recent enough kernel, you can skip this part.
Now, you can build and install the library
cd linux/tools/lib/bpf/
make && sudo make install
sudo make install_headers
sudo ldconfig
Compile and install Suricata¶
To get Suricata source, you can use the usual
git clone https://github.com/OISF/suricata.git
cd suricata && git clone https://github.com/OISF/libhtp.git -b 0.5.x
./autogen.sh
Then you need to add the ebpf flags to configure
CC=clang ./configure --prefix=/usr/ --sysconfdir=/etc/ --localstatedir=/var/ \
--enable-ebpf --enable-ebpf-build
make clean && make
sudo make install-full
sudo ldconfig
sudo mkdir /etc/suricata/ebpf/
Setup bypass¶
If you plan to use eBPF or XDP for a kernel/hardware level bypass, you need to do the following:
First, enable bypass in the stream section
stream:
bypass: true
If you want, you can also bypass encrypted flows by setting encrypt-handling to bypass in the app-layer tls section
app-layer:
protocols:
tls:
enabled: yes
detection-ports:
dp: 443
encrypt-handling: bypass
Setup eBPF filter¶
The file ebpf/vlan_filter.c contains a list of vlan id in a switch that you need to edit to get something adapted to your network. Another really basic filter dropping IPv6 packets is also available in ebpf/filter.c.
Suricata can load as eBPF filter any eBPF code exposing a filter section.
Once modifications and build via make are done, you can copy the resulting eBPF filter as needed
cp ebpf/vlan_filter.bpf /etc/suricata/ebpf/
Then setup the ebpf-filter-file variable in af-packet section
- interface: eth3
threads: 16
cluster-id: 97
cluster-type: cluster_flow # choose any type suitable
defrag: yes
# eBPF file containing a 'loadbalancer' function that will be inserted into the
# kernel and used as load balancing function
ebpf-filter-file: /etc/suricata/ebpf/vlan_filter.bpf
use-mmap: yes
ring-size: 200000
You can then run suricata normally
/usr/bin/suricata --pidfile /var/run/suricata.pid --af-packet=eth3 -vvv
Setup eBPF bypass¶
You can also use eBPF bypass. To do that load the bypass_filter.bpf file and update af-packet configuration to set bypass to yes
- interface: eth3
threads: 16
cluster-id: 97
cluster-type: cluster_qm # symmetric RSS hashing is mandatory to use this mode
# eBPF file containing a 'filter' function that will be inserted into the
# kernel and used as packet filter function
ebpf-filter-file: /etc/suricata/ebpf/bypass_filter.bpf
bypass: yes
use-mmap: yes
ring-size: 200000
Constraints on eBPF code to have a bypass compliant code are stronger than for regular filter. The filter must expose flow_table_v4 and flow_table_v6 per CPU array maps with similar definitions as the one available in bypass_filter.c. These two maps will be accessed and maintained by Suricata to handle the lists of flow to bypass.
Setup eBPF load balancing¶
eBPF load balancing allows to load balance the traffic on the listening sockets
With any logic implemented in the eBPF filter. The value returned by the function
tagged with the loadbalancer section is used with a modulo on the CPU count to know in
which socket the packet has to be send.
An implementation of a simple IP pair hashing function is provided in the lb.bpf
file.
Copy the resulting eBPF filter as needed
cp ebpf/lb.bpf /etc/suricata/ebpf/
Then use cluster_ebpf as load balancing method in the interface section of af-packet
and point the ebpf-lb-file variable to the lb.bpf file
- interface: eth3
threads: 16
cluster-id: 97
cluster-type: cluster_ebpf
defrag: yes
# eBPF file containing a 'loadbalancer' function that will be inserted into the
# kernel and used as load balancing function
ebpf-lb-file: /etc/suricata/ebpf/lb.bpf
use-mmap: yes
ring-size: 200000
Setup XDP bypass¶
XDP bypass will allow Suricata to tell the kernel that packets for some flows have to be dropped via the XDP mechanism. This is a really early drop that occurs before the datagram is reaching the Linux kernel network stack.
Linux 4.15 or newer are recommended to use that feature. You can use it
on older kernel if you set BUILD_CPUMAP to 0 in ebpf/xdp_filter.c.
Copy the resulting xdp filter as needed:
cp ebpf/xdp_filter.bpf /etc/suricata/ebpf/
Setup af-packet section/interface in suricata.yaml.
We will use cluster_qm as we have symmetric hashing on the NIC, xdp-mode: driver and we will
also use the /etc/suricata/ebpf/xdp_filter.bpf (in our example TCP offloading/bypass)
- interface: eth3
threads: 16
cluster-id: 97
cluster-type: cluster_qm # symmetric hashing is a must!
defrag: yes
# Xdp mode, "soft" for skb based version, "driver" for network card based
# and "hw" for card supporting eBPF.
xdp-mode: driver
xdp-filter-file: /etc/suricata/ebpf/xdp_filter.bpf
# if the ebpf filter implements a bypass function, you can set 'bypass' to
# yes and benefit from these feature
bypass: yes
use-mmap: yes
ring-size: 200000
XDP bypass is compatible with AF_PACKET IPS mode. Packets from bypassed flows will be send directly from one card to the second card without going by the kernel network stack.
Setup symmetric hashing on the NIC¶
Intel network card don’t support symmetric hashing but it is possible to emulate it by using a specific hashing function.
Follow these instructions closely for desired result:
ifconfig eth3 down
Use in tree kernel drivers: XDP support is not available in Intel drivers available on Intel website.
Enable symmetric hashing
ifconfig eth3 down
ethtool -L eth3 combined 16 # if you have at least 16 cores
ethtool -K eth3 rxhash on
ethtool -K eth3 ntuple on
ifconfig eth3 up
./set_irq_affinity 0-15 eth3
ethtool -X eth3 hkey 6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A:6D:5A equal 16
ethtool -x eth3
ethtool -n eth3
In the above setup you are free to use any recent set_irq_affinity script. It is available in any Intel x520/710 NIC sources driver download.
NOTE: We use a special low entropy key for the symmetric hashing. More info about the research for symmetric hashing set up
Disable any NIC offloading¶
Run the following command to disable offloading
for i in rx tx tso ufo gso gro lro tx nocache copy sg txvlan rxvlan; do
/sbin/ethtool -K eth3 $i off 2>&1 > /dev/null;
done
Balance as much as you can¶
Try to use the network’s card balancing as much as possible
for proto in tcp4 udp4 ah4 esp4 sctp4 tcp6 udp6 ah6 esp6 sctp6; do
/sbin/ethtool -N eth3 rx-flow-hash $proto sdfn
done
The XDP CPU redirect case¶
If ever your hardware is not able to do a symmetric load balancing but support XDP in driver mode, you can then use the CPU redirect map support available in the xdp_filter.bpf file. In this mode, the load balancing will be done by the XDP filter and each CPU will handle the whole packet treatment including the creation of the skb structure in kernel.
You will need Linux 4.15 or newer to use that feature.
To do so set the xdp-cpu-redirect variable in af-packet interface configuration to a set of CPUs. Then use the cluster_cpu as load balancing function. You will also need to set the affinity accordingly.
It is possible to use xdp_monitor to have information about the behavior of CPU redirect. This program is available in Linux tree under the samples/bpf directory and will be build by the make command. Sample output is the following
sudo ./xdp_monitor --stats
XDP-event CPU:to pps drop-pps extra-info
XDP_REDIRECT 11 2,880,212 0 Success
XDP_REDIRECT total 2,880,212 0 Success
XDP_REDIRECT total 0 0 Error
cpumap-enqueue 11:0 575,954 0 5.27 bulk-average
cpumap-enqueue sum:0 575,954 0 5.27 bulk-average
cpumap-kthread 0 575,990 0 56,409 sched
cpumap-kthread 1 576,090 0 54,897 sched
Start Suricata with XDP¶
You can now start Suricata with XDP bypass activated
/usr/bin/suricata -c /etc/suricata/xdp-suricata.yaml --pidfile /var/run/suricata.pid --af-packet=eth3 -vvv
Confirm you have the XDP filter engaged in the output (example):
...
...
(runmode-af-packet.c:220) <Config> (ParseAFPConfig) -- Enabling locked memory for mmap on iface eth3
(runmode-af-packet.c:231) <Config> (ParseAFPConfig) -- Enabling tpacket v3 capture on iface eth3
(runmode-af-packet.c:326) <Config> (ParseAFPConfig) -- Using queue based cluster mode for AF_PACKET (iface eth3)
(runmode-af-packet.c:424) <Info> (ParseAFPConfig) -- af-packet will use '/etc/suricata/ebpf/xdp_filter.bpf' as XDP filter file
(runmode-af-packet.c:429) <Config> (ParseAFPConfig) -- Using bypass kernel functionality for AF_PACKET (iface eth3)
(runmode-af-packet.c:609) <Config> (ParseAFPConfig) -- eth3: enabling zero copy mode by using data release call
(util-runmodes.c:296) <Info> (RunModeSetLiveCaptureWorkersForDevice) -- Going to use 8 thread(s)
...
...
Getting live info about bypass¶
You can get information about bypass via the stats event and through the unix socket. ìface-stat will return the number of bypassed packets (adding packets for a flow when it timeout). ebpf-bypassed-stats command will return the number of element in IPv4 and IPv6 flow tables for each interfaces.
Interacting via Unix Socket¶
Introduction¶
Suricata can listen to a unix socket and accept commands from the user. The exchange protocol is JSON-based and the format of the message has been done to be generic.
An example script called suricatasc is provided in the source and installed automatically when installing/updating Suricata.
The unix socket is enabled by default if libjansson is available.
You need to have libjansson installed:
- libjansson4 - C library for encoding, decoding and manipulating JSON data
- libjansson-dev - C library for encoding, decoding and manipulating JSON data (dev)
- python-simplejson - simple, fast, extensible JSON encoder/decoder for Python
Debian/Ubuntu:
apt-get install libjansson4 libjansson-dev python-simplejson
If libjansson is present on the system , unix socket will be compiled in automatically.
The creation of the socket is managed by setting enabled to ‘yes’ or ‘auto’ under unix-command in Suricata YAML configuration file:
unix-command:
enabled: yes
#filename: custom.socket # use this to specify an alternate file
The filename variable can be used to set an alternate socket
filename. The filename is always relative to the local state base
directory.
Clients are implemented for some language and can be used as code example to write custom scripts:
- Python: https://github.com/inliniac/suricata/blob/master/scripts/suricatasc/suricatasc.in (provided with suricata and used in this document)
- Perl: https://github.com/aflab/suricatac (a simple Perl client with interactive mode)
- C: https://github.com/regit/SuricataC (a Unix socket mode client in C without interactive mode)
Commands in standard running mode¶
You may need to install suricatasc if you have not done so, running the following command from scripts/suricatasc
sudo python setup.py install
The set of existing commands is the following:
- command-list: list available commands
- shutdown: this shutdown Suricata
- iface-list: list interfaces where Suricata is sniffing packets
- iface-stat: list statistic for an interface
- help: alias of command-list
- version: display Suricata’s version
- uptime: display Suricata’s uptime
- running-mode: display running mode (workers, autofp, simple)
- capture-mode: display capture system used
- conf-get: get configuration item (see example below)
- dump-counters: dump Suricata’s performance counters
- reopen-log-files: reopen log files (to be run after external log rotation)
- ruleset-reload-rules: reload ruleset and wait for completion
- ruleset-reload-nonblocking: reload ruleset and proceed without waiting
- ruleset-reload-time: return time of last reload
- ruleset-stats: display the number of rules loaded and failed
- ruleset-failed-rules: display the list of failed rules
- memcap-set: update memcap value of an item specified
- memcap-show: show memcap value of an item specified
- memcap-list: list all memcap values available
You can access to these commands with the provided example script which
is named suricatasc. A typical session with suricatasc will looks like:
# suricatasc
Command list: shutdown, command-list, help, version, uptime, running-mode, capture-mode, conf-get, dump-counters, iface-stat, iface-list, quit
>>> iface-list
Success: {'count': 2, 'ifaces': ['eth0', 'eth1']}
>>> iface-stat eth0
Success: {'pkts': 378, 'drop': 0, 'invalid-checksums': 0}
>>> conf-get unix-command.enabled
Success:
"yes"
Commands on the cmd prompt¶
You can use suricatasc directly on the command prompt:
root@debian64:~# suricatasc -c version
{'message': '2.1beta2 RELEASE', 'return': 'OK'}
root@debian64:~#
root@debian64:~# suricatasc -c uptime
{'message': 35264, 'return': 'OK'}
root@debian64:~#
NOTE: You need to quote commands involving more than one argument:
root@debian64:~# suricatasc -c "iface-stat eth0"
{'message': {'pkts': 5110429, 'drop': 0, 'invalid-checksums': 0}, 'return': 'OK'}
root@debian64:~#
Pcap processing mode¶
This mode is one of main motivation behind this code. The idea is to be able to ask to Suricata to treat different pcap files without having to restart Suricata between the files. This provides you a huge gain in time as you don’t need to wait for the signature engine to initialize.
To use this mode, start suricata with your preferred YAML file and
provide the option --unix-socket as argument:
suricata -c /etc/suricata-full-sigs.yaml --unix-socket
It is also possible to specify the socket filename as argument:
suricata --unix-socket=custom.socket
In this last case, you will need to provide the complete path to the
socket to suricatasc. To do so, you need to pass the filename as
first argument of suricatasc:
suricatasc custom.socket
Once Suricata is started, you can use the provided script
suricatasc to connect to the command socket and ask for pcap
treatment:
root@tiger:~# suricatasc
>>> pcap-file /home/benches/file1.pcap /tmp/file1
Success: Successfully added file to list
>>> pcap-file /home/benches/file2.pcap /tmp/file2
Success: Successfully added file to list
>>> pcap-file-continuous /home/pcaps /tmp/dirout
Success: Successfully added file to list
You can add multiple files without waiting the result: they will be
sequentially processed and the generated log/alert files will be put
into the directory specified as second arguments of the pcap-file
command. You need to provide absolute path to the files and directory
as Suricata doesn’t know from where the script has been run. If you pass
a directory instead of a file, all files in the directory will be processed. If
using pcap-file-continuous and passing in a directory, the directory will
be monitored for new files being added until you use pcap-interrupt or
delete/move the directory.
To know how many files are waiting to get processed, you can do:
>>> pcap-file-number
Success: 3
To get the list of queued files, do:
>>> pcap-file-list
Success: {'count': 2, 'files': ['/home/benches/file1.pcap', '/home/benches/file2.pcap']}
To get current processed file:
>>> pcap-current
Success:
"/tmp/test.pcap"
When passing in a directory, you can see last processed time (modified time of last file) in milliseconds since epoch:
>>> pcap-last-processed
Success:
1509138964000
To interrupt directory processing which terminates the current state:
>>> pcap-interrupt
Success:
"Interrupted"
Build your own client¶
The protocol is documented in the following page https://redmine.openinfosecfoundation.org/projects/suricata/wiki/Unix_Socket#Protocol
The following session show what is send (SND) and received (RCV) by the server. Initial negotiation is the following:
# suricatasc
SND: {"version": "0.1"}
RCV: {"return": "OK"}
Once this is done, command can be issued:
>>> iface-list
SND: {"command": "iface-list"}
RCV: {"message": {"count": 1, "ifaces": ["wlan0"]}, "return": "OK"}
Success: {'count': 1, 'ifaces': ['wlan0']}
>>> iface-stat wlan0
SND: {"command": "iface-stat", "arguments": {"iface": "wlan0"}}
RCV: {"message": {"pkts": 41508, "drop": 0, "invalid-checksums": 0}, "return": "OK"}
Success: {'pkts': 41508, 'drop': 0, 'invalid-checksums': 0}
In pcap-file mode, this gives:
>>> pcap-file /home/eric/git/oisf/benches/sandnet.pcap /tmp/bench
SND: {"command": "pcap-file", "arguments": {"output-dir": "/tmp/bench", "filename": "/home/eric/git/oisf/benches/sandnet.pcap"}}
RCV: {"message": "Successfully added file to list", "return": "OK"}
Success: Successfully added file to list
>>> pcap-file-number
SND: {"command": "pcap-file-number"}
RCV: {"message": 1, "return": "OK"}
>>> pcap-file-list
SND: {"command": "pcap-file-list"}
RCV: {"message": {"count": 1, "files": ["/home/eric/git/oisf/benches/sandnet.pcap"]}, "return": "OK"}
Success: {'count': 1, 'files': ['/home/eric/git/oisf/benches/sandnet.pcap']}
There is one thing to be careful about: a Suricata message is sent in multiple send operations. This result in possible incomplete read on client side. The worse workaround is to sleep a bit before trying a recv call. An other solution is to use non blocking socket and retry a recv if the previous one has failed.
Man Pages¶
Suricata¶
SYNOPSIS¶
suricata [OPTIONS] [BPF FILTER]
DESCRIPTION¶
Suricata is a high performance Network IDS, IPS and Network Security Monitoring engine. Open Source and owned by a community run non-profit foundation, the Open Information Security Foundation (OISF).
OPTIONS¶
-
-h¶ 显示使用概要
-
-V¶ 显示Suricata版本
-
-c<path>¶ 指定配置文件路径
-
-T¶ 测试配置
-
-v¶ -v 选项让Suricata输出详细信息。 多次使用这个选项可以得到更多详细信息。
-
-r<path>¶ 以pcap离线模式运行从pcap文件中读取数据。 如果<path>指定一个目录，则将按照修改时间的顺序处理该目录中的所有文件，以保持流状态。
-
--pcap-file-continuous¶ 与-r 选项一起使用以指示运行模式应该保持活动直到被中断。 在以目录方式添加处理文件时，加上这个选项不会重置流状态。
-
-i<interface>¶ -i 选项指定用于嗅探数据包的网卡，此选项将尝试可用的最佳数据包捕获方法。
-
--pcap[=<device>]¶ 以PCAP模式运行。 如果未指定设备，则将使用配置文件中*pcap*部分提供的接口。
-
--af-packet[=<device>]¶ 指定Linux上使用AF_PACKET进行数据包捕获。 如果没有指定设备，则使用yaml配置文件中af-packet部分的设备列表。
-
-q<queue id>¶ 根据NFQUEUE队列ID运行在inline nfqueue模式，这个ID可能会多次提供
-
-s<filename.rules>¶ -s 选项指定一个规则文件，这个文件可以与yaml配置文件中指定的规则文件一起加载使用。
-
-S<filename.rules>¶ -S 选项指定一个规则文件，这个文件会单独加载使用，而不加载yaml配置文件中指定的规则文件。
-
-l<directory>¶ -l 选项指定默认日志目录。 即使您已经在yaml配置文件中设置了default-log-dir，Suricata也会忽略这个设置。如果您没有使用-l选项，Suricata将使用yaml中设置的目录。
-
-D¶ 通常当你使用console运行suricata时，这个console会被suricata占用而不能用来处理其他工作，console关闭时suricata也会被关闭。使用-D选项，suricata会在后台运行，这时你可以使用console干处理其他工作而不会影响到suricata运行。
-
--runmode<runmode>¶ –runmode 选项指定你想要使用的运行模式，此命令行选项可以覆盖yaml中的运行模式设置。
运行模式包括: workers, autofp and single.
更多关于运行模式的资料请参考手册中的 Runmodes .
-
-F<bpf filter file>¶ 指定BPF过滤器文件.
-
-k[all|none]¶ 强制（all）或禁用（none）校验和检查。
-
--pidfile<file>¶ 将进程ID写到指定文件，这个选项会覆盖配置文件中*pid-file*部分的设定值，并且当进程不是以后台进程运行时，强制写文件。
-
--init-errors-fatal¶ 设置若加载规则文件时出错，就失败退出。
-
--disable-detection¶ 禁用检测引擎
-
--dump-config¶ 将配置文件中加载的配置转储到终端并退出。
-
--build-info¶ 显示Suricata编译时的编译选项信息。
-
--list-app-layer-protos¶ 列出支持的所有应用层协议。
-
--list-keywords=[all|csv|<kword>]¶ 列出支持的所有规则关键字。
-
--list-runmodes¶ 列出支持的所有运行模式。
-
--set<key>=<value>¶ 设置一个配置值来覆盖基本配置参数。例如，更改默认日志目录:
--set default-log-dir=/var/tmp
-
--engine-analysis¶ 打印关于引擎中不同部分的分析报告，要知道可以打印什么报告请参考配置文件engine-analysis部分。
-
--pcap-buffer-size=<size>¶ 指定PCAP包捕获缓冲区大小 (0 - 2147483647).
-
--netmap[=<device>]¶ 在FreeBSD或Linux上启用NETMAP数据包捕获，如果未指定设备，则使用yaml配置文件中netmap部分设置的设备列表。
-
--pfring[=<device>]¶ 启用PF_RING数据包捕获，如果未指定设备，将使用Suricata配置文件中指定的设备。
-
--pfring-cluster-id<id>¶ 设置PF_RING集群ID.
-
--pfring-cluster-type<type>¶ 设置PF_RING集群类型 (cluster_round_robin, cluster_flow).
-
-d<divert-port>¶ 使用IPFW转发模式运行
-
--dag<device>¶ 启用DAG卡数据包捕获。可使用诸如“dag0：4”之类的设备名称来选择捕获特定流，这个选项可以多次使用以指定多个设备和/或流。
-
--napatech¶ 使用Napatech Streams API进行数据包捕获。
-
--mpipe¶ 使用TileGX mpipe接口进行数据包捕获.
-
--erf-in=<file>¶ 从指定的ERF(Endace可扩展记录格式)文件中读取数据以离线模式运行
-
--simulate-ips¶ 以非IPS模式运行时伪装成IPS模式
OPTIONS FOR DEVELOPERS¶
-
-u¶ 运行单元测试并退出，要求Suricata编译时添加 –enable-unittests 选项.
-
-U,--unittest-filter=REGEX¶ -U 选项指定将要运行的单元测试，这个选项可以使用正则表达式，例如: suricata -u -U http
-
--list-unittests¶ 列出所有单元测试
-
--fatal-unittests¶ 单元测试出错时启动致命失败，Suricata将会退出而不是继续进行测试。
-
--unittests-coverage¶ 显示单元测试覆盖报告。
SIGNALS¶
Suricata will respond to the following signals:
- SIGUSR2
- Causes Suricata to perform a live rule reload.
- SIGHUP
- Causes Suricata to close and re-open all log files. This can be used to re-open log files after they may have been moved away by log rotation utilities.
FILES AND DIRECTORIES¶
- /etc/suricata/suricata.yaml
- Default location of the Suricata configuration file.
- /var/log/suricata
- Default Suricata log directory.
BUGS¶
Please visit Suricata’s support page for information about submitting bugs or feature requests.
NOTES¶
Suricata Home Page
Suricata Support Page
Acknowledgements¶
Thank you to the following for their Wiki and documentation contributions that have made this user guide possible:
- Andreas Herz
- Andreas Moe
- Anne-Fleur Koolstra
- Christophe Vandeplas
- Darren Spruell
- David Cannings
- David Diallo
- David Wharton
- Eric Leblond
- god lol
- Haris Haq
- Ignacio Sanchez
- Jason Ish
- Jason Taylor
- Josh Smith
- Ken Steele
- Les Syv
- Mark Solaris
- Martin Holste
- Mats Klepsland
- Matt Jonkman
- Michael Bentley
- Michael Hrishenko
- Nathan Jimerson
- Nicolas Merle
- Peter Manev
- Philipp Buehler
- Ralph Broenink
- Rob MacGregor
- Russel Fulton
- Victor Julien
- Vincent Fang
- Zach Rasmor
Licenses¶
GNU General Public License¶
Version 2, June 1991 Copyright © 1989, 1991 Free Software Foundation, Inc. 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA
Everyone is permitted to copy and distribute verbatim copies of this license document, but changing it is not allowed.
Preamble¶
The licenses for most software are designed to take away your freedom to share and change it. By contrast, the GNU General Public License is intended to guarantee your freedom to share and change free software–to make sure the software is free for all its users. This General Public License applies to most of the Free Software Foundation’s software and to any other program whose authors commit to using it. (Some other Free Software Foundation software is covered by the GNU Lesser General Public License instead.) You can apply it to your programs, too.
When we speak of free software, we are referring to freedom, not price. Our General Public Licenses are designed to make sure that you have the freedom to distribute copies of free software (and charge for this service if you wish), that you receive source code or can get it if you want it, that you can change the software or use pieces of it in new free programs; and that you know you can do these things.
To protect your rights, we need to make restrictions that forbid anyone to deny you these rights or to ask you to surrender the rights. These restrictions translate to certain responsibilities for you if you distribute copies of the software, or if you modify it.
For example, if you distribute copies of such a program, whether gratis or for a fee, you must give the recipients all the rights that you have. You must make sure that they, too, receive or can get the source code. And you must show them these terms so they know their rights.
We protect your rights with two steps: (1) copyright the software, and (2) offer you this license which gives you legal permission to copy, distribute and/or modify the software.
Also, for each author’s protection and ours, we want to make certain that everyone understands that there is no warranty for this free software. If the software is modified by someone else and passed on, we want its recipients to know that what they have is not the original, so that any problems introduced by others will not reflect on the original authors’ reputations.
Finally, any free program is threatened constantly by software patents. We wish to avoid the danger that redistributors of a free program will individually obtain patent licenses, in effect making the program proprietary. To prevent this, we have made it clear that any patent must be licensed for everyone’s free use or not licensed at all.
The precise terms and conditions for copying, distribution and modification follow.
TERMS AND CONDITIONS FOR COPYING, DISTRIBUTION AND MODIFICATION¶
0. This License applies to any program or other work which contains a notice placed by the copyright holder saying it may be distributed under the terms of this General Public License. The “Program”, below, refers to any such program or work, and a “work based on the Program” means either the Program or any derivative work under copyright law: that is to say, a work containing the Program or a portion of it, either verbatim or with modifications and/or translated into another language. (Hereinafter, translation is included without limitation in the term “modification”.) Each licensee is addressed as “you”.
Activities other than copying, distribution and modification are not covered by this License; they are outside its scope. The act of running the Program is not restricted, and the output from the Program is covered only if its contents constitute a work based on the Program (independent of having been made by running the Program). Whether that is true depends on what the Program does.
1. You may copy and distribute verbatim copies of the Program’s source code as you receive it, in any medium, provided that you conspicuously and appropriately publish on each copy an appropriate copyright notice and disclaimer of warranty; keep intact all the notices that refer to this License and to the absence of any warranty; and give any other recipients of the Program a copy of this License along with the Program.
You may charge a fee for the physical act of transferring a copy, and you may at your option offer warranty protection in exchange for a fee.
2. You may modify your copy or copies of the Program or any portion of it, thus forming a work based on the Program, and copy and distribute such modifications or work under the terms of Section 1 above, provided that you also meet all of these conditions:
- a) You must cause the modified files to carry prominent notices stating that you changed the files and the date of any change.
- b) You must cause any work that you distribute or publish, that in whole or in part contains or is derived from the Program or any part thereof, to be licensed as a whole at no charge to all third parties under the terms of this License.
- c) If the modified program normally reads commands interactively when run, you must cause it, when started running for such interactive use in the most ordinary way, to print or display an announcement including an appropriate copyright notice and a notice that there is no warranty (or else, saying that you provide a warranty) and that users may redistribute the program under these conditions, and telling the user how to view a copy of this License. (Exception: if the Program itself is interactive but does not normally print such an announcement, your work based on the Program is not required to print an announcement.)
These requirements apply to the modified work as a whole. If identifiable sections of that work are not derived from the Program, and can be reasonably considered independent and separate works in themselves, then this License, and its terms, do not apply to those sections when you distribute them as separate works. But when you distribute the same sections as part of a whole which is a work based on the Program, the distribution of the whole must be on the terms of this License, whose permissions for other licensees extend to the entire whole, and thus to each and every part regardless of who wrote it.
Thus, it is not the intent of this section to claim rights or contest your rights to work written entirely by you; rather, the intent is to exercise the right to control the distribution of derivative or collective works based on the Program.
In addition, mere aggregation of another work not based on the Program with the Program (or with a work based on the Program) on a volume of a storage or distribution medium does not bring the other work under the scope of this License.
3. You may copy and distribute the Program (or a work based on it, under Section 2) in object code or executable form under the terms of Sections 1 and 2 above provided that you also do one of the following:
- a) Accompany it with the complete corresponding machine-readable source code, which must be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
- b) Accompany it with a written offer, valid for at least three years, to give any third party, for a charge no more than your cost of physically performing source distribution, a complete machine-readable copy of the corresponding source code, to be distributed under the terms of Sections 1 and 2 above on a medium customarily used for software interchange; or,
- c) Accompany it with the information you received as to the offer to distribute corresponding source code. (This alternative is allowed only for noncommercial distribution and only if you received the program in object code or executable form with such an offer, in accord with Subsection b above.)
The source code for a work means the preferred form of the work for making modifications to it. For an executable work, complete source code means all the source code for all modules it contains, plus any associated interface definition files, plus the scripts used to control compilation and installation of the executable. However, as a special exception, the source code distributed need not include anything that is normally distributed (in either source or binary form) with the major components (compiler, kernel, and so on) of the operating system on which the executable runs, unless that component itself accompanies the executable.
If distribution of executable or object code is made by offering access to copy from a designated place, then offering equivalent access to copy the source code from the same place counts as distribution of the source code, even though third parties are not compelled to copy the source along with the object code.
4. You may not copy, modify, sublicense, or distribute the Program except as expressly provided under this License. Any attempt otherwise to copy, modify, sublicense or distribute the Program is void, and will automatically terminate your rights under this License. However, parties who have received copies, or rights, from you under this License will not have their licenses terminated so long as such parties remain in full compliance.
5. You are not required to accept this License, since you have not signed it. However, nothing else grants you permission to modify or distribute the Program or its derivative works. These actions are prohibited by law if you do not accept this License. Therefore, by modifying or distributing the Program (or any work based on the Program), you indicate your acceptance of this License to do so, and all its terms and conditions for copying, distributing or modifying the Program or works based on it.
6. Each time you redistribute the Program (or any work based on the Program), the recipient automatically receives a license from the original licensor to copy, distribute or modify the Program subject to these terms and conditions. You may not impose any further restrictions on the recipients’ exercise of the rights granted herein. You are not responsible for enforcing compliance by third parties to this License.
7. If, as a consequence of a court judgment or allegation of patent infringement or for any other reason (not limited to patent issues), conditions are imposed on you (whether by court order, agreement or otherwise) that contradict the conditions of this License, they do not excuse you from the conditions of this License. If you cannot distribute so as to satisfy simultaneously your obligations under this License and any other pertinent obligations, then as a consequence you may not distribute the Program at all. For example, if a patent license would not permit royalty-free redistribution of the Program by all those who receive copies directly or indirectly through you, then the only way you could satisfy both it and this License would be to refrain entirely from distribution of the Program.
If any portion of this section is held invalid or unenforceable under any particular circumstance, the balance of the section is intended to apply and the section as a whole is intended to apply in other circumstances.
It is not the purpose of this section to induce you to infringe any patents or other property right claims or to contest validity of any such claims; this section has the sole purpose of protecting the integrity of the free software distribution system, which is implemented by public license practices. Many people have made generous contributions to the wide range of software distributed through that system in reliance on consistent application of that system; it is up to the author/donor to decide if he or she is willing to distribute software through any other system and a licensee cannot impose that choice.
This section is intended to make thoroughly clear what is believed to be a consequence of the rest of this License.
8. If the distribution and/or use of the Program is restricted in certain countries either by patents or by copyrighted interfaces, the original copyright holder who places the Program under this License may add an explicit geographical distribution limitation excluding those countries, so that distribution is permitted only in or among countries not thus excluded. In such case, this License incorporates the limitation as if written in the body of this License.
9. The Free Software Foundation may publish revised and/or new versions of the General Public License from time to time. Such new versions will be similar in spirit to the present version, but may differ in detail to address new problems or concerns.
Each version is given a distinguishing version number. If the Program specifies a version number of this License which applies to it and “any later version”, you have the option of following the terms and conditions either of that version or of any later version published by the Free Software Foundation. If the Program does not specify a version number of this License, you may choose any version ever published by the Free Software Foundation.
10. If you wish to incorporate parts of the Program into other free programs whose distribution conditions are different, write to the author to ask for permission. For software which is copyrighted by the Free Software Foundation, write to the Free Software Foundation; we sometimes make exceptions for this. Our decision will be guided by the two goals of preserving the free status of all derivatives of our free software and of promoting the sharing and reuse of software generally.
NO WARRANTY¶
11. BECAUSE THE PROGRAM IS LICENSED FREE OF CHARGE, THERE IS NO WARRANTY FOR THE PROGRAM, TO THE EXTENT PERMITTED BY APPLICABLE LAW. EXCEPT WHEN OTHERWISE STATED IN WRITING THE COPYRIGHT HOLDERS AND/OR OTHER PARTIES PROVIDE THE PROGRAM “AS IS” WITHOUT WARRANTY OF ANY KIND, EITHER EXPRESSED OR IMPLIED, INCLUDING, BUT NOT LIMITED TO, THE IMPLIED WARRANTIES OF MERCHANTABILITY AND FITNESS FOR A PARTICULAR PURPOSE. THE ENTIRE RISK AS TO THE QUALITY AND PERFORMANCE OF THE PROGRAM IS WITH YOU. SHOULD THE PROGRAM PROVE DEFECTIVE, YOU ASSUME THE COST OF ALL NECESSARY SERVICING, REPAIR OR CORRECTION.
12. IN NO EVENT UNLESS REQUIRED BY APPLICABLE LAW OR AGREED TO IN WRITING WILL ANY COPYRIGHT HOLDER, OR ANY OTHER PARTY WHO MAY MODIFY AND/OR REDISTRIBUTE THE PROGRAM AS PERMITTED ABOVE, BE LIABLE TO YOU FOR DAMAGES, INCLUDING ANY GENERAL, SPECIAL, INCIDENTAL OR CONSEQUENTIAL DAMAGES ARISING OUT OF THE USE OR INABILITY TO USE THE PROGRAM (INCLUDING BUT NOT LIMITED TO LOSS OF DATA OR DATA BEING RENDERED INACCURATE OR LOSSES SUSTAINED BY YOU OR THIRD PARTIES OR A FAILURE OF THE PROGRAM TO OPERATE WITH ANY OTHER PROGRAMS), EVEN IF SUCH HOLDER OR OTHER PARTY HAS BEEN ADVISED OF THE POSSIBILITY OF SUCH DAMAGES.
END OF TERMS AND CONDITIONS
How to Apply These Terms to Your New Programs¶
If you develop a new program, and you want it to be of the greatest possible use to the public, the best way to achieve this is to make it free software which everyone can redistribute and change under these terms.
To do so, attach the following notices to the program. It is safest to attach them to the start of each source file to most effectively convey the exclusion of warranty; and each file should have at least the “copyright” line and a pointer to where the full notice is found.
<one line to give the program’s name and a brief idea of what it does.> Copyright (C) <year> <name of author>
This program is free software; you can redistribute it and/or modify it under the terms of the GNU General Public License as published by the Free Software Foundation; either version 2 of the License, or (at your option) any later version.
This program is distributed in the hope that it will be useful, but WITHOUT ANY WARRANTY; without even the implied warranty of MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE. See the GNU General Public License for more details.
You should have received a copy of the GNU General Public License along with this program; if not, write to the Free Software Foundation, Inc., 51 Franklin Street, Fifth Floor, Boston, MA 02110-1301 USA.
Also add information on how to contact you by electronic and paper mail.
If the program is interactive, make it output a short notice like this when it starts in an interactive mode:
Gnomovision version 69, Copyright (C) year name of author Gnomovision comes with ABSOLUTELY NO WARRANTY; for details type `show w’. This is free software, and you are welcome to redistribute it under certain conditions; type `show c’ for details.
The hypothetical commands `show w` and `show c` should show the appropriate parts of the General Public License. Of course, the commands you use may be called something other than `show w’ and `show c`; they could even be mouse-clicks or menu items–whatever suits your program.
You should also get your employer (if you work as a programmer) or your school, if any, to sign a “copyright disclaimer” for the program, if necessary. Here is a sample; alter the names:
Yoyodyne, Inc., hereby disclaims all copyright interest in the program `Gnomovision’ (which makes passes at compilers) written by James Hacker.
<signature of Ty Coon>, 1 April 1989 Ty Coon, President of Vice
This General Public License does not permit incorporating your program into proprietary programs. If your program is a subroutine library, you may consider it more useful to permit linking proprietary applications with the library. If this is what you want to do, use the GNU Lesser General Public License instead of this License.
Creative Commons Attribution-NonCommercial 4.0 International Public License¶
By exercising the Licensed Rights (defined below), You accept and agree to be bound by the terms and conditions of this Creative Commons Attribution-NonCommercial 4.0 International Public License (“Public License”). To the extent this Public License may be interpreted as a contract, You are granted the Licensed Rights in consideration of Your acceptance of these terms and conditions, and the Licensor grants You such rights in consideration of benefits the Licensor receives from making the Licensed Material available under these terms and conditions.
Section 1 – Definitions.¶
- Adapted Material means material subject to Copyright and Similar Rights that is derived from or based upon the Licensed Material and in which the Licensed Material is translated, altered, arranged, transformed, or otherwise modified in a manner requiring permission under the Copyright and Similar Rights held by the Licensor. For purposes of this Public License, where the Licensed Material is a musical work, performance, or sound recording, Adapted Material is always produced where the Licensed Material is synched in timed relation with a moving image.
- Adapter’s License means the license You apply to Your Copyright and Similar Rights in Your contributions to Adapted Material in accordance with the terms and conditions of this Public License.
- Copyright and Similar Rights means copyright and/or similar rights closely related to copyright including, without limitation, performance, broadcast, sound recording, and Sui Generis Database Rights, without regard to how the rights are labeled or categorized. For purposes of this Public License, the rights specified in Section 2(b)(1)-(2) are not Copyright and Similar Rights.
- Effective Technological Measures means those measures that, in the absence of proper authority, may not be circumvented under laws fulfilling obligations under Article 11 of the WIPO Copyright Treaty adopted on December 20, 1996, and/or similar international agreements.
- Exceptions and Limitations means fair use, fair dealing, and/or any other exception or limitation to Copyright and Similar Rights that applies to Your use of the Licensed Material.
- Licensed Material means the artistic or literary work, database, or other material to which the Licensor applied this Public License.
- Licensed Rights means the rights granted to You subject to the terms and conditions of this Public License, which are limited to all Copyright and Similar Rights that apply to Your use of the Licensed Material and that the Licensor has authority to license.
- Licensor means the individual(s) or entity(ies) granting rights under this Public License.
- NonCommercial means not primarily intended for or directed towards commercial advantage or monetary compensation. For purposes of this Public License, the exchange of the Licensed Material for other material subject to Copyright and Similar Rights by digital file-sharing or similar means is NonCommercial provided there is no payment of monetary compensation in connection with the exchange.
- Share means to provide material to the public by any means or process that requires permission under the Licensed Rights, such as reproduction, public display, public performance, distribution, dissemination, communication, or importation, and to make material available to the public including in ways that members of the public may access the material from a place and at a time individually chosen by them.
- Sui Generis Database Rights means rights other than copyright resulting from Directive 96/9/EC of the European Parliament and of the Council of 11 March 1996 on the legal protection of databases, as amended and/or succeeded, as well as other essentially equivalent rights anywhere in the world.
- You means the individual or entity exercising the Licensed Rights under this Public License. Your has a corresponding meaning.
Section 2 – Scope.¶
- License grant.
- Subject to the terms and conditions of this Public License, the Licensor hereby grants You a worldwide, royalty-free, non-sublicensable, non-exclusive, irrevocable license to exercise the Licensed Rights in the Licensed Material to:
- reproduce and Share the Licensed Material, in whole or in part, for NonCommercial purposes only; and
- produce, reproduce, and Share Adapted Material for NonCommercial purposes only.
- Exceptions and Limitations. For the avoidance of doubt, where Exceptions and Limitations apply to Your use, this Public License does not apply, and You do not need to comply with its terms and conditions.
- Term. The term of this Public License is specified in Section 6(a).
- Media and formats; technical modifications allowed. The Licensor authorizes You to exercise the Licensed Rights in all media and formats whether now known or hereafter created, and to make technical modifications necessary to do so. The Licensor waives and/or agrees not to assert any right or authority to forbid You from making technical modifications necessary to exercise the Licensed Rights, including technical modifications necessary to circumvent Effective Technological Measures. For purposes of this Public License, simply making modifications authorized by this Section 2(a)(4) never produces Adapted Material.
- Downstream recipients.
- Offer from the Licensor – Licensed Material. Every recipient of the Licensed Material automatically receives an offer from the Licensor to exercise the Licensed Rights under the terms and conditions of this Public License.
- No downstream restrictions. You may not offer or impose any additional or different terms or conditions on, or apply any Effective Technological Measures to, the Licensed Material if doing so restricts exercise of the Licensed Rights by any recipient of the Licensed Material.
- No endorsement. Nothing in this Public License constitutes or may be construed as permission to assert or imply that You are, or that Your use of the Licensed Material is, connected with, or sponsored, endorsed, or granted official status by, the Licensor or others designated to receive attribution as provided in Section 3(a)(1)(A)(i).
- Other rights.
- Moral rights, such as the right of integrity, are not licensed under this Public License, nor are publicity, privacy, and/or other similar personality rights; however, to the extent possible, the Licensor waives and/or agrees not to assert any such rights held by the Licensor to the limited extent necessary to allow You to exercise the Licensed Rights, but not otherwise.
- Patent and trademark rights are not licensed under this Public License.
- To the extent possible, the Licensor waives any right to collect royalties from You for the exercise of the Licensed Rights, whether directly or through a collecting society under any voluntary or waivable statutory or compulsory licensing scheme. In all other cases the Licensor expressly reserves any right to collect such royalties, including when the Licensed Material is used other than for NonCommercial purposes.
Section 3 – License Conditions.¶
Your exercise of the Licensed Rights is expressly made subject to the following conditions.
- Attribution.
- If You Share the Licensed Material (including in modified form), You must:
- retain the following if it is supplied by the Licensor with the Licensed Material:
- identification of the creator(s) of the Licensed Material and any others designated to receive attribution, in any reasonable manner requested by the Licensor (including by pseudonym if designated);
- a copyright notice;
- a notice that refers to this Public License;
- a notice that refers to the disclaimer of warranties;
- a URI or hyperlink to the Licensed Material to the extent reasonably practicable;
- indicate if You modified the Licensed Material and retain an indication of any previous modifications; and
- indicate the Licensed Material is licensed under this Public License, and include the text of, or the URI or hyperlink to, this Public License.
- You may satisfy the conditions in Section 3(a)(1) in any reasonable manner based on the medium, means, and context in which You Share the Licensed Material. For example, it may be reasonable to satisfy the conditions by providing a URI or hyperlink to a resource that includes the required information.
- If requested by the Licensor, You must remove any of the information required by Section 3(a)(1)(A) to the extent reasonably practicable.
- If You Share Adapted Material You produce, the Adapter’s License You apply must not prevent recipients of the Adapted Material from complying with this Public License.
Section 4 – Sui Generis Database Rights.¶
Where the Licensed Rights include Sui Generis Database Rights that apply to Your use of the Licensed Material:
- for the avoidance of doubt, Section 2(a)(1) grants You the right to extract, reuse, reproduce, and Share all or a substantial portion of the contents of the database for NonCommercial purposes only;
- if You include all or a substantial portion of the database contents in a database in which You have Sui Generis Database Rights, then the database in which You have Sui Generis Database Rights (but not its individual contents) is Adapted Material; and
- You must comply with the conditions in Section 3(a) if You Share all or a substantial portion of the contents of the database.
For the avoidance of doubt, this Section 4 supplements and does not replace Your obligations under this Public License where the Licensed Rights include other Copyright and Similar Rights.
Section 5 – Disclaimer of Warranties and Limitation of Liability.¶
- Unless otherwise separately undertaken by the Licensor, to the extent possible, the Licensor offers the Licensed Material as-is and as-available, and makes no representations or warranties of any kind concerning the Licensed Material, whether express, implied, statutory, or other. This includes, without limitation, warranties of title, merchantability, fitness for a particular purpose, non-infringement, absence of latent or other defects, accuracy, or the presence or absence of errors, whether or not known or discoverable. Where disclaimers of warranties are not allowed in full or in part, this disclaimer may not apply to You.
- To the extent possible, in no event will the Licensor be liable to You on any legal theory (including, without limitation, negligence) or otherwise for any direct, special, indirect, incidental, consequential, punitive, exemplary, or other losses, costs, expenses, or damages arising out of this Public License or use of the Licensed Material, even if the Licensor has been advised of the possibility of such losses, costs, expenses, or damages. Where a limitation of liability is not allowed in full or in part, this limitation may not apply to You.
- The disclaimer of warranties and limitation of liability provided above shall be interpreted in a manner that, to the extent possible, most closely approximates an absolute disclaimer and waiver of all liability.
Section 6 – Term and Termination.¶
- This Public License applies for the term of the Copyright and Similar Rights licensed here. However, if You fail to comply with this Public License, then Your rights under this Public License terminate automatically.
- Where Your right to use the Licensed Material has terminated under Section 6(a), it reinstates:
- automatically as of the date the violation is cured, provided it is cured within 30 days of Your discovery of the violation; or
- upon express reinstatement by the Licensor.
For the avoidance of doubt, this Section 6(b) does not affect any right the Licensor may have to seek remedies for Your violations of this Public License.
- For the avoidance of doubt, the Licensor may also offer the Licensed Material under separate terms or conditions or stop distributing the Licensed Material at any time; however, doing so will not terminate this Public License.
- Sections 1, 5, 6, 7, and 8 survive termination of this Public License.
Section 7 – Other Terms and Conditions.¶
- The Licensor shall not be bound by any additional or different terms or conditions communicated by You unless expressly agreed.
- Any arrangements, understandings, or agreements regarding the Licensed Material not stated herein are separate from and independent of the terms and conditions of this Public License.
Section 8 – Interpretation.¶
- For the avoidance of doubt, this Public License does not, and shall not be interpreted to, reduce, limit, restrict, or impose conditions on any use of the Licensed Material that could lawfully be made without permission under this Public License.
- To the extent possible, if any provision of this Public License is deemed unenforceable, it shall be automatically reformed to the minimum extent necessary to make it enforceable. If the provision cannot be reformed, it shall be severed from this Public License without affecting the enforceability of the remaining terms and conditions.
- No term or condition of this Public License will be waived and no failure to comply consented to unless expressly agreed to by the Licensor.
- Nothing in this Public License constitutes or may be interpreted as a limitation upon, or waiver of, any privileges and immunities that apply to the Licensor or You, including from the legal processes of any jurisdiction or authority.
Creative Commons is not a party to its public licenses. Notwithstanding, Creative Commons may elect to apply one of its public licenses to material it publishes and in those instances will be considered the “Licensor.” Except for the limited purpose of indicating that material is shared under a Creative Commons public license or as otherwise permitted by the Creative Commons policies published at creativecommons.org/policies, Creative Commons does not authorize the use of the trademark “Creative Commons” or any other trademark or logo of Creative Commons without its prior written consent including, without limitation, in connection with any unauthorized modifications to any of its public licenses or any other arrangements, understandings, or agreements concerning use of licensed material. For the avoidance of doubt, this paragraph does not form part of the public licenses.
Creative Commons may be contacted at creativecommons.org.
Suricata Source Code¶
The Suricata source code is licensed under version 2 of the GNU General Public License.
Suricata Documentation¶
The Suricata documentation (this documentation) is licensed under the Creative Commons Attribution-NonCommercial 4.0 International Public License.