nms documentation¶

Summary & Notes¶

Given a list of rectangles (or rotated rectangles or polygons) and a corresponding list of scores (confidences), the Non Maximal Suppression functions below will return a list of indicies. These indicies correspond to the rectangles (or rotated rectangles or polygons) that have the highest scores and the minimum overlap with other elements in the list.

Rectangles (sometimes referred to as rects and boxes) are expressed as a 4-tuple: (x, y, w, h). Where x, y is the coordinate of the upper left hand corner of the rectangle. w and h are the width and height of the rectangle.

Rotated rectangles (sometimes referred to rrects or rboxes) are expressed as a 5-tuple: (x, y, w, h, r). Where the first four members are as described for rectangles and r is the angle, in radians, of rotation around the center of the rectangle.

Polygons are expressed as a list of 2-tuples of (x, y). Each x, y describes a verticie of the polygon. The edges of the polygon are described by each pair of consecutive vertices. That is (x0, y0), (x1, y1) describe and edge of the polygon. The polygons are closed by the edge (xn, yn), (x0, y0).

If you only have a need for NMS on rectangles and are already using OpenCV, you may want to consider using cv2.dnn.NMSBoxes. However, you might see a small performance gain using nms over NMSBoxes – nms defaults to using the Malisiiewicz et. al. algorithm vs. the Fast algorithm in NMSBoxes.

PSA: It’s not well documented that the bboxes parameter for NMSBoxes is a numpy array of ((x, y), (w, h)).

nms was developed with Python 3.7 and OpenCV 4.0.0-pre. It’s very likely that nms will work just fine with early versions of OpenCV.

This code is beta and lacking in test coverage (read: no coverage)

Install¶

nms has dependencies on cv2 (OpenCV 4.0.0-pre), numpy and math. Adrian Rosenbrock has excellent instructions for installing OpenCV 4.0 on MacOS and Ubuntu

Include nms in your requirements.txt or install into your environment:

pip install nms

Basic Usage¶

Given a list of rotated rectangles rrects and confidences scores:

indicies = nms.nms.nms_rboxes(rrects, scores)

By default, nms.nms.nms_boxes(), nms.nms.nms_polygons() and nms.nms.nms_rboxes() use the Malisiiewicz et. al. NMS Algorithm. If you would prefer to use the OpenCV2 Fast algorithm or Felzenszwalb et. al. algorithm you can do so with the optional parameter nms_algorithm. For example:

indicies = nms.nms.nms_rboxes(rrects, scores, nms_algorithm=malisiewicz.nms)

nms.nms.nms_boxes(), nms.nms.nms_polygons() and nms.nms.nms_rboxes() will also accept a list of key word arguments (**kwargs). The kwargs are passed to the NMS function to set values for the NMS threshold, confidence threshold, top_k and eta (eta is only applicable when using Fast).

For example, to set the NMS threshold to 0.5 (note that use of kwargs requires that nms_algorithm parameter be present):

indicies = nms.nms.nms_rboxes(rrects, scores, nms_algorithm=malisiewicz.nms, nms_threshold=0.5)

Please see the docs for nms.fast.nms(), nms.malisiewicz.nms() and nms.felzenszwalb.nms() for additional information on kwargs

Example¶

For a simple example of using nms with geometry returned from EAST, take a look at opencv-text-detection.

Most Useful Functions¶

These are the NMS functions you are looking for.

nms.nms.boxes(rects, scores, nms_algorithm=<function nms>, **kwargs)[source]

Non Maxima Suppression for rectangles.

This function is provided for completeness as it replicates the functionality of cv2.dnn.NMSBoxes. This may be slightly faster as NMSBoxes uses the FAST comparison algorithm and by default this used Malisiewicz et al.

Parameters:	rects (list) – a list of rectangles, each described by (x, y, w, h) (same as cv2.NMSBoxes) scores (list) – a list of the scores associated with rects nms_algorithm (function) – the NMS comparison function to use, kwargs will be passed to this function. Defaults to `nms.malisiewicz.nms()`
Returns:	a list of indicies of the best rects

nms.nms.polygons(polys, scores, nms_algorithm=<function nms>, **kwargs)[source]

Non Maxima Suppression for polygons

Parameters:	polys (list) – a list of polygons, each described by their xy verticies scores (list) – a list of the scores associated with the polygons nms_algorithm (function) – the NMS comparison function to use, kwargs will be passed to this function. Defaults to `nms.malisiewicz.nms()`
Returns:	an array of indicies of the best polys

nms.nms.rboxes(rrects, scores, nms_algorithm=<function nms>, **kwargs)[source]

Non Maxima Suppression for rotated rectangles

Parameters:	rrects (list) – a list of polygons, each described by ((cx, cy), (w,h), deg) scores (list) – a list of the scores associated with the rects nms_algorithm (function) – the NMS comparison function to use, kwargs will be passed to this function. Defaults to `nms.malisiewicz.NMS()`
Returns:	an array of indicies of the best rrects

nms¶

nms package¶

Submodules¶

nms.fast module¶

NMS implementation based on OpenCV NMSFast

The functions in this module are not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxes(), or nms.nms.polygons() and set nms_algorithm=nms.fast

nms.fast.nms(boxes, scores, **kwargs)[source]¶

Do Non Maximal Suppression

As translated from the OpenCV c++ source in nms.inl.hpp which was in turn inspired by Piotr Dollar’s NMS implementation in EdgeBox.

This function is not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxes(), or nms.nms.polygons()

Parameters:	boxes (list) – the boxes to compare, the structure of the boxes must be compatible with the compare_function. scores (list) – the scores associated with boxes kwargs (dict (see below)) – optional keyword parameters
Returns:	an list of indicies of the best boxes
Return type:	list
Kwargs:

score_threshold (float): the minimum score necessary to be a viable solution, default 0.3
nms_threshold (float): the minimum nms value to be a viable solution, default: 0.4
compare_function (function): function that accepts two boxes and returns their overlap ratio, this function must accept two boxes and return an overlap ratio
eta (float): a coefficient in adaptive threshold formula: nms_threshold_(i+1)=eta*nms_threshold_i, default: 1.0
top_k (int): if >0, keep at most top_k picked indices. default:0

nms.fast.polygon_iou(poly1, poly2, useCV2=True)[source]¶

Computes the ratio of the intersection area of the input polygons to the (sum of polygon areas - intersection area) Used with the NMS function

Parameters:	poly1 (list) – a polygon described by its verticies poly2 (list) – a polygon describe by it verticies useCV2 (bool) – if True (default), use cv2.contourArea to calculate polygon areas. If false use `nms.helpers.polygon_intersection_area()`.
Returns:	The ratio of the intersection area / (sum of rectangle areas - intersection area)
Return type:	float

nms.fast.rectangle_iou(rectA, rectB)[source]¶

Computes the ratio of the intersection area of the input rectangles to the (sum of rectangle areas - intersection area). Used with the NMS function.

Parameters:	rectA (tuple) – a rectangle described by (x,y,w,h) rectB (tuple) – a rectangle describe by (x,y,w,h)
Returns:	The ratio of overlap between rectA and rectB (intersection area/(rectA area + rectB area - intersection area)

nms.fast.rotated_rect_iou(rectA, rectB)[source]¶

Computes the ratio of the intersection area of the input rectangles to the (sum of rectangle areas - intersection area) Used with the NMS function

Parameters:	rectA (list) – a polygon (rectangle) described by its verticies rectB (list) – a polygon (rectangle) describe by it verticies
Returns:	The ratio of the intersection area / (sum of rectangle areas - intersection area)

nms.felzenszwalb module¶

Felzenszwalb et al implementation of NMS

The functions in this module are not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxess(), or nms.nms.polygons()

nms.felzenszwalb.nms(boxes, scores, **kwargs)[source]¶

NMS using Felzenszwalb et al. method

Adapted from non_max_suppression_slow(boxes, overlapThresh) from Non-Maximum Suppression for Object Detection in Python

This function is not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxes(),: or nms.nms.polygons() and set nms_algorithm=nms.felzenszwalb

Parameters:	boxes (list) – a list of boxes to perform NMS on scores (list) – a list of scores corresponding to boxes kwargs (dict (see below)) – optional keyword parameters (see below)
Returns:	a list of the indicies of the best boxes
Return type:	list
Kwargs:

top_k (int): if >0, keep at most top_k picked indices. default:0, int
score_threshold (float): the minimum score necessary to be a viable solution, default 0.3, float
nms_threshold (float): the minimum nms value to be a viable solution, default: 0.4, float
compare_function (function): function that accepts two boxes and returns their overlap ratio, this function must accept two boxes and return an overlap ratio between 0 and 1
area_function (function): function used to calculate the area of an element of boxes

nms.felzenszwalb.poly_areas(polys)[source]¶

Calculate the area of each polygon in polys

Parameters:	polys (list) – a list of polygons, each specified by its verticies
Returns:	a list of areas corresponding the list of polygons
Return type:	list

nms.felzenszwalb.poly_compare(poly1, poly2, area)[source]¶

Calculate the ratio of overlap between two polygons and the given area

Parameters:	poly1 (list) – polygon specified by its verticies poly2 (list) – polygon specified by its verticies area (float) – the area to compare the overlap of poly1 and poly2
Returns:	the ratio of overlap of poly1 and poly2 to the area e.g. overlap(poly1, poly2)/area
Return type:	float

nms.felzenszwalb.rect_areas(rects)[source]¶

Return an np.array of the areas of the rectangles

Parameters:	rects (list) – a list of rectangles, each specified as (x, y, w, h)
Returns:	an numpy array of corresponding areas
Return type:	`numpy.ndarray`

nms.felzenszwalb.rect_compare(rect1, rect2, area)[source]¶

Calculate the ratio of overlap between two rectangles and the given area

Parameters:	rect1 (tuple) – rectangle specified as (x, y, w, h) rect2 (tuple) – rectangle specificed as (x, y, w, h) area (float) – the area to compare to
Returns:	the ratio of the overlap of rect1 and rect2 to the area, e.g overlap(rect1, rect2)/area
Return type:	float

nms.helpers module¶

Helper functions and probably not much use outside the scope of nms

nms.helpers.createImage(width=800, height=800, depth=3)[source]¶

Return a black image with an optional scale on the edge

Parameters:	width (int) – width of the returned image height (int) – height of the returned image depth (int) – either 3 (rgb/bgr) or 1 (mono). If 1, no scale is drawn
Returns:	A zero’d out matrix/black image of size (width, height)
Return type:	`numpy.ndarray`

nms.helpers.get_max_score_index(scores, threshold=0, top_k=0, descending=True)[source]¶

Get the max scores with corresponding indicies

Adapted from the OpenCV c++ source in nms.inl.hpp

Parameters:	scores (list) – a list of scores threshold (float) – consider scores higher than this threshold top_k (int) – return at most top_k scores; if 0, keep all descending – if True, list is returened in descending order, else ascending
Returns:	a sorted by score list of [score, index]

nms.helpers.polygon_intersection_area(polygons)[source]¶

Compute the area of intersection of an array of polygons

Parameters:	polygons (list) – a list of polygons
Returns:	the area of intersection of the polygons
Return type:	int

nms.malisiewicz module¶

Malisiewicz et al implementation of NMS

The functions in this module are not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxes(), or nms.nms.polygons()

nms.malisiewicz.nms(boxes, scores, **kwargs)[source]¶

NMS using Malisiewicz et al. method

Adapted from non_max_suppression_fast(boxes, overlapThresh) from (Faster) Non-Maximum Suppression in Python

Note that when using this on rotated rectangles or polygons (as opposed to upright rectangles), some of the vectorized performance gains are lost as comparisons between rrects and polys cannot(?) be vectorized

This function is not usually called directly. Instead use nms.nms.boxes(), nms.nms.rboxes(), or nms.nms.polygons() and set parameter nms_algorithm=nms.malisiewicz

Parameters:	boxes (list) – a list of boxes to perform NMS on scores (list) – a list of scores corresponding to boxes kwargs (dict) – optional keyword parameters (see below)
Returns:	a list of the indicies of the best boxes
Kwargs:

top_k (int): if >0, keep at most top_k picked indices. default:0
score_threshold (float): the minimum score necessary to be a viable solution, default 0.3
nms_threshold (float): the minimum nms value to be a viable solution, default: 0.4
compare_function (function): function that accepts two boxes and returns their overlap ratio, this function must accept two boxes and return an overlap ratio between 0 and 1
area_function (function): function used to calculate the area of an element of boxes

nms.malisiewicz.poly_areas(polys)[source]¶

Calculate the area of the list of polygons

Parameters:	polys (list) – a list of polygons, each specified by a list of its verticies
Returns:	numpy array of areas of the polygons
Return type:	`numpy.ndarray`

nms.malisiewicz.poly_compare(poly1, polygons, area)[source]¶

Calculate the intersection of poly1 to polygons divided by area

Parameters:	poly1 (list) – a polygon specified by a list of its verticies polygons (list) – a list of polygons, each specified a list of its verticies area (list) – a list of areas of the corresponding polygons
Returns:	a numpy array of the ratio of overlap of poly1 to each of polygons to the corresponding area. e.g. overlap(poly1, polygons[n])/area[n]
Return type:	`numpy.ndarray`

nms.malisiewicz.rect_areas(rects)[source]¶