Table of Contents¶
Scan Point Generator¶
Scan point generator contains a number of python iterators that are used in GDA and malcolm to determine the motor demand positions and dataset indexes that various scan types will produce
Installation¶
To install the latest release, type:
pip install scanpointgenerator
To install the latest code directly from source, type:
pip install git+git://github.com/dls-controls/scanpointgenerator.git
Contributing¶
See CONTRIBUTING
Documentation¶
Full documentation is available at http://scanpointgenerator.readthedocs.org
Architecture¶
Every scan point generator inherits from the ScanPointGenerator baseclass. This baseclass provides the following API:
-
class
scanpointgenerator.Generator[source]¶ Base class for all malcolm scan point generators
Variables: - position_units (dict) – Dict of str position_name -> str position_unit for each scannable dimension. E.g. {“x”: “mm”, “y”: “mm”}
- index_dims (list) – List of the int dimension sizes for the dataset. This will have the same length as the position_units list for square scans but will be shorter for things like spiral scans. E.g. [15]
- index_names (list) – List of the str dimension names for the dataset. This will have the same length as the index_dims. E.g. [“spiral_i”]
- axes (list) – List of scannable names, used in GDA to reconstruct Point in CompoundGenerators
-
iterator()[source]¶ An iterator yielding positions at each scan point
Yields: Point – The next scan Point
Each point produced by the iterator represents a scan point, with the following API:
-
class
scanpointgenerator.Point[source]¶ Contains information about for each scan point
Variables: - positions (dict) – Dict of str position_name -> float position for each scannable dimension. E.g. {“x”: 0.1, “y”: 2.2}
- lower (dict) – Dict of str position_name -> float lower_bound for each scannable dimension. E.g. {“x”: 0.95, “y”: 2.15}
- upper (dict) – Dict of str position_name -> float upper_bound for each scannable dimension. E.g. {“x”: 1.05, “y”: 2.25}
- indexes (list) – List of int indexes for each dataset dimension, fastest changing last. E.g. [15]
- duration (int) – Int or None for duration of the point exposure
Using the API¶
You would use a generator in a step scan like this:
>>> for point in generator.iterator():
>>> for mname, mpos in point.positions():
>>> motors[mname].move(mpos)
>>> det.write_data_to_index(point.indexes)
Line Generator¶
-
class
scanpointgenerator.LineGenerator(name, units, start, stop, num, alternate_direction=False)[source]¶ Generate a line of equally spaced N-dimensional points
Parameters: - name (str/list(str) – The scannable name(s) E.g. “x” or [“x”, “y”]
- units (str) – The scannable units. E.g. “mm”
- start (float/list(float) – The first position to be generated. e.g. 1.0 or [1.0, 2.0]
- stop (float or list(float) – The first position to be generated. e.g. 5.0 or [5.0, 10.0]
- num (int) – The number of points to generate. E.g. 5
- alternate_direction (bool) – Specifier to reverse direction if generator is nested
-
classmethod
from_dict(d)[source]¶ Create a LineGenerator instance from a serialised dictionary
Parameters: d (dict) – Dictionary of attributes Returns: New LineGenerator instance Return type: LineGenerator
Examples¶
This example defines a motor “x” with engineering units “mm” which is being scanned from 0mm to 1mm with 5 scan points inclusive of the start. Note that the capture points are as given, so the bounds will be +-0.5*step of each capture point.
from scanpointgenerator import LineGenerator, plot_generator
gen = LineGenerator("x", "mm", 0.0, 1.0, 5)
plot_generator(gen)
LineGenerator is N dimensional; just pass in ND lists for name, start and stop.
from scanpointgenerator import LineGenerator, plot_generator
gen = LineGenerator(["x", "y"], "mm", [1.0, 2.0], [5.0, 10.0], 5)
plot_generator(gen)
Spiral Generator¶
-
class
scanpointgenerator.SpiralGenerator(names, units, centre, radius, scale=1.0, alternate_direction=False)[source]¶ Generate the points of an Archimedean spiral
Parameters: - names (list(str) – The scannable names e.g. [“x”, “y”]
- units (str) – The scannable units e.g. “mm”
- centre (list) – List of two coordinates of centre point of spiral
- radius (float) – Radius of spiral
- scale (float) – Rate at which spiral expands; higher scale gives fewer points for same radius
- alternate_direction (bool) – Specifier to reverse direction if generator is nested
-
classmethod
from_dict(d)[source]¶ Create a SpiralGenerator instance from a serialised dictionary
Parameters: d (dict) – Dictionary of attributes Returns: New SpiralGenerator instance Return type: SpiralGenerator
Examples¶
This example defines motors “x” and “y” with engineering units “mm” which will be scanned in a spiral filling a circle of radius 5mm.
from scanpointgenerator import SpiralGenerator, plot_generator
gen = SpiralGenerator(["x", "y"], "mm", [0.0, 0.0], 5.0)
plot_generator(gen)
In this example the spiral is scaled to be more sparse.
from scanpointgenerator import SpiralGenerator, plot_generator
gen = SpiralGenerator(["x", "y"], "mm", [0.0, 0.0], 5.0, scale=2.0)
plot_generator(gen)
Lissajous Generator¶
-
class
scanpointgenerator.LissajousGenerator(names, units, box, num_lobes, num_points=None)[source]¶ Generate the points of a Lissajous curve
Parameters: - names (list(str) – The scannable names e.g. [“x”, “y”]
- units (str) – The scannable units e.g. “mm”
- box (dict) – Dictionary of centre, width and height representing box to fill with points
- num_lobes (int) – Number of x-direction lobes for curve; will have num_lobes+1 y-direction lobes
- num_points (int) – The number of points to fill the Lissajous curve. Default is 250 * num_lobes
-
classmethod
from_dict(d)[source]¶ Create a LissajousGenerator instance from a serialised dictionary
Parameters: d (dict) – Dictionary of attributes Returns: New LissajousGenerator instance Return type: LissajousGenerator
Examples¶
This example defines motors “x” and “y” with engineering units “mm” which will be scanned over a 3x4 lobe Lissajous curve with filling a 1x1mm rectangle.
from scanpointgenerator import LissajousGenerator, plot_generator
box = dict(centre=[0.0, 0.0], width=1.0, height=1.0)
gen = LissajousGenerator(['x', 'y'], "mm", box=box, num_lobes=3, num_points=50)
plot_generator(gen)
The number of points has been lowered from the default to make the plot more visible. The following plot is for 10x11 lobes with the default number of points.
from scanpointgenerator import LissajousGenerator, plot_generator
box = dict(centre=[0.0, 0.0], width=1.0, height=1.0)
gen = LissajousGenerator(['x', 'y'], "mm", box=box, num_lobes=20)
plot_generator(gen, show_indexes=False)
Array Generator¶
-
class
scanpointgenerator.ArrayGenerator(name, units, points, lower_bounds=None, upper_bounds=None)[source]¶ Generate a given n-dimensional array of points
Parameters: - name (str/list(str) – ND list of scannable names e.g. “x” or [“x”, “y”]
- units (str) – The scannable units. E.g. “mm”
- points (list) – List of ND lists of coordinates e.g. [1.0, 2.0, 3.0] or [[1.0, 2.0], [3.0, 4.0], [5.0, 6.0]]
- lower_bounds (list) – List of ND lists of lower bound coordinates
- upper_bounds (list) – List of ND lists of upper bound coordinates
-
classmethod
from_dict(d)[source]¶ Create a ArrayGenerator instance from a serialised dictionary
Parameters: d (dict) – Dictionary of attributes Returns: New ArrayGenerator instance Return type: ArrayGenerator
Examples¶
The ArrayGenerator takes an N-Dimensional array of coordinates and creates Points with calculated upper and lower bounds. You can also provide your own bounds.
from scanpointgenerator import ArrayGenerator, plot_generator
points = [0.0, 2.0, 3.0, 5.0, 7.0, 8.0]
array = ArrayGenerator("x", "mm", points)
plot_generator(array)
And a 2D scan.
from scanpointgenerator import ArrayGenerator, plot_generator
points = [[0.0, 2.0], [2.0, 3.0], [3.0, 5.0], [5.0, 6.0], [7.0, 7.0], [8.0, 9.0]]
array = ArrayGenerator(["x", "y"], "mm", points)
plot_generator(array)
Compound Generator¶
-
class
scanpointgenerator.CompoundGenerator(generators, excluders, mutators)[source]¶ Nest N generators, apply exclusion regions to relevant generator pairs and apply any mutators before yielding points
Parameters: - generators (list(Generator) – List of Generators to nest
- excluders (list(Excluder) – List of Excluders to filter points by
- mutators (list(Mutator) – List of Mutators to apply to each point
-
iterator()[source]¶ Top level iterator to mutate points and yield them
Yields: Point – Mutated points
-
contains_point(point)[source]¶ Filter a Point through all Excluders
Parameters: point (Point) – Point to check Returns: Whether point is contained by all Excluders Return type: bool
-
classmethod
from_dict(d)[source]¶ Create a CompoundGenerator instance from a serialised dictionary
Parameters: d (dict) – Dictionary of attributes Returns: New CompoundGenerator instance Return type: CompoundGenerator
Raster Scan Example¶
This scan will create an outer “y” line scan with 4 points, then nest an “x” line scan inside it with 5 points.
from scanpointgenerator import LineGenerator, CompoundGenerator, plot_generator
xs = LineGenerator("x", "mm", 0.0, 0.5, 5, alternate_direction=False)
ys = LineGenerator("y", "mm", 0.0, 0.5, 4)
gen = CompoundGenerator([ys, xs], [], [])
plot_generator(gen)
Snake Scan Example¶
This scan will create an outer “y” line scan with 4 points, then nest an “x” line scan inside it with 5 points. On every second row, the “x” line scan will be run in reverse to give a snake scan.
from scanpointgenerator import LineGenerator, CompoundGenerator, plot_generator
xs = LineGenerator("x", "mm", 0.0, 0.5, 5, alternate_direction=True)
ys = LineGenerator("y", "mm", 0.0, 0.5, 4)
gen = CompoundGenerator([ys, xs], [], [])
plot_generator(gen)
Excluders¶
Excluders are used to filter points in a generator based on a pair of coordinates and a region of interest.
-
class
scanpointgenerator.Excluder(roi, scannables)[source]¶ A class to remove points that lie outside of a given 2D region of interest
Parameters: - roi (ROI) – Region of interest to filter points by
- scannables (list) – List of two scannables to filter points by
CircularROI Example¶
Here we use a CircularROI to filter the points of a snake scan
from scanpointgenerator import LineGenerator, CompoundGenerator, \
Excluder, plot_generator
from scanpointgenerator.circular_roi import CircularROI
x = LineGenerator("x", "mm", 0.0, 4.0, 5, alternate_direction=True)
y = LineGenerator("y", "mm", 0.0, 3.0, 4)
circle = Excluder(CircularROI([2.0, 1.0], 2.0), ["x", "y"])
gen = CompoundGenerator([y, x], [], [])
plot_generator(gen, circle)
And with the excluder applied
from scanpointgenerator import LineGenerator, CompoundGenerator, \
Excluder, plot_generator
from scanpointgenerator.circular_roi import CircularROI
x = LineGenerator("x", "mm", 0.0, 4.0, 5, alternate_direction=True)
y = LineGenerator("y", "mm", 0.0, 3.0, 4)
circle = Excluder(CircularROI([2.0, 1.0], 2.0), ["x", "y"])
excluder = Excluder(circle, ['x', 'y'])
gen = CompoundGenerator([y, x], [circle], [])
plot_generator(gen, circle)
Mutators¶
Mutators are used for post processing points after they have been generated and filtered by any regions of interest.
-
class
scanpointgenerator.Mutator[source]¶ Abstract class to apply a mutation to the points of an ND ScanPointGenerator
-
mutate(iterator)[source]¶ Abstract method to take each point from the given iterator, apply a mutation and then yield the new point
Parameters: iterator (iter) – Iterator to mutate Yields: Point – Mutated points from generator
-
RandomOffsetMutator¶
This is used to apply a random offset to each point in an iterator. Here we apply it to a snake scan
from scanpointgenerator import LineGenerator, CompoundGenerator, plot_generator
xs = LineGenerator("x", "mm", 0.0, 0.5, 5, alternate_direction=True)
ys = LineGenerator("y", "mm", 0.0, 0.5, 4)
gen = CompoundGenerator([ys, xs], [], [])
plot_generator(gen)
And with the random offset
from scanpointgenerator import LineGenerator, CompoundGenerator, RandomOffsetMutator, plot_generator
xs = LineGenerator("x", "mm", 0.0, 0.5, 5, alternate_direction=True)
ys = LineGenerator("y", "mm", 0.0, 0.5, 4)
random_offset = RandomOffsetMutator(seed=1, axes = ["x", "y"], max_offset=dict(x=0.05, y=0.05))
gen = CompoundGenerator([ys, xs], [], [random_offset])
plot_generator(gen)
Creating a Generator¶
The idea of CompoundGenerator is that you can combine generators, excluders and mutators arbitrarily. The following will show some more extensive examples to show the capabilities of scanpointgenerator.
A spiral scan with an offset rectangular roi overlay and randomly offset points in the y direction
from scanpointgenerator import LineGenerator, SpiralGenerator, \
CompoundGenerator, Excluder, RandomOffsetMutator, plot_generator
from scanpointgenerator.rectangular_roi import RectangularROI
spiral = SpiralGenerator(["x", "y"], "mm", [0.0, 0.0], 10.0,
alternate_direction=True)
rectangle = Excluder(RectangularROI([1.0, 1.0], 8.0, 8.0), ["x", "y"])
mutator = RandomOffsetMutator(2, ["x", "y"], dict(x=0.0, y=0.25))
gen = CompoundGenerator([spiral], [rectangle], [mutator])
plot_generator(gen, rectangle)
A spiral scan at each point of a line scan with alternating direction
from scanpointgenerator import LineGenerator, SpiralGenerator, \
CompoundGenerator
line = LineGenerator("z", "mm", 0.0, 20.0, 3)
spiral = SpiralGenerator(["x", "y"], "mm", [0.0, 0.0], 1.2,
alternate_direction=True)
gen = CompoundGenerator([line, spiral], [], [])
for point in gen.iterator():
for axis, value in point.positions.items():
point.positions[axis] = round(value, 3)
print(point.positions)
{'y': -0.321, 'x': 0.237, 'z': 0.0}
{'y': -0.25, 'x': -0.644, 'z': 0.0}
{'y': 0.695, 'x': -0.56, 'z': 0.0}
{'y': 0.992, 'x': 0.361, 'z': 0.0}
{'y': 0.992, 'x': 0.361, 'z': 10.0}
{'y': 0.695, 'x': -0.56, 'z': 10.0}
{'y': -0.25, 'x': -0.644, 'z': 10.0}
{'y': -0.321, 'x': 0.237, 'z': 10.0}
{'y': -0.321, 'x': 0.237, 'z': 20.0}
{'y': -0.25, 'x': -0.644, 'z': 20.0}
{'y': 0.695, 'x': -0.56, 'z': 20.0}
{'y': 0.992, 'x': 0.361, 'z': 20.0}
Three nested line scans with an excluder operating on the innermost and outermost axes
from scanpointgenerator import LineGenerator, CompoundGenerator, \
Excluder
from scanpointgenerator.circular_roi import CircularROI
line1 = LineGenerator("x", "mm", 0.0, 2.0, 3)
line2 = LineGenerator("y", "mm", 0.0, 1.0, 2)
line3 = LineGenerator("z", "mm", 0.0, 1.0, 2)
circle = Excluder(CircularROI([1.0, 1.0], 1.0), ["x", "z"])
gen = CompoundGenerator([line3, line2, line1], [circle], [])
for point in gen.iterator():
print(point.positions)
{'y': 0.0, 'x': 1.0, 'z': 0.0}
{'y': 0.0, 'x': 1.0, 'z': 1.0}
{'y': 1.0, 'x': 0.0, 'z': 0.0}
{'y': 1.0, 'x': 1.0, 'z': 0.0}
{'y': 1.0, 'x': 2.0, 'z': 0.0}
{'y': 1.0, 'x': 0.0, 'z': 1.0}
{'y': 1.0, 'x': 1.0, 'z': 1.0}
{'y': 1.0, 'x': 2.0, 'z': 1.0}
Serialisation¶
These generators are designed to be serialised and sent over json. The model for the CompoundGenerator is as follows:
{
typeid: "scanpointgenerator:generator/CompoundGenerator:1.0",
generators: [
{
typeid: "scanpointgenerator:generator/LineGenerator:1.0"
name: "y"
units: "mm"
start: 0.0
stop: 1.0
num: 5
alternate_direction = False
},
{
typeid: "scanpointgenerator:generator/LineGenerator:1.0"
name: "x"
units: "mm"
start: 0.0
stop: 5.0
num: 5
alternate_direction = True
}
],
excluders: [
{
roi: {
typeid: "scanpointgenerator:roi/CircularROI:1.0"
centre: [0.0, 0.0]
radius: 0.5
}
scannables: ["x", "y"]
}
],
mutators: [
{
typeid: "scanpointgenerator:mutator/RandomOffsetMutator:1.0"
seed: 10
axes: ["x", "y"]
max_offset: {
x: 0.1
y: 0.2
}
}
]
}
The models for each base generator are:
ArrayGenerator (where name and points can be N-dimensional and upper_bounds and lower_bounds are optional):
{
typeid: "scanpointgenerator:generator/ArrayGenerator:1.0"
name: "x" or ["x", "y"]
units: "mm"
points: [1.0, 2.0, 3.0] or [[1.0, 2.0], [2.0, 4.0], [3.0, 6.0]]
upper_bounds: [1.5, 2.5, 3.5]
lower_bounds: [0.5, 1.5, 2.5]
}
LineGenerator (name, start and stop can be N-dimensional to create and ND scan):
{
typeid: "scanpointgenerator:generator/LineGenerator:1.0"
name: "x" or ["x", "y"]
units: "mm"
start: 0.0 or [0.0, 0.0]
num: 5
alternate_direction = True
}
LissajousGenerator (where num_points is optional):
{
typeid: "scanpointgenerator:generator/LissajousGenerator:1.0"
names: ["x", "y"]
units: "mm"
box: {
centre: [0.0, 0.0]
width: 10.0
height: 10.0
}
num_lobes: 20
num_points: 1000
}
SpiralGenerator (where scale is optional):
{
typeid: "scanpointgenerator:generator/SpiralGenerator:1.0"
names: ["x", "y"]
units: "mm"
centre: [0.0, 0.0]
radius: 5.0
scale: 2.0
alternate_direction = True
}
And for the mutators:
RandomOffsetMutator:
{
typeid: "scanpointgenerator:mutator/RandomOffsetMutator:1.0"
seed: 10
axes: ["x", "y"]
max_offset: {
x: 0.1
y: 0.2
}
}
And the excluders:
To be added...
As an example of serialising, here is a simple snake scan.
from scanpointgenerator import LineGenerator, CompoundGenerator, \
plot_generator
x = LineGenerator("x", "mm", 0.0, 4.0, 5, alternate_direction=True)
y = LineGenerator("y", "mm", 0.0, 3.0, 4)
gen = CompoundGenerator([y, x], [], [])
plot_generator(gen)
It is the same after being serialised and deserialised.
from scanpointgenerator import LineGenerator, CompoundGenerator, \
plot_generator
x = LineGenerator("x", "mm", 0.0, 4.0, 5, alternate_direction=True)
y = LineGenerator("y", "mm", 0.0, 3.0, 4)
gen = CompoundGenerator([y, x], [], [])
gen_dict = gen.to_dict()
new_gen = CompoundGenerator.from_dict(gen_dict)
plot_generator(new_gen)
Writing new scan point generators¶
Let’s walk through the simplest generator, LineGenerator, and see how
it is written.
We import the baseclass Generator and the Point class
that we will be generating instances of.
Our new subclass includes a docstring giving a short explanation of what it does
The initialiser stores the arguments given to it, then generates the three properties that are required by the baseclass:
- position_units: Dict of str position_name -> str position_unit
- index_dims: List of int dimension sizes for the dataset
- index_names: List of str dimension names for the dataset
It is important to note that the position_units property will have the same number of elements as index_dims for grid based scans (like LineGenerator). For non grid based scans (like SpiralGenerator), index_dims will typically have less elements, because the last two or more dimensions will be unrolled into one long array. This avoids sparse datasets.
We have a repeated bit of code here, so have pulled it out into a function. It calculates the position of a point given an index
This is the entry point for external code. It is expecting us to produce a
number of Point instances, one for each point in the scan. We are
required to fill in the following dictionaries of str position_name -> float
position:
- positions: The capture position corresponding to the centre of the scan frame
- lower: The lower bound of the scan frame if the scan is to be used for continuous scanning
- upper: The upper bound of the scan frame if the scan is to be used for continuous scanning
We also fill in the list of datapoint indexes:
- indexes: The index into the dataset that the data frame should be stored in
The yield keyword turns the python function into a generator, which can then be used by the external program to iterate through points without evaluating them all at the start.
Contributing¶
Contributions and issues are most welcome! All issues and pull requests are handled through github on the dls_controls repository. Also, please check for any existing issues before filing a new one. If you have a great idea but it involves big changes, please file a ticket before making a pull request! We want to make sure you don’t spend your time coding something that might not fit the scope of the project.
Running the tests¶
To get the source source code and run the unit tests, run:
$ git clone git://github.com/dls_controls/scanpointgenerator.git
$ cd scanpointgenerator
$ virtualenv env
$ . env/bin/activate
$ pip install nose
$ python setup.py install
$ python setup.py nosetests
While 100% code coverage does not make a library bug-free, it significantly reduces the number of easily caught bugs! Please make sure coverage is at 100% before submitting a pull request!
Code Quality¶
Landscape.io will test code quality when you create a pull request. Please follow PEP8.
Code Styling¶
Please arrange imports with the following style
# Standard library imports
import os
# Third party package imports
from mock import patch
# Local package imports
from scanpointgenerator.version import __version__
Please follow Google’s python style guide wherever possible.
Building the docs¶
When in the project directory:
$ pip install -r requirements/docs.txt
$ python setup.py build_sphinx
$ open docs/html/index.html
Release Checklist¶
Before a new release, please go through the following checklist:
Bump version in scanpointgenerator/version.py
Add a release note and diff URL in CHANGELOG.rst
Git tag the version
Upload to pypi:
make publish
Change Log¶
All notable changes to this project will be documented in this file. This project adheres to Semantic Versioning.
1-6 - 2016-10-18¶
Fixed:
- CompoundGenerator to set the right number of points if excluders are used
Changed:
- Refactored internal structure of modules
1-3 - 2016-08-31¶
Added:
- Remove OrderedDict entirely for 2.5 back-compatibility
Changed:
- type is now typeid to make it compatible with malcolm