Empirical SpecMatch¶

Empirical SpecMatch, SpecMatch-Emp for short, is an algorithm for characterizing the properties of stars based on their optical spectra. Target spectra are compared against a dense spectral library of well-characterized touchstone stars. A key component of SpecMatch-Emp is the library of high resolution (R~55,000), high signal-to-noise (> 100) spectra taken with Keck/HIRES by the California Planet Search. Spectra in the library may be accessed by a convenient Python API. For non-Python users spectra are also available monolithic memory-efficient HDF5 archive and a la carte as individual FITS files.

Code availability¶

The SpecMatch-Emp codebase is availble on GitHub. Simply clone the repository and run python setup.py install. For more details, go to the Installation page.

Basic Usage¶

The most basic method of using SpecMatch-Emp is to run the command line script:

smemp specmatch rjXXX.XXXX.fits

This performs the shifting and matching process, prints the results and saves it into a .h5 file which can be read by SpecMatch.read_hdf() in python.

For more details on using the command line interface, visit Command Line Interface. For a more general usage primer, check out the Quickstart page.

Attribution¶

If you make use of the specmatch-emp spectral library or matching code, please cite Yee et al. (2017) [ADS Record]

Contents:

Installation¶

Download code from

https://github.com/samuelyeewl/specmatch-emp

or clone git repo

$ git clone https://github.com/samuelyeewl/specmatch-emp

Then, simply run

$ python setup.py install

This downloads all dependencies and data files, and creates the command line scripts.

The python package setuptools is required to install SpecMatch-Emp. If not already installed on your system, run

$ pip install setuptools

Minimal Install¶

As of v0.3, it is possible to install SpecMatch-Emp without downloading the full library. This will allow only the shifting code to be used.

$ python setup.py install --shift-only

Software dependencies¶

To install

setuptools
wget (facilitates downloading files).

Python Packages

To use SpecMatch-Emp:

numpy
scipy
matplotlib
h5py
astropy (>=1.3)
pandas (>=0.18)
lmfit (>=0.9)

To build library: - astroquery - isochrones

Data Files¶

These data files are automatically downloaded upon installation.

library.h5: the HDF5 archive of spectra.
hires_telluric_mask.csv: mask of telluric lines in HIRES spectra
detrend.csv: parameter calibration info
nso.fits, j26.532_adj.fits, j72.718_adj.fits, j59.1926_adj.fits spectra used as wavelength standards

Quickstart¶

Here’s how to get up and running with specmatch-emp

Library¶

specmatch-emp comes with a large library high-resolution optical spectra shifted onto the rest wavelength scale. We’ll import the library, along with some other useful modules.

import pandas as pd
from pylab import *
import specmatchemp.library
import specmatchemp.plots as smplot

Now we’ll load in the library around the Mgb triplet. By default, SpecMatch create the following directory ${HOME}/.specmatchemp/ and download the library into it.

lib = specmatchemp.library.read_hdf(wavlim=[5140,5200])

Here’s how the library spans the HR diagram.

fig = figure()
plot(lib.library_params.Teff, lib.library_params.radius,'k.',)
smplot.label_axes('Teff','radius')

And here’s the library with the sources labeled.

fig = figure()
g = lib.library_params.groupby('source')
colors = ['Red','Orange','LimeGreen','Cyan','RoyalBlue','Magenta','ForestGreen']
i = 0
for source, idx in g.groups.items():
    cut = lib.library_params.ix[idx]
    color = colors[i]
    plot(
        cut.Teff, cut.radius,'+', label=source, color=color, alpha=1, ms=5, 
        mew=1.5
    ) 
    i+=1
legend()
smplot.label_axes('Teff','radius')

The parameters are stored in a pandas DataFrame which makes querying easy. Let’s grab some representative dwarf star spectra.

cut = lib.library_params.query('radius < 1.5 and -0.25 < feh < 0.25')
g = cut.groupby(pd.cut(cut.Teff,bins=arange(3000,7000,500)))
cut = g.first()

fig = figure()
plot(lib.library_params.Teff, lib.library_params.radius,'b.', label='_nolegend_')
plot(cut.Teff, cut.radius,'ro', label='Selected Stars')
legend()
smplot.label_axes('Teff','radius')
fig.savefig('quickstart-library-selected-stars.png')

_images/quickstart-library-selected-stars.png

Plot the Mgb region for the spectra sorted by effective temperature

from matplotlib.transforms import blended_transform_factory
fig,ax = subplots(figsize=(8,4))
trans = blended_transform_factory(ax.transAxes,ax.transData)
bbox = dict(facecolor='white', edgecolor='none',alpha=0.8)
step = 1
shift = 0
for _,row in cut.iterrows():
    spec = lib.library_spectra[row.lib_index,0,:]
    plot(lib.wav,spec.T + shift,color='RoyalBlue',lw=0.5)
    s = "{cps_name:s}, Teff={Teff:.0f}".format(**row)    
    text(0.01, 1+shift, s, bbox=bbox, transform=trans)
    shift+=step

grid()
xlabel('Wavelength (Angstroms)')
ylabel('Normalized Flux (Arbitrary Offset)')

_images/quickstart-spectra-selected-stars.png

Shifting¶

In the rest of this document, we will look at two example stars: HD 190406 (spectral class G0V), and Barnard’s star (GL 699), an M dwarf. Both of these stars are already in our library, so we will remove them from the library.

idx1 = lib.get_index('190406')
G_star = lib.pop(idx1)
idx2 = lib.get_index('GL699')
M_star = lib.pop(idx2)

To begin using SpecMatch, we first load in a spectrum and shift it onto the library wavelength scale. The raw HIRES spectra have been provided in the samples folder.

We read in the spectrum of HD 190406, in the same wavelength region as the library.

from specmatchemp import spectrum
G_spectrum = spectrum.read_hires_fits('../samples/rj59.1923.fits').cut(5130,5210)
G_spectrum.name = 'HD190406'

To shift the spectrum, we create the SpecMatch object and run shift().

This method runs a cross-correlation between the target spectrum and several reference spectra in the library for a select region of the spectrum. The reference spectrum which gives the largest cross-correlation peak is then used to shift the entire spectrum.

from specmatchemp.specmatch import SpecMatch
sm_G = SpecMatch(G_spectrum, lib)
sm_G.shift()

We can see the results of the shifting process. In this case, the code chose to use the NSO spectrum as the reference. We make use of the spectrum.Spectrum.plot() method to quickly plot each of the unshifted, shifted and reference spectra. Note that HIRES has overlapping orders so some spectral lines appear twice. In the final resampled spectrum, the small segments of overlapping orders are averaged.

fig = plt.figure(figsize=(10,5))
sm_G.target_unshifted.plot(normalize=True, plt_kw={'color':'forestgreen'}, text='Target (unshifted)')
sm_G.target.plot(offset=0.5, plt_kw={'color':'royalblue'}, text='Target (shifted): HD190406')
sm_G.shift_ref.plot(offset=1, plt_kw={'color':'firebrick'}, text='Reference: '+sm_G.shift_ref.name)
plt.xlim(5160,5200)
plt.ylim(0,2.2)

We repeat the same process for the M star spectrum. In this case, we see that the code chose to shift the spectrum to a different reference: HD216899, which is another M dwarf in our library. We have used the convenience function SpecMatch.plot_shifted_spectrum() to easily plot the shift results.

# Load spectrum
M_spectrum = spectrum.read_hires_fits('../samples/rj130.2075.fits').cut(5130,5210)
M_spectrum.name = 'GL699'

# Shift spectrum
sm_M = SpecMatch(M_spectrum, lib)
sm_M.shift()

# Plot shifts
fig = plt.figure(figsize=(10, 5))
sm_M.plot_shifted_spectrum(wavlim=(5160, 5200))

Matching¶

Now, we can perform the match. We first run SpecMatch.match(), which performs a match between the target and every other star in the library, allowing v sin i to float and fitting a spline to the continuum.

We can then plot the chi-squared surfaces to see where the best matches lie in parameter space.

sm_G.match(wavlim=(5140,5200))

# Plot chi-squared surfaces
fig = figure(figsize=(12, 8))
sm_G.plot_chi_squared_surface()
# Indicate library parameters for target star.
axes = fig.axes
axes[0].axvline(G_star[0]['Teff'], color='k')
axes[1].axvline(G_star[0]['radius'], color='k')
axes[2].axvline(G_star[0]['feh'], color='k')

_images/quickstart-Gstar-chisquared-surface.png

We see that the code does a good job in finding the library stars which are close to parameter space to the target star.

Next, running SpecMatch.lincomb(), the code synthesizes linear combinations of the best matching spectra to obtain an even better match to the target.

The respective coefficients will be used to form a weighted average of the library parameters, which are taken to be the derived properties of the target. These are stored in the results attribute.

sm_G.lincomb()

print('Derived Parameters: ')
print('Teff: {0:.0f}, Radius: {1:.2f}, [Fe/H]: {2:.2f}'.format(
    sm_G.results['Teff'], sm_G.results['radius'], sm_G.results['feh']))
print('Library Parameters: ')
print('Teff: {0:.0f}, Radius: {1:.2f}, [Fe/H]: {2:.2f}'.format(
    G_star[0]['Teff'], G_star[0]['radius'], G_star[0]['feh']))

Derived Parameters:
Teff: 5911, Radius: 1.17, [Fe/H]: 0.08
Library Parameters:
Teff: 5763, Radius: 1.11, [Fe/H]: 0.03

We can take a closer look at the workings of this process by plotting the positions of the best matching library stars in a HR diagram, as well as their spectra.

# Plot HR diagram
fig1 = figure(figsize=(12, 10))
sm_G.plot_references(verbose=True)
# plot target onto HR diagram
axes = fig1.axes
axes[0].plot(G_star[0]['Teff'], G_star[0]['radius'], '*', ms=15, color='red', label='Target')
axes[1].plot(G_star[0]['Teff'], G_star[0]['radius'], '*', ms=15, color='red')
axes[2].plot(G_star[0]['feh'], G_star[0]['radius'], '*', ms=15, color='red')
axes[3].plot(G_star[0]['feh'], G_star[0]['radius'], '*', ms=15, color='red')
axes[0].legend(numpoints=1, fontsize='small', loc='best')


# Plot reference spectra and linear combinations
fig2 = plt.figure(figsize=(12,6))
sm_G.plot_lincomb()

_images/quickstart-Gstar-lincomb-references.png

_images/quickstart-Gstar-lincomb-spectra.png

Finally, we can repeat the whole process with Barnard’s star.

# Perform match
sm_M.match(wavlim=(5140,5200))

# Plot chi-squared surfaces
fig1 = figure(figsize=(12,8))
sm_M.plot_chi_squared_surface()
# Indicate library parameters for target star.
axes = fig1.axes
axes[0].axvline(M_star[0]['Teff'], color='k')
axes[1].axvline(M_star[0]['radius'], color='k')
axes[2].axvline(M_star[0]['feh'], color='k')

# Perform lincomb
sm_M.lincomb()

print('Derived Parameters: ')
print('Teff: {0:.0f}, Radius: {1:.2f}, [Fe/H]: {2:.2f}'.format(
    sm_M.results['Teff'], sm_M.results['radius'], sm_M.results['feh']))
print('Library Parameters: ')
print('Teff: {0:.0f}, Radius: {1:.2f}, [Fe/H]: {2:.2f}'.format(
    M_star[0]['Teff'], M_star[0]['radius'], M_star[0]['feh']))

# Plot HR diagram
fig2 = figure(figsize=(12,10))
sm_M.plot_references(verbose=True)
# plot target onto HR diagram
axes = fig2.axes
axes[0].plot(M_star[0]['Teff'], M_star[0]['radius'], '*', ms=15, color='red', label='Target')
axes[1].plot(M_star[0]['Teff'], M_star[0]['radius'], '*', ms=15, color='red')
axes[2].plot(M_star[0]['feh'], M_star[0]['radius'], '*', ms=15, color='red')
axes[3].plot(M_star[0]['feh'], M_star[0]['radius'], '*', ms=15, color='red')
axes[0].legend(numpoints=1, fontsize='small', loc='best')

# Plot reference spectra and linear combinations
fig3 = plt.figure(figsize=(12,6))
sm_M.plot_lincomb()

Derived Parameters:
Teff: 3185, Radius: 0.19, [Fe/H]: -0.39
Library Parameters:
Teff: 3222, Radius: 0.19, [Fe/H]: -0.39

_images/quickstart-Mstar-chisquared-surface.png

_images/quickstart-Mstar-lincomb-references.png

_images/quickstart-Mstar-lincomb-spectra.png

Command Line Interface¶

Here, we describe how to use the command-line interface for specmatch-emp.

Installation¶

The command-line hooks are generated when you install specmatch-emp. Run the installation script to get this to work.

$ python setup.py install

Specmatch¶

To run the full specmatch algorithm on a target spectrum, use the smemp specmatch command. The full signature is:

$ smemp specmatch [-o OUTDIR] [-p] [-n NUM_BEST] [-s SUFFIX] [i LIB_NAME] spectrum

spectrum: The full path to the target spectrum.

Options¶

-o <outdir>: Specifies the directory to place the output files. If no directory is specified, output files are placed in the current directory.
-p: Set this flag to generate plots. -p generates a set of representative plots for the shifting, matching, and linear combination processes, using the results from order 2. -pp generates the above plots, but for every order used.
-n <num_best>: Sets the number of best-matching spectra to use in the linear combination process. Defaults to 5.
-s <suffix>: Suffix to append to output file names.
-i <lib_name>: Excludes the spectrum with the given name in the library, from the specmatch algorithm. Meant for validation purposes.
–n_lib_subset <n>: Run SpecMatch-Emp with only n random stars (default = 25). Useful for debugging purposes.

Shift¶

This command runs only the shifting part of the code, placing the target spectrum onto the library (NSO) wavelength scale.

$ smemp shift [-d DIRECTORY] [-o OUTDIR] [-f] [-nb] [-p] [-s SUFFIX] spectrum

spectrum: Either the full path to the target spectrum, or a CPS id (jXX.XXXX), where the -d option specifies the directory to look for the file. If a CPS id is provided, SpecMatch-Emp will search for all chips and shift them simultaneously.

Options¶

-d <directory>: Input directory to check for spectrum file, if a CPS id, instead of a path, was provided. Defaults to ${SMEMP_WKDIR}/spectra/.
-o <outdir>: Specifies the directory to place the output files. If no directory is specified, output files are placed in the current directory.
-f –flatten: If the spectra from multiple chips are provided, flatten all into a single file.
-nb –no_bootstrap: Set this flag to perform the shift directly against the NSO spectrum, without bootstrapping. Useful for shifting non-HIRES spectra which have wavelength ranges not overlapping the library spectra.
-p: Set this flag to generate plots. -p generates a set of representative plots for the shifting process, using the results from order 2. -pp generates the above plots, but for every order used.
-s <suffix>: Suffix to append to output file names. Defaults to _adj

Match¶

This command reads in a shifted and flattened spectrum, and performs a match with the library spectra. It outputs a .hdf file containing the specmatch-emp results, as well as a .csv file with the best chi-squared values when compared against each library spectrum.

smemp match [-h] [-d DIRECTORY] [-o OUTDIR] [-p] [-i IN_LIBRARY] [-s SUFFIX] spectrum

spectrum: Either the full path to the target spectrum, or a CPS id (jXX.XXXX), where the -d option specifies the directory to look for the file.

Options¶

-d <directory>: Input directory to check for spectrum file, if a CPS id, instead of a path, was provided. Defaults to ${SMEMP_WKDIR}/shifted_spectra/.
-o <outdir>: Specifies the directory to place the output files. If no directory is specified, output files are placed in the current directory.
-p: Set this flag to generate plots. -p generates a set of representative plots for the matching process, using the results from order 2. -pp generates the above plots, but for every order used.
-i <lib_name>: Excludes the spectrum with the given name in the library, from the specmatch algorithm. Meant for validation purposes.
-s <suffix>: Suffix to append to output file names.

Lincomb¶

This takes an intermediate .hdf file generated by the match command, and runs the linear combination step, generating the final stellar parameters.

smemp lincomb [-h] [-o OUTDIR] [-p] [-i IN_LIBRARY] [-n NUM_BEST] [-s SUFFIX] match_results

match_results: Path to .hdf file to be resumed.

Options¶

-o <outdir>: Specifies the directory to place the output files. If no directory is specified, output files are placed in the current directory.
-p: Set this flag to generate plots. -p generates a set of representative plots for the lincomb process, using the results from order 2. -pp generates the above plots, but for every order used.
-i <lib_name>: Excludes the spectrum with the given name in the library, from the specmatch algorithm. Meant for validation purposes.
-n <num_best>: Sets the number of best-matching spectra to use in the linear combination process. Defaults to 5.
-s <suffix>: Suffix to append to output file names.

Building the Library¶

The SpecMatch libary is stored as a monolithic HDF5 file and is downloaded upon installation of the code. It’s is constructed from a number of different data files. For simplicity, we do not make all the source files public. This document serves as a recipe for the specmatchemp developers.

Establish Wavelength References¶

Raw spectra from CPS are not wavelength-calibrated. They are stored as a list of intensities and pixel values. Spectra are resampled onto the rest wavelength scale using the NSO solar spectrum by cross-correlating spectral segments of ~500 HIRES pixels. When the spectra are significantly different from solar, the cross-correlation peak is not well-defined. We have implemented a bootstrapping approach to accomodate a range of spectral types. We establish a ladder of spectra seperated in effective temperature by 500–1000 K. These spectra are:

$ ls ${SMEMP_WKDIR}/spectra/
nso.fits # solar spectrum, ~5800 K
*j55.1872.fits # ~4800 K
*j26.532.fits # ~3700 K
*j59.1926.fits # ~6200 K

Where at least the r chip must be provided.

We shift the spectra using by

$ python specmatchemp/buildlib/shift_references.py
Shifting spectrum j55.1872 onto reference nso
Shifting spectrum j26.532 onto reference nso
Shifting spectrum j59.1926 onto reference j55.1872

Which leaves the wavelength calibrated spectra here:

$ ls ${SMEMP_WKDIR}/shifted_spectra/
nso_adj.fits
j26.532_adj.fits
j55.1872_adj.fits
j59.1926_adj.fits

Build Table of Library Parameters¶

The parameters of the stars in the SpecMatch library come from a variety of literature sources. Run python buildlib/read_catalogs.py to read the parameters into a master csv file. The library sources are heterogeneous, giving (some have logg while others give stellar radius). Missing values are given as nans.

By default, the parameters are stored in the file:

${SMEMP_WKDIR}/libstars.csv

Fill in Parameters with Isochrones¶

The missing values are filled in with Dartmouth isochrones. Run python buildlib/get_isochrones.py to fill in the missing values.

This script creates the following files:

${SMEMP_WKDIR}/libstars_mask.csv
${SMEMP_WKDIR}/isochrone_models/
${SMEMP_WKDIR}/isochrone_models/errors.txt

The libstars_mask.csv file contains a boolean mask, marked with True for literature parameters, and False for parameters derived using isochrones.

The isochrone_models/ directory is used to contain the models generated by isochrones. By default, get_isochrones.py looks in this folder for existing models with the same cps_name, and does not run the fit again. Overwrite this behavior with the -o flag.

The script checks the Dartmouth models for consistency with the existing library parameters (and does not overwrite them). If the MCMC/MultiNest 2-sigma range fails to overlap with the library values, it prints a warning message into errors.txt.

Shift Library Spectra onto Rest Wavelength Scale¶

We shift the remainder of the library spectra onto the rest wavelength scale with the following command, where jXXX.XXXX is the cps observation id. Only the rj chip is required.

$ smemp shift jXXX.XXXX

Build Library HDF5 File¶

Finally, we assemble all the intermediate files into a monolithic HDF5 file that contains the spectra and their associated parameters using the following script. This requires the parameter csv file, parameter mask csv file, and all the spectra contained in *_adj.fits files.

$ python buildlib/combine_library.py

If the -a flag is specified, this will append the parameters and spectra to the existing library at ${SMEMP_WKDIR}/library.h5

Library¶

class specmatchemp.library.Library(wav=None, library_spectra=None, library_params=None, header={}, wavlim=None, param_mask=None, nso=None)¶

A container for a library of spectrum and corresponding stellar parameters.

The Library class is a container for the library spectrum and stellar parameters for the library stars. The library is indexed using the library_index column.

library_params¶: pd.DataFrame – The parameter table, which can be queried directly as a pandas DataFrame.

library_spectra¶: np.ndarray – 3D array containing the library spectra ordered according to the index column.

wav¶: np.ndarray – Common wavelength scale for the library spectra.

param_mask¶: pd.DataFrame – Boolean dataframe with same rows and columns as library_params, marked True for model-independent parameters.

nso¶: Spectrum – NSO spectrum.

Parameters:

wav (np.ndarray, optional) – Wavelength scale for the library spectra.
library_params (pd.DataFrame, optional) – Pandas DataFrame containing the parameters for the library stars. It should have the columns specified in LIB_COLS, although values can be np.nan. The library_index of each row should be the index of the specturm in library_spectra.
library_spectra (np.ndarray, optional) – 3D array containing the library spectra ordered according to the index column. Each entry contains the spectrum, its uncertainty, and a mask array.
header (dict, optional) – Any additional metadata to store with the library.
wavlim (tuple, optional) – The upper and lower wavelength limits to be read.
param_mask (pd.DataFrame, optional) – Boolean dataframe with same rows and columns as library_params, marked True for model-independent parameters.
nso (Spectrum) – NSO spectrum

LIB_COLS = ['lib_index', 'cps_name', 'lib_obs', 'source', 'source_name', 'Teff', 'u_Teff', 'radius', 'u_radius', 'logg', 'u_logg', 'feh', 'u_feh', 'mass', 'u_mass', 'age', 'u_age', 'vsini', 'Plx', 'u_Plx', 'Plx_source', 'Vmag', 'snr']¶: Columns required in library

STAR_PROPS = ['Teff', 'radius', 'logg', 'feh', 'mass', 'age']¶: Numeric star properties

append(params, spectra=None, param_mask=None)¶

Adds spectrum and associated stellar parameters into library.

Parameters:

params (pd.Series or pd.DataFrame) – Row(s) to be added to the parameter table. It should have the fields specified in Library.LIB_COLS.
spectra (Spectrum, list of Spectrum, or np.ndarray) – Spectra to be added. - If Spectrum objects are provided, they will be interpolated onto the library wavelength scale if not already on it. - If np.ndarray is provided, it must be of shape (len(params), 3, len(library.wav)) and should have already been shifted onto the library wavelength scale.
param_mask (pd.Series or pd.DataFrame, optional) – Parameter mask. Must be same length as params.

copy()¶: Return a deep copy of the library object.

get_index(searchlist)¶

Searches the library for the given search string. Checks columns lib_obs, cps_name, source_name in order.

Parameters:	searchlist (str or list of str) – String to search for.
Returns:	Library index of the found star. Returns None if no object found.
Return type:	int

get_spectrum(indices)¶

Returns the spectrum at the given index.

Parameters:	indices (int or list) – Library indices of spectrum.
Returns:	Spectra at given indices.
Return type:	Spectrum or list of Spectrum

plot(paramx, paramy, grouped=False, marker='.', ptlabels=False, plt_kw={})¶

Create a plot of the library in parameter space

Parameters:	paramx (str) – Parameter to plot on the x-axis paramy (str) – Parameter to plot on the y-axis grouped (bool, optional) – Whether to group the stars by source ptlabels (str, optional) – Library column to annotate points with plt_kw (dict, optional) – Additional keyword arguments to pass to pyplot.plot

pop(index)¶

Removes the spectrum and parameters with the given index from the library.

Parameters:	index (int) – Index of spectrum to remove.
Returns:	params (pd.Series): Parameters of star spectrum (Spectrum): Spectrum
Return type:	(pd.Series, spectrum.Spectrum)

query_params(querystr)¶

Query the library_params table and return the indices of matching stars.

Parameters:	querystr (str) – Query string
Returns:	Index or indices of library entries matching the query string.
Return type:	int or list of int

remove(indices)¶

Removes the spectrum and parameters with the given index from the library.

Parameters:	index (int or array of ints) – Indices of spectra to remove.

to_csv(path)¶

Saves library parameters as a CSV file, using Pandas native to_csv function.

Parameters:	path (str) – Path to store csv file.

to_hdf(path)¶

Saves library as a HDF file

Parameters:	path (str) – Path to store library

to_tex(path, cols='standard', mode='w')¶

Outputs library parameters into a tex file.

Parameters:	path (str) – Path to store tex file. cols (str or list) – ‘standard’ or list of columns to be used mode (str) – Writing mode to pass to open(), either ‘w’ or ‘a’

wav_cut(minw, maxw, deepcopy=False)¶

Returns a library with spectra within the given limits.

Parameters:	minw (float) – Minimum wavelength maxw (float) – Maximum wavelength deepcopy (bool, optional) – True to return a library with all attributes copied. Default is False.
Returns:	New library object.
Return type:	Library

specmatchemp.library.read_hdf(path=None, wavlim='all', lib_index_subset=None)¶

Reads in the library from a HDF file.

Parameters:	path (str, optional) – path to h5 file containing library. If no path is given, reads in from default directory. wavlim (2-element iterable or str, optional) – The upper and lower wavelength limits to be read. If ‘none’, reads in library without spectra. If ‘all’, reads in complete stored spectra. lib_index_subset (iterable, optional) – Load a subset of the library useful for debuging and testing code. May be a list or a np.s_ object. If passing a list, it must be in strictly increasing order due to the memory access protocol of h5py.
Returns:	Library object
Return type:	Library

Spectrum¶

class specmatchemp.spectrum.Spectrum(w, s, serr=None, mask=None, name=None, header=None, attrs={})¶

Spectrum class.

This object is a container for a spectrum and its properties.

Parameters:	w (np.ndarray) – Wavelength scale s (np.ndarray) – Spectrum serr (np.ndarray, optional) – Error in spectrum mask (np.ndarray, optional) – Boolean array to mask out telluric lines name (str, optional) – Name associated with spectrum header (FITS header, optional) – Header from fits file attrs (dict, optional) – Any further attributes

w¶: np.ndarray – Wavelength scale

s¶: np.ndarray – Spectrum

serr¶: np.ndarray – Measurement error in spectrum

mask¶: np.ndarray – Boolean array with telluric line positions

name¶: str – Name associted with spectrum

header¶: FITS header – Header from FITS file

attrs¶: dict – A dictionary of further attributes

static combine_spectra(spectra, w_ref, name=None, prefixes=None)¶

Combines several spectra into one spectrum object.

Combine multiple spectrum objects into a single spectrum. Places np.nan for the gaps between the spectra, and averages overlapping portions.

Parameters:	spectra (iterable) – Iterable of spectrum objects w_ref (np.ndarray) – Reference wavelength scale. This scale is used to fill in the gaps between the spectra. name (str, optional) – Name for the flattend spectrum prefixes (str, optional) – Prefixes for attributes

copy()¶: Returns a deep copy of the Spectrum object

cut(minw, maxw)¶

Truncate the spectrum between the given limits

Parameters:	minw (float) – Minimum wavelength maxw (float) – Maximum wavelength
Returns:	Truncated spectrum object
Return type:	Spectrum

extend(w)¶

Extend the spectrum onto a new wavelength scale, placing np.nan for points which do not exist on the old scale.

Parameters:	w (np.ndarray) – New wavlength array.

on_scale(w)¶

Checks if spectrum is on the given wavelength scale

Parameters:	w (np.ndarray) – Wavelength scale to check against

plot(wavlim='all', offset=0, label='_nolegend_', showmask=False, normalize=True, plt_kw={'color': 'RoyalBlue'}, text='', text_kw={})¶

Plots the spectrum.

Parameters:

offset (float, optional) – Vertical offset of the spectrum
label (str, optional) – Label of spectrum (appears in plt.legend)
showmask (bool, optional) – Whether to highlight the telluric mask in the plot. Defaults to False.
plt_kw (dict, optional) – Keyword arguments to pass to plt.plot
text (str, optional) – String to label the spectrum
text_kw (dict, optional) – Keyword arguments to pass to plt.text

rescale(w)¶

Puts the spectrum onto a new wavelength scale, interpolating between points if necessary. Places np.nan for points beyond existing range.

Parameters:	w (np.ndarray) – New wavelength scale.

to_fits(outpath, clobber=True)¶

Saves the spectrum to a fits file.

Parameters:	outpath (str) – Path to output file

to_hdf(outfile, suffix='')¶

Saves the spectrum to a hdf file.

Parameters:	outfile (str or h5 file) – Output path or file handle suffix (str, optional) – Suffix to append to h5 field names

to_hdu()¶

Creates a fits.BinTableHDU object from spectrum data

Returns:	A binary table HDU object
Return type:	fits.BinTableHDU

wavlim()¶: Gets the wavelength range of the spectrum

class specmatchemp.spectrum.HiresSpectrum(w, s, serr=None, mask=None, mask_table=None, name=None, header=None, attrs={})¶

Spectrum class for raw HIRES spectra.

Is read by the read_hires_fits function.

w¶: np.ndarray – Wavelength scale

s¶: np.ndarray – Spectrum

serr¶: None or np.ndarray – Measurement error in spectrum

mask¶: np.ndarray – Boolean array with telluric line positions

name¶: str – Name associted with spectrum

header¶: FITS header – Header from FITS file

attrs¶: dict – A dictionary of further attributes

Parameters:

w (np.ndarray) – Wavelength scale
s (np.ndarray) – Spectrum
serr (np.ndarray, optional) – Error in spectrum
mask (np.ndarray, optional) – Boolean array to mask out telluric lines
mask_table (pd.DataFrame, optional) – Table containing masked regions
name (str, optional) – Name associated with spectrum
header (FITS header, optional) – Header from fits file
attrs (dict, optional) – Any further attributes

static combine_spectra(spectra, name=None, prefixes=None)¶

Combines several raw HIRES spectra into one HIRES spectrum object.

Parameters:	spectra (iterable) – Iterable of spectrum objects name (str, optional) – Name for the flattend spectrum prefixes (str, optional) – Prefixes for attributes

plot(wavlim='all', offset=0, label='_nolegend_', normalize=False, showmask=False, plt_kw={'color': 'RoyalBlue'}, text='', text_kw={})¶

Plots the spectrum

Parameters:

wavlim (optional [tuple]) – Wavelength limit to plot
offset (optional [float]) – Vertical offset of the spectrum
label (optional [str]) – Label of spectrum (appears in plt.legend)
normalize (optional [bool]) – Whether to normalize the spectrum. Defaults to False
showmask (bool, optional) – Whether to highlight the telluric mask in the plot. Defaults to False.
plt_kw (optional [dict]) – Keyword arguments to pass to plt.plot
text (str, optional) – String to label the spectrum
text_kw (dict, optional) – Keyword arguments to pass to plt.text

to_hdulist(primary=True)¶

Creates a list of fits.ImageHDU to store HIRES spectrum.

Parameters:	primary (bool) – True if first HDU should be a primary HDU
Returns:	l[0] = spectrum l[1] = spectrum error l[2] = wavelength
Return type:	list of fits.ImageHDU

specmatchemp.spectrum.read_fits(infile, wavlim=None)¶

Reads a spectrum from a fits file

Parameters:	infile (str) – Path to input fits file wavlim (tuple, optional) – Wavelength limits to read
Returns:	Spectrum object
Return type:	Spectrum

specmatchemp.spectrum.read_hires_fits(infile, maskfile=None)¶

Reads a spectrum from a fits file that was produced by HIRES.

Parameters:	infile (str) – Path to input fits file maskfile (optional [str]) – Path to file containing telluric lines mask.
Returns:	Spectrum object
Return type:	spec (HiresSpectrum)

specmatchemp.spectrum.read_hdf(infile, suffix='')¶

Reads a spectrum from a hdf file

Parameters:	infile (str or h5 file) – Input path or file handle suffix (str, optional) – Suffix on h5 keys
Returns:	Spectrum object
Return type:	spec (Spectrum)

Specmatch¶

class specmatchemp.specmatch.SpecMatch(target, lib=None, wavlim=(5000, 5900))¶

SpecMatch class to perform the SpecMatch routine.

Begin by using the shift() method to shift the target spectrum onto the library wavelength scale. This uses a bootstrapping approach to find the best reference spectrum to use as a template for shifting.

Next, the match() method runs a match against each library spectrum, computing a chi-squared value for each target-library pair.

Finally, use the lincomb() method to interpolate between the best library matches to obtain the final derived parameters, stored in the results dict.

Parameters:	target (spectrum.Spectrum) – Target spectrum and uncertainty lib (library.Library) – Library object to match against wavlim (tuple, optional) – Wavelength limits to perform matching on.

target¶: spectrum.Spectrum – Target spectrum object

target_unshifted¶: spectrum.Spectrum – An unshifted spectrum

shift_ref¶: spectrum.Spectrum – Reference used to shift the spectrum

shift_results¶: dict – Dictionary containing results from shifting.

regions¶: list of tuples – List of wavelength regions used for matching.

match_results¶: pd.DataFrame – Parameter table including chi_squared and fit_params from single match routine.

lincomb_matches¶: list of match.MatchLincomb – MatchLincomb objects for final matching to a linear combination of spectra

lincomb_results¶: list of dicts – Results from each MatchLincomb fit.

results¶: dict – Dictionary of final stellar parameters produced by SpecMatch. These are derived by taking the average of the detrended parameters from each wavelength region. Keys are elements of library.STAR_PROPS.

lincomb(num_best=5, regions='all')¶

Interpolate between library spectra to get more accurate parameters.

Takes the best num_best matches obtained in the match() step and creates linear combinations of their spectra. The respective coefficients for each spectrum will be used to take a weighted average of their parameters for use as the final derived parameters.

Parameters:	num_best (int, optional) – Specify the number of best matches to be used to synthesize the linear combinations. regions (list of tuples, optional) – Select wavelength regions to perform the matching procedure on. Defaults to ‘all’, which uses all the regions used in the previous match() step.

match(wavlim=(5000, 5900), wavstep=100, ignore=None)¶

Match the target against the library spectra.

Performs a pairwise match between the target spectrum and every library spectra, fitting for vsini broadening and a spline to the continuum level. For each target-reference pair, a best chi-squared value is calculated to assess the match between the two spectra.

The results are stored in specmatchemp.SpecMatch.match_results

Parameters:

wavlim (tuple or list of tuples, optional) – Wavelength region(s) to use in matching process. Defaults to None - use the entire region overlapped by the target spectrum and library, split into sections specified by wavstep. If a tuple is given, it will be split into sections specified by wavstep. If a list of tuples is given, each tuple is a section to be used for matching.
wavstep (float, optional) – Length of wavelength regions to be used. Defaults to 100 Angstrom regions. If None, uses the entire region specified in wavlims
ignore (int, optional) – A library index to ignore. Useful for n-1 library test.

plot_best_match_spectra(region=0, wavlim='all', num_best=None)¶

Plots the reference, modified reference and residuals for each of the best matches.

Parameters:

region (int or tuple) – The region used in matching. If an int is given, refers to the index of the region in self.regions. If a tuple is given, should be present in self.regions.
wavlim (str or tuple) – The wavelength limits to plot. If ‘all’ is given, plots the entire region.
num_best (optional [int]) – Number of best spectra to highlight. (default: self.num_best)

plot_chi_squared_surface(region=0, num_best=None)¶

Plot the chi-squared surface from the pairwise matching procedure.

Creates a three-column plot of the best chi-squared obtained with each library spectrum as a function of the library parameters.

Parameters:	region (int or tuple) – The wavelength region to plot. If an int is given, refers to the index of the region in self.regions. If a tuple is given, should be present in self.regions. num_best (optional [int]) – Number of best spectra to highlight. (default: self.num_best)

plot_lincomb(region=0, wavlim='all')¶: Plots the spectra used to synthesize the linear combination, the synthesized spectrum, and residuals.

plot_references(region=0, num_best=None, verbose=True)¶

Plots the location of the best references used in the linear combination step.

Parameters:	region (int or tuple) – The region used in matching. If an int is given, refers to the index of the region in self.regions. If a tuple is given, should be present in self.regions. num_best (optional [int]) – Number of best spectra to highlight. (default: self.num_best) verbose (optional [bool]) – Whether to annotate the points with the lincomb coefficients

plot_shift_lags(orders='all', legend=True)¶

Plot the lags for each order as a function of pixel number.

Parameters:	orders (str or int or list of ints) – ‘all’ (default): Plot all orders int: Plot only given order list of ints: Plot the orders provided.

plot_shifted_spectrum(wavlim=(5158, 5172))¶

Plot a comparison of the shifted, reference and unshifted spectra.

Parameters:	wavlim (tuple or list of tuples, optional) – Wavelength limits to plot.

plot_xcorr(order=0, highlightpeak=False, legend=True)¶

Plot the correlation array for each section of the given order.

Parameters:	order (int) – Order on chip to plot. Defaults to 0. highlightpeak (bool) – Whether to highlight the peak value.

classmethod read_hdf(infile, lib)¶

Reads a SpecMatch object from and hdf file.

Parameters:	infile (str or h5 file) – Input path or file handle. lib (library.Library) – Library used to create SpecMatch object.

results_to_txt(outfile, verbose=False)¶

Save final results to text.

Parameters:	outfile (str or file object) – Output file verbose (bool) – Whether to only print the final derived results or to also include not-detrended results and list of best matching spectra for each region

shift()¶

Shift the target spectrum onto the library wavelength scale.

Uses the Mgb triplet region (5120 - 5200 A), a section of spectrum containing much information, to determine which spectrum to use as the reference for shifting. It does this by initially shifting the target spectrum in that region to the allowed references and comparing the heights of the cross-correlation peaks. The reference with the highest median peak is used as the reference to shift the entire spectrum.

Returns:	The shifted spectrum, which is also stored in self.target
Return type:	spectrum.Spectrum

to_fits(outpath)¶

Saves the current state of the SpecMatch object to a fits file.

Parameters:	outpath (str) – Path to store output file.

to_hdf(outfile)¶

Saves the current state of the SpecMatch object to an hdf file.

Parameters:	outfile (str or h5 file) – Output path or file handle.

Match¶

class specmatchemp.match.Match(target, reference, mode='default', opt='nelder')¶

The Match class used for matching two spectra

target¶: Spectrum – Target spectrum

reference¶: Spectrum – Reference spectrum

modified¶: Spectrum – Modified reference, after broadening and fitting a spline to correct for continuum differences

spl¶: np.ndarray – Spline used to fit continuum level

best_params¶: lmfit.Parameters – Parameters used to create the best match.

best_chisq¶: float – Chi-squared from the best match

Parameters:	target (Spectrum) – Target spectrum reference (Spectrum) – Reference spectrum mode – default (unnormalized chi-square), normalized (normalized chi-square) opt – lm (Levenberg-Marquadt optimization), nelder (Nelder-Mead)

best_fit(params=None)¶: Calculates the best fit model by minimizing over the parameters: - spline fitting to the continuum - rotational broadening

best_residuals()¶

Returns the residuals between the target spectrum and best-fit spectrum.

Returns:	np.ndarray

broaden(vsini, spec)¶

Applies a broadening kernel to the given spectrum (or error)

Parameters:	vsini (float) – vsini to determine width of broadening spec (Spectrum) – spectrum to broaden
Returns:	Broadened spectrum
Return type:	broadened (Spectrum)

create_model(params)¶: Creates a tweaked model based on the parameters passed, based on the reference spectrum. Stores the tweaked model in spectra.s_mod and serr_mod.

get_spline_positions()¶

Wrapper function for getting spline positions

Returns:	knotx (np.ndarray)

load_params(params)¶: Method to create a model based on pre-determined parameters, storing it as the best fit model.

objective(params)¶

Objective function evaluating goodness of fit given the passed parameters.

Parameters:	params –
Returns:	Reduced chi-squared value between the target spectra and the model spectrum generated by the parameters

classmethod read_hdf(infile)¶

Reads the Match object from an HDF file.

Parameters:	infile (str or h5 file) – Input path or file handle.

to_hdf(outfile)¶

Saves the Match object to an hdf file.

Parameters:	outfile (str or h5 file) – Output path or file handle.

class specmatchemp.match.MatchLincomb(target, refs, vsini, mode='default', ref_chisq=None)¶

MatchLincomb class used to match a linear combination of spectra to match a target spectrum.

target¶: Spectrum – Target spectrum

refs¶: list of Spectrum – List of reference spectra

vsini¶: np.ndarray – Array of vsini used to broaden the reference spectra.

spl¶: np.ndarray – Spline used for continuum fitting.

modified¶: Spectrum – Best linear combination of spectra.

best_params¶: lmfit.Parameters – Parameters used to create the best match.

best_chisq¶: floatt – Chi-squared from the best match.

Parameters:	target (Spectrum) – Target spectrum refs (list of Spectrum) – Array of reference spectra vsini (np.ndarray) – array containing vsini broadening for each reference spectrum

best_fit()¶: Calculates the best fit model by minimizing over the parameters: - Coefficients of reference spectra - spline fitting to the continuum - rotational broadening

create_model(params)¶: Creates a tweaked model based on the parameters passed, based on the reference spectrum. Stores the tweaked model in spectra.s_mod and serr_mod.

get_lincomb_coeffs()¶

Wrapper function to get lincomb coefficients from MatchLincomb object

Returns:	coeffs (np.ndarray)

get_vsini()¶

Wrapper function to get vsini list from MatchLincomb object

Returns:	vsini (np.ndarray)

objective(params)¶

Objective function evaluating goodness of fit given the passed parameters.

Parameters:	params –
Returns:	Reduced chi-squared value between the target spectra and the model spectrum generated by the parameters

classmethod read_hdf(infile)¶

Reads the Match object from an HDF file.

Parameters:	infile (str or h5 file) – Input path or file handle.

to_hdf(outfile)¶

Saves the Match object to an hdf file.

Parameters:	outfile (str or h5 file) – Output path or file handle.

Shift¶

@filename shift.py

Shift a target spectrum onto a reference spectrum.

specmatchemp.shift.bootstrap_shift(targ, ref_list, store=None)¶

Shift a target spectrum using a bootstrapping approach.

First performs a cross-correlation of the target spectrum in a fixed wavelength region against each of the provided references. The reference spectrum which gives the highest median cross-correlation peak is used to shift the rest of the target spectrum.

Parameters:	targ (spectrum.Spectrum) – Target spectrum ref_list (list of spectrum.Spectrum) – List of reference spectra store (dict-like object, optional) – h5 file or dict to record diagnostic data.
Returns:	Shifted and flattened spectrum.
Return type:	shifted (Spectrum)

specmatchemp.shift.correlate(a, v, lowfilter=0)¶

Custom function to perform 1-dimensional cross-correlation

Parameters:	a (np.ndarray) – Input sequence v (np.ndarray) – Input sequence lowfilter (int) – Filter out components with wavelength above this number of pixels
Returns:	Symmetric cross-correlation array
Return type:	np.ndarray

specmatchemp.shift.flatten(w, s, serr=None, mask=None, w_ref=None, wavlim=None)¶

Flattens a given 2-D spectrum into a 1-D array. Merges overlapped points by taking the mean. If w_ref is given, fills values that don’t occur in the 2D spectrum with np.nan

Parameters:	w (np.ndarray) – Wavelength array s (np.ndarray) – Spectrum serr (np.ndarray, optional) – Uncertainty in spectrum mask (np.ndarray, optional) – Boolean mask w_ref (np.nadarray, optional) – Reference, 1-D wavelength array wavlim (2-element iterable, optional) – Wavelength limits
Returns:	Wavelength, spectrum and uncertainty in spectrum
Return type:	w, s, serr

specmatchemp.shift.rescale_w(s, serr, w, m, w_ref)¶

Place the given spectrum on the wavelength scale specified by w_ref

Parameters:	serr, w (s,) – The spectrum and original wavelength scale. w_ref – The desired wavelength scale
Returns:	The spectrum and associated error on the desired scale.

specmatchemp.shift.save_shift_to_fits(outpath, shifted, unshifted, shift_data, clobber=False)¶

Saves the complete shift data to a FITS file.

Parameters:	outpath (str) – Path to save output file shifted (Spectrum) – Shifted spectrum unshifted (HiresSpectrum) – Raw spectrum shift_data (dict) – Shift data clobber (bool) – Overwrite existing file at destination

specmatchemp.shift.shift(targ, ref, store=None, lowfilter=20)¶

Shifts the given spectrum by placing it on the same wavelength scale as the specified reference spectrum, then solves for shifts between the two spectra through cross-correlation.

Parameters:	targ (Spectrum) – Target spectrum ref (Spectrum) – Reference spectrum store (optional [file or dict]) – h5 file or dict to record diagnostic data.
Returns:	Adjusted and flattened spectrum
Return type:	shifted (Spectrum)

specmatchemp.shift.shift_data_to_hdu(shift_data)¶

Saves the shift data to a BinTableHDU.

Parameters:	shift_data (dict) – Shift data output from shift()

specmatchemp.shift.solve_for_shifts(s, mask, s_ref, mask_ref, lowfilter=20)¶

Solve for the pixel shifts required to align two spectra that are on the same wavelength scale.

Correlates the two spectra, then fits a quadratic to the peak in order to solve for sub-pixel shifts.

Parameters:	s – The target spectrum array mask – Mask array for the target spectrum s_ref – The reference spectrum mask_ref – Mask array for the reference spectrum
Returns:	The pixel shift, the lag and correlation data

Empirical SpecMatch¶

Code availability¶

Basic Usage¶

Attribution¶

Installation¶

Minimal Install¶

Software dependencies¶

Data Files¶

Quickstart¶

Library¶

Shifting¶

Matching¶

Command Line Interface¶

Installation¶

Specmatch¶

Options¶

Shift¶

Options¶

Match¶

Options¶

Lincomb¶

Options¶

Building the Library¶

Establish Wavelength References¶

Build Table of Library Parameters¶

Fill in Parameters with Isochrones¶

Shift Library Spectra onto Rest Wavelength Scale¶

Build Library HDF5 File¶

Library¶

Spectrum¶

Specmatch¶

Match¶

Shift¶

Indices and tables¶