Pulsarpy¶

This project is read from GitHub

Scripts¶

get_id_from_name.py¶

Given a file with a column of names of records of a given type of model in Pulsar, fetches the IDs. The record IDs are written to a new file.

usage: get_id_from_name.py [-h] -m MODEL -i INFILE -o OUTFILE

Named Arguments¶

`-m, --model`	The name of the Rails model class of the records for which we need to get to IDs.
`-i, --infile`	The input file containing record names, one per row.
`-o, --outfile`	The output file containing a column of record IDs. Each row corresponds to the same row in the input file.

get_missing.py¶

Given an input file containing record names, one per row, indicates whether the record exists by outputting a 1 if it exists and a 0 if it doesn’t. The input file of record names will be first sorted and deduplicated.

usage: get_missing.py [-h] -m MODEL -i INFILE -o OUTFILE

Named Arguments¶

`-m, --model`	The name of the model to import the records to, i.e. Biosample or CrisprModification.
`-i, --infile`	One or more record names, one per line.
`-o, --outfile`	The output file with two columns: 1) name, 2) status (1 for present, 0 for absent).

tab_import.py¶

Given a tab-delimited sheet, creates new records of the specified Model into Pulsar LIMS or updates existing records if the patch option is provided. Array values should be comma-delimted as this program will split on the comma and add array literals. Array fields are only assumed when the field name has an ‘ids’ suffix.

usage: tab_import.py [-h] -m MODEL -i INFILE [-p] [--no-append] [--skip-dups]
                     [-u]

Named Arguments¶

`-m, --model`	The name of the model to import the records to, i.e. Biosample or CrisprModification.
`-i, --infile`	The tab-delimited input file containing records (1 per row). There must be a field-header line as the first row, and field names must match record attribute names. Any field names that start with a ‘#’ will be skipped. Any rows that start with a ‘#’ will also be skipped (apart from the header line).
`-p, --patch`	Presence of this option means to PATCH instead of POST. The input file must contain a column by the name of record_id to designate the existing record to PATCH. You can use a record’s primary ID or name as the identifier. Default: False
`--no-append`	This option only has meaning when the –patch option is also specified. It’s presence indicates to not extend array values with the content to be patched when dealing with array data types. Thus, if you don’t want to overwrite the existing value for arrays, then skip this option. Default: False
`--skip-dups`	If an attempt to POST a duplicate record is made, the server will respond with a ActiveRecord::RecordNotUnique exception. Including this flag indicates to catch this exception and skip on to the next record to POST. Default: False
`-u, --upstream-ids`
	If patching records and you are providing the record identidifers using the value of a records upstream_identifier attribute, set this to True. Default: False

Client API Modules¶

pulsarpy.elasticsearch_utils¶

exception pulsarpy.elasticsearch_utils.MultipleHitsException[source]¶

Bases: Exception

Raised when a search that is expected to return as most 1 hit has more than this.

pulsarpy.models¶

A client that contains classes named after each model in Pulsar to handle RESTful communication with the Pulsar API.

exception pulsarpy.models.RecordNotFound[source]¶

Bases: Exception

” Raised when looking up a record by name, id, etc. and it isn’t found on the server.

exception pulsarpy.models.RecordNotUnique[source]¶

Bases: Exception

Raised when posting a record and the Rails server returns with the exception ActiveRecord::RecordNotUnique.

pulsarpy.models.remove_model_prefix(uid)[source]¶: Removes the optional model prefix from the given primary ID. For example, given the biosample record whose primary ID is 8, and given the fact that the model prefix for the Biosample model is “B-“, the record ID can be specified either as 8 or B-8. However, the model prefix needs to be stripped off prior to making API calls.

class pulsarpy.models.Model(uid=None, upstream=None)[source]¶

Bases: object

The superclass of all model classes. A model subclass is defined for each Rails model. An instance of a model class represents a record of the given Rails model.

Subclasses don’t typically define their own init method, but if they do, they need to make a call to ‘super’ to run the init method defined here as well.

Subclasses must be instantiated with the rec_id argument set to a record’s ID. A GET will immediately be done and the record’s attributes will be stored in the self.attrs dict. The record’s attributes can be accessed as normal instance attributes (i.e. record.name) rather than explicitly indexing the attrs dictionary, thanks to the employment of ``__getattr__(). Similarly, record attributes can be updated via normal assignment operations, i.e. (record.name = "bob"), thanks to employment of __setattr__().

Required Environment Variables:

PULSAR_API_URL
PULSAR_TOKEN

Find the record of the given model specified by self.MODEL_NAME. The record can be looked up in a few ways, depending on which argument is specified (uid or upstream). If both are specified, then the upstream argument will be ignored.

Parameters:	uid – The database identifier of the record to fetch, which can be specified either as the primary id (i.e. 8) or the model prefix plus the primary id (i.e. B-8). Could also be the record’s name if it has a name attribute (not all models do) and if so will be converted to the record ID. upstream – If set, then the record will be searched on its upstream_identifier attribute.

UPSTREAM_ATTR = 'upstream_identifier'¶: Most models have an attribute alled upstream_identifier that is used to store the value of the record in an “upstream” database that is submitted to, i.e. the ENCODE Portal. Not all models have this attribute since not all are used for submission to an upstream portal.

FKEY_MAP = {}¶: Abstract attribute of type dict that each subclass should fulfull if it has any foreign keys. Each key is a foreign key name and the value is the class name of the model it refers to.

PULSAR_LIMS_PREFIX = 'p'¶: A prefix that can be added in front of record IDs, names, model-record ID. This is useful when its necessary to add emphasis that these records exist or came from Pulsar ( i.e. when submitting them to an upstream database.

debug_logger = <Logger ppy_debug (DEBUG)>¶: This class adds a file handler, such that all messages sent to it are logged to this file in addition to STDOUT.

error_logger = <Logger ppy_error (ERROR)>¶: A logging instance with a file handler for logging terse error messages. The log file resides locally within the directory specified by the constant p.LOG_DIR. Accepts messages >= logging.ERROR.

post_logger = <Logger ppy_post (INFO)>¶: A logging instance with a file handler for logging successful POST operations. The log file resides locally within the directory specified by the constant p.LOG_DIR. Accepts messages >= logging.INFO.

ES = <pulsarpy.elasticsearch_utils.Connection object>¶: Connection to Elasticsearch. Expects that the envrionment variables ES_URL, ES_USER, and ES_PW are set, which signifiy the Elasticsearch cluster URL, login username and login password, respectively.

classmethod replace_name_with_id(name)[source]¶

Used to replace a foreign key reference using a name with an ID. Works by searching the record in Pulsar and expects to find exactly one hit. First, will check if the foreign key reference is an integer value and if so, returns that as it is presumed to be the foreign key.

Raises:	pulsarpy.elasticsearch_utils.MultipleHitsException – Multiple hits were returned from the name search. pulsarpy.models.RecordNotFound – No results were produced from the name search.

classmethod add_model_name_to_payload(payload)[source]¶

Checks whether the model name in question is in the payload. If not, the entire payload is set as a value of a key by the name of the model. This method is useful when some server-side Rails API calls expect the parameters to include the parameterized model name. For example, server-side endpoints that handle the updating of a biosample record or the creation of a new biosmample record will expect the payload to be of the form:

{ "biosample": {
    "name": "new biosample",
    "donor": 3,
    ...
   }
}

Parameters:	payload – dict. The data to send in an HTTP request.
Returns:	dict.

abbrev_id()[source]¶: This method is called when posting to the ENCODE Portal to grab an alias for the record being submitted. The alias here is composed of the record ID in Pulsar (i.e. B-1 for the Biosample with ID 1). However, the record ID is prefexed with a ‘p’ to designate that this record was submitted from Pulsar. This is used to generate a unique alias considering that we used to uses a different LIMS (Syapse) to submit records. Many of the models in Syapse used the same model prefix as is used in Pulsar, i.e. (B)Biosample and (L)Library. Thus, w/o the ‘p’ prefix, the same alias could be generated in Pulsar as a previous one used in Syapse.

delete()[source]¶: Deletes the record.

classmethod find_by(payload, require=False)[source]¶

Searches the model in question by AND joining the query parameters.

Implements a Railsy way of looking for a record using a method by the same name and passing in the query as a dict. as well. Only the first hit is returned, and there is no particular ordering specified in the server-side API method.

Parameters:	payload – dict. The attributes of a record to restrict the search to. require – bool. True means to raise a pulsarpy.models.RecordNotFound exception if no record is found.
Returns:	The JSON serialization of the record, if any, found by the API call. None: If the API call didnt’ return any results.
Return type:	dict
Raises:	pulsarpy.models.RecordNotFound – No records were found, and the require parameter is True.

classmethod find_by_or(payload)[source]¶

Searches the model in question by OR joining the query parameters.

Implements a Railsy way of looking for a record using a method by the same name and passing in the query as a string (for the OR operator joining to be specified).

Only the first hit is returned, and there is not particular ordering specified in the server-side API method.

Parameters:	payload – dict. The attributes of a record to search for by using OR operator joining for each query parameter.
Returns:	The JSON serialization of the record, if any, found by the API call. None: If the API call didnt’ return any results.
Return type:	dict

classmethod index()[source]¶

Fetches all records.

Returns:	dict. The JSON formatted response.
Raises:	requests.exceptions.HTTPError – The status code is not ok.

patch(payload, append_to_arrays=True)[source]¶

Patches current record and udpates the current instance’s ‘attrs’ attribute to reflect the new changes.

Parameters:	- hash. This will be JSON-formatted prior to sending the request. (payload) –
Returns:	dict. The JSON formatted response.
Raises:	requests.exceptions.HTTPError – The status code is not ok.

classmethod set_id_in_fkeys(payload)[source]¶

Looks for any keys in the payload that end with either _id or _ids, signaling a foreign key field. For each foreign key field, checks whether the value is using the name of the record or the actual primary ID of the record (which may include the model abbreviation, i.e. B-1). If the former case, the name is replaced with the record’s primary ID.

Parameters:	payload – dict. The payload to POST or PATCH.
Returns:	dict. The payload.

classmethod pre_post(payload)[source]¶: This class method should be implemented in subclasses only when there is sub-class specific logic that needs to occur prior to using the generalized post class method defined below in this class.

classmethod post(payload)[source]¶

Posts the data to the specified record.

Parameters:	payload – dict. This will be JSON-formatted prior to sending the request.
Returns:	dict. The JSON formatted response.
Raises:	Requests.exceptions.HTTPError – The status code is not ok. RecordNotUnique – The Rails server returned the exception ActiveRecord::RecordNotUnique.

classmethod log_error(msg)[source]¶

Logs the provided error message to both the error logger and the debug logger logging instances.

Parameters:	msg – str. The error message to log.

static write_response_html_to_file(response, filename)[source]¶

An aid in troubleshooting internal application errors, i.e. <Response [500]>, to be mainly beneficial when developing the server-side API. This method will write the response HTML for viewing the error details in the browesr.

Parameters:	response – requests.models.Response instance. filename – str. The output file name.

class pulsarpy.models.Address(uid=None, upstream=None)[source]¶

Pulsarpy¶

Scripts¶

get_id_from_name.py¶

Named Arguments¶

get_missing.py¶

Named Arguments¶

tab_import.py¶

Named Arguments¶

Client API Modules¶

pulsarpy.elasticsearch_utils¶

pulsarpy.models¶

pulsarpy¶

pulsarpy.utils¶

Indices and tables¶