File Recommendations¶

The first line of the file should contain names for the columns in the rest of the file.

The file should preferably be encoded using UTF-8.

Concepts¶

In order to start an import, we need to define three major components, an import file, an import configuration, and an import job.

The import file contains not only the contents of the file itself, but also some additional information, such as what delimiter is used to separate the columns.

The import configuration contains information about what file to import, what type of LIME objects we want to import the file to, and how to map what’s in the file to fields on the fields and relation of that LIME type.

Finally, when the import job is created it will start the actual import as soon as it can. The import job will then contain the status of the import, and any errors that might have occured during the process.

Importing a file¶

Let’s say we have the following file containing information about people we want to add to LIME:

name;e-mail;ship;rank
Ripley;ellen@nostromo.com;Nostromo;Warrant Officer
Dallas;dallas@weyland.com;Nostromo;Captain
Ash;ash@weyland.com;Nostromo;Science Officer
...

We have a LIME database that contains information about employees and ships, and we’ve been tasked with importing this file in to LIME.

We install the latest version of limeclient, create a module called weyland.py.

Authenticating¶

First of all, we need to authenticate ourselves with the LIME Pro server. If we’re using the API-client this done by creating an instance of LimeClient and logging in:

from limeclient import LimeClient

 client = LimeClient('https://myserver:2134', 'weyland_db')
 with client.login('user', 'pass') as c:
     # Do stuff here...

This will start a new session in LIME that will automatically be closed when program is finished.

If you’re importing data to a hosted LIME installation, you should not pass the database name argument.

Uploading a file¶

Once we’ve authenticated ourselves, it’s time for us to tell LIME about the file we want to import.

with client.login('user', 'pass') as c:
    with open('nostromo_crew.txt') as content:
        f = ImportFiles(c).create(filename='nostromo_crew.txt',
                                  content=content)
        f.delimiter = ';'
        f.save()

Here we uploaded a file to LIME, which returns with a file object populated with default values. We then told LIME a little bit more about how it should interpret the file, and saved the new information on the server.

Retrieving the LIME data type¶

The second piece of information we need before we can start configuring how to interpret the content of the file is an object that represents the data type in LIME.

If the information in the file contains information about people we might want to load the ‘crew’ LimeTypes:

with client.login('user', 'pass') as c:
    with open('nostromo_crew.txt') as content:
        f = ImportFiles(c).create(filename='nostromo_crew.txt',
                                 content=content)
        f.delimiter(';')
        f.save()

    crew = LimeTypes(c).get_by_name('crew')

Creating an import configuration¶

With that we have enough information to start configuring our import:

with client.login('user', 'pass') as c:
    with open('nostromo_crew.txt') as content:
        f = ImportFiles(c).create(filename='nostromo_crew.txt',
                                 content=content)
        f.delimiter(';')
        f.save()

    crew = LimeTypes(c).get_by_name('crew')

    config = ImportConfigs(c).create(lime_type=crew, importfile=f)

Behavior¶

We can tell LIME what it should do for each row it finds in our import file.

config.behavior = ImportConfig.CreateAndUpdate Update existing objects if they match what’s in the file, and create new objects if nothing matches. This is the default value for a new ImportConfig

config.behavior = ImportConfig.OnlyUpdate Only update objects that match what’s in the import file. Don’t create any new objects.

config.behavior = ImportConfig.OnlyCreate Create a new object for each row in the import file. Don’t try to match against existing objects in LIME.

Mapping¶

For each row in the file to import, LIME needs to know what to do with the data. We tell LIME how to accomplish this by telling it how to map each column to something a field or relation of the data type we’re import to.

The LIME API supports three types of mappings: mappings of simple types, such as strings and numbers, mappings option fields where the value can be one of several predefined values, and finally relations to other types of entities in LIME, such as persons being related to companies.

Adding a simple field mapping¶

In our example, the name and e-mail of the crew members are simple types, so we add simple field mappings for those. We also mark the name field as a key field, meaning that we will use this when determining if this row matches an existing object in LIME.

with client.login('user', 'pass') as c:
    # ...

    crew = LimeTypes(c).get_by_name('crew')

    config = ImportConfigs(c).create(lime_type=crew, importfile=f)

    name = SimpleFieldMapping(field=crew.fields['name'],
                               column='name',
                               key=False)
    config.add_mapping(name)

    email = SimpleFieldMapping(field=crew.fields['email'],
                               column='e-mail',
                               key=True)
    config.add_mapping(email)

We can specify ‘key=True’ for multiple mappings. In that case all values must match for the import to consider updating a person in LIME instead of adding a new.

Adding a mapping to an option field¶

You can map a column in the import file to an option field in LIME by adding an OptionFieldMapping to your import configuration.

Within the OptionFieldMapping, you specify how a value in a column translates to one of the possible values of an option field in LIME.

with client.login('user', 'pass') as c:
    # ...

    field = crew.fields['rank']
    position = OptionFieldMapping(field=field, column='rank')
    position.default = field.option_by_key('Engineer')
    position.map_option(column_val='Captain',
                        option=field.option_by_key('Captain'))
    position.map_option(column_val='Warrant Officer',
                        option=field.option_by_key('Warrant Officer'))
    config.add_mapping(position)

In the example above we first say that any values for the ‘rank’ column that haven’t been explicitly mapped, we should assume that the crew member is engineer.

We then proceed to explicitly map the values for captain and warrant officer.

Mapping relations¶

Finally, we need to import the ship of each crew member in the file. ‘Ship’ is a separate table in the LIME database and we need to tell the import about this:

with client.login('user', 'pass') as c:
    # ...

    crew = LimeTypes(c).get_by_name('crew')

    # ...

    relation = crew.relations['ship']
    ship = relation.related
    relation_mapping = RelationMapping(column='ship', relation=relation,
                                       key_field=ship.fields['name'])
    config.add_mapping(relation_mapping)

    config.save()

We ask the lime type for the relation to the ship type, we use that to get a hold of the actual ship type. We then tell the importer that the ‘ship’ column contains names of ships.

Now, we can save the import configuration and are ready to start the import.

Starting an import job¶

We can now start the import job:

with client.login(user=args.user, password=args.password) as c:
    # ...

    job = ImportJobs(c).create(config)

    for i in range(10):
       time.sleep(1)
       job = job.refresh()
       print('Current job status: {}'.format(job.status))
       if job.has_errors:
           print('Oh noes! Errors!')
           print(job.errors.errors[:10])
       if job.status != 'pending' and job.status != 'running':
           break

This tells LIME to put the import job on a queue. We then proceed to poll the status of the job. If something goes wrong, the ten first errors will be printed to the console.

LimeClient¶

class limeclient.LimeClient(host, database=None, debug=False, verify_ssl_cert=True)¶

Handles all communication with LIME’s API

Parameters:	host – name of host to connect to database – name of database to logon to. Should not be specified when logging on to a hosted LIME solution. debug – if True, print traffic to stdout. Defaults to False verify_ssl_cert – if False, ignore SSL certificate verification. Defaults to True.

login(user=None, password=None)¶

Log in to LIME.

LimeClient should be used as a context manager. That way logging out and closing a session will be done automatically, even if an error is encountered.

client = LimeClient('localhost', 'mydatabase')
with client.login('user', 'pass') as c:
    # do stuff

LimeTypes¶

class limeclient.LimeTypes(lime_client)¶

Retrieve type information about entities in LIME Pro.

Parameters:	lime_client – a logged in LimeClient instance

get_by_name(name)¶

Retrieve a LimeType given its name in LIME Pro.

Parameters:	name – name in LIME Pro (e.g. ‘company’)

get_by_url(url)¶

Retrieve a LimeType given its url.

Parameters:	url – this is the url that uniquely identifies an lime type.

class limeclient.LimeType(hal, lime_client)¶

Represents a type of object in LIME Pro.

name¶

Name of lime type.

is_system¶

True if this is a system type.

fields¶

Retrieve all fields for this lime type.

relations¶

Retrieve all relations (Relation) for this lime type.

ImportFiles¶

class limeclient.ImportFiles(lime_client)¶

Handles uploading of import files to LIME

Parameters:	lime_client – a logged in LimeClient instance

create(filename, content)¶

Upload an import file to LIME. Returns an ImportFile instance

Parameters:	filename – name of uploaded file content – a file object containing the data to import

class limeclient.ImportFile(hal, lime_client)¶

Represents a file to import to LIME.

Parameters:	hal – representation of an import file as returned from LIME. hal – lime_client LimeClient to use for communication with LIME

delimiter¶

Use this to set the delimiter used in the file so LIME knows how to parse it.

filename¶

The name of the file.

headers¶

Returns the headers (ImportFileHeaders) of the file.

save()¶

Save the file information in LIME.

class limeclient.ImportFileHeaders(hal, lime_client)¶

Contains the headers of a parsed import file.

Parameters:	hal – representation of an import file as returned from LIME. hal – lime_client LimeClient to use for communication with LIME

headers¶

A list of the header names in the file

Import Configuration¶

class limeclient.ImportConfigs(lime_client)¶

Manages creation of ImportConfig instances in LIME

Parameters:	lime_client – a logged in LimeClient instance

create(lime_type, importfile)¶

Create a new ImportConfig instance in LIME.

Parameters:	lime_type – LimeType instance that references what type of data to import importfile – ImportFile instance to import

class limeclient.ImportConfig(hal, lime_client)¶

Used for configuring an import.

Parameters:	hal – representation of an import file as returned from LIME. hal – lime_client LimeClient to use for communication with LIME

behavior¶

Determines how the import handles existing objects in LIME Pro. Can be one of the following:

• ImportConfig.CreateAndUpdate
• ImportConfig.UpdateOnly
• ImportConfig.CreateOnly

add_mapping(mapping)¶

Add information about how to map a column in the import file to data in LIME.

Parameters:	mapping – One of SimpleFieldMapping, OptionFieldMapping, or RelationMapping.

static create(lime_client, lime_type, importfile)¶

Create a new instance of ImportConfig

Parameters:	lime_client – a logged in LimeClient instance lime_type – LimeType instance that references what type of data to import importfile – ImportFile instance to import

save()¶

Save the import configuration in LIME.

validate()¶

Ask LIME to validate the import configuration. Returns an

ImportConfigStatus instance.

Import Jobs¶

class limeclient.ImportJobs(lime_client)¶

Handles the creation of a new import job.

Parameters:	lime_client – a logged in LimeClient instance

create(import_config)¶

Create a new ImportJob. This indicates to the server that it can start executing the job as soon as possible.

Parameters:	import_config – a ready ImportConfig instance.

get(url)¶

Retrieve an existing ImportJob from the server.

Parameters:	url – the url that identifies the job on the server.

class limeclient.ImportJob(hal, lime_client)¶

Represents an import job on the server.

Parameters:	lime_client – a logged in LimeClient instance

created_time¶

Time when the job was created.

started_time¶

Time when job was actually started.

completed_time¶

Time when job was completed.

created_count¶

Number of new items created.

updated_count¶

Number of already existing items that where updated.

errors_count¶

Number of errors encountered.

status¶

Current status of the job. Can be one of ‘pending’, ‘running’, ‘done’, ‘done_with_errors’, or ‘failed’

errors¶

Retrieve a ImportJobErrors that contains all errors for this job

has_errors¶

Determine if this job has encountered any errors.

refresh()¶

Retrieve a fresh version of the import job from the server.

class limeclient.ImportJobErrors(hal, lime_client)¶

count¶

The number of errors encountered during the job.

errors¶

A collection of all errors, grouped by rows in the imported file.

Example:

[{'row': 4, 'errors': ["Value of field 'name' is longer than the allowed 32 characters"]}]