SQLian: handles SQL so you don’t need to

SQLian is an all-in-one library that “shephards” you through interaction with SQL. Unlike an ORM (like the ones come with Django, SQLAlchemy, etc.), it does not try to hide the fact you’re writing SQL, but on the other hand still:

  • Frees you from handling pesky SQL syntax oddities, so you can better debug your SQL like your Python code.
  • Provides a unified interface to connect to different database implementations.
  • Automatic cursor handling anda better way to interact with the data returned by the SQL database.

The Basics

Connect to a database:

import sqlian
db = sqlian.connect('postgresql://...')

and perform a query:

rows = db.select(
    'name', 'occupation',
    from_='person',
    where={'main_language': 'Python'},
)

Now you can access the data directly:

>>> rows[0]
<Record {"name": "Mosky", "occupation": "Pinkoi"}>

or iterate over them:

for r in rows:
    print('{} works at {}'.format(r.name, r.occupation))

Interested? Read on!

Table of Contents

Installation Guide

I highly recommend you check out Pipenv by Kenneth Reitz to handle you project dependencies. With Pipenv, you can insall SQLian simply with:

$ pipenv install sqlian

The “non-modern” way to install SQLian is from the PyPI, through Pip:

$ pip install sqlian

The source code is also available at GitHub. You can download release packages directly on it, or use Pip and Git to install the in-development snapshot:

$ pip install git+https://github.com/uranusjr/sqlian.git

Or even just clone and install manually yourself:

$ git clone https://github.com/uranusjr/sqlian.git
$ cd sqlian
$ python setup.py install

Quickstart

SQLian is composed of three main parts:

  • Databases represent database connections.
  • Statement builders take native objects and convert them to a SQL command string. The built command are then passed to the associated database connection to be executed.
  • Records are returned by database queries. They offer a clean, nice interface to interact with the data retrieved from the database.

Let’s do a quick walk through on them one by one.

Connecting to a database

SQLian uses the 12factor-inspired database URL syntax to describe a database. This syntax is compatible with popular tools, including DJ-Database-URL, SQLAlchemy, and everything that builds on top of them. Which means, like, everything?

As an example, let’s connect to a PostgreSQL database:

import sqlian
db = sqlian.connect('postgresql://user:pa55@localhost/contactbook')

SQLian has some database support built-in. Some databases require extra dependencies to actually connect to, like psycopg2 for PostgreSQL. You can also build your own database support, but we’ll save that discussion for later.

The connect() function returns a Database instance, which conforms to the DB-API 2.0 specification (PEP 249), so you can get to work directly if you know your way around. But there’s a better way to do it.

Issuing commands

The Database instance provides a rich set of “statement builders” that format and execute SQL for you, and make it easier to convert native Python objects for SQL usage.

Inserting data:

db.insert('person', values={
    'name': 'Mosky',
    'occupation': 'Pinkoi',
    'main_language': 'Python',
})

This roughly translates to:

INSERT INTO "person" ("name", "occupation", "main_language")
VALUES ('Mosky', 'Pinkoi', 'Python')

but saves you from dealing with column and value clauses and all those %(name)s stuff.

You can still use column name–value sequences if you wish to:

db.insert(
    'person',
    columns=('name', 'occupation', 'main_language'),
    values=[
        ('Tim', 'GilaCloud', 'Python'),
        ('Adam', 'Pinkoi', 'JavaScript'),
    ],
)

Did I mention you can insert multiple rows at one go? Yeah, you can.

It’s also easy to update data:

db.update('person', where={'name': 'Adam'}, set={'main_language': 'CSS'})

Notice the key ordering does not matter.

You’d guess how deletion works by now, so let’s add a little twist:

db.delete('person', where={'occupation !=': 'Pinkoi'})

The builders automatically parse trailing operators and do the right thing.

Handling results

Some statements produce data. For every such query, SQLian returns an iterable object RecordCollection so you can handle them.

>>> rows = db.select(sqlian.star, from_='person')
>>> rows
<RecordCollection (pending)>

Accessing the content in any way automatically resolves it:

>>> rows[0]
<Record {"name": "Mosky", "occupation": "Pinkoi", "main_language": "Python"}>
>>> rows
<RecordCollection (1+ rows, pending)>
>>> for row in rows:
...     print(row)
<Record {"name": "Mosky", "occupation": "Pinkoi", "main_language": "Python"}>
<Record {"name": "Adam", "occupation": "Pinkoi", "main_language": "CSS"}>
>>> rows
<RecordCollection (2 rows)>

A Record can be accessed like a sequence, mapping, or even object:

>>> row = rows[0]
>>> row[0]
'Mosky'
>>> row['occupation']
Pinkoi
>>> row.main_language
Python

Database Conenctions

SQLian provide an interface to connect to relational database backends through the database API. The main way to do this is by passing a database URL to the connect() function, but it is also possible to explicitly create Database instances for various backends.

The Main Interface

sqlian.connect(url)

Create a database connection.

The database URL takes the form:

scheme://username:password@host:port/database

Multiple parts of the form can be omitted if not present. Common examples:

  • postgresql:///db connects to the local PostgreSQL instance via Un*x sockets without authentication, to the database db. Psycopg2, the default PostgreSQL driver, is used.
  • mysql+pymysql://localhost/db connects to the db database in the MySQL instance at localhost, using no authentication. The PyMySQL driver is used.
  • sqlite:///db.sqlite3 connects to the SQLite3 file at relative path db.sqlite3. The third slash is to seperate path from the host. Use four slashes if you need to specify an absolute path. The default driver (built-in sqlite3) is used.
Parameters:url – URL of the database to connect to.
Returns:A Database instance with open connection.
class sqlian.UnrecognizableScheme(scheme)

Raised when connect() fails to recognize a scheme.

class sqlian.Database(**kwargs)

A database connection.

This class provides a wrapper to a DB-API 2.0 Connection instance, offering additional SQL-building methods alongside with the standard API.

Keyord arguments passed to the constructor are used to create the underlying Connection instance. The implementor is responsible for converting them for the underlying DB-API 2.0 interface.

Instances of this class implement the context manager interface. The instance itself is assigned to the as expression, and the connection is closed when the context manager exists, committing done automatically if there are no exceptions.

Parameters:
  • host – Network location of the database.
  • port – Network port to access the database.
  • database – Name of the database.
  • username – Username to connect to the database.
  • password – Password to connect to the database.
  • options – Database options as a string-string mapping.
close()

Close the connection.

This method exists to conform to DB-API 2.0.

commit()

Commit any pending transaction to the database.

This method exists to conform to DB-API 2.0.

connect(dbapi, **kwargs)

Connect to the database.

This is called by create_connection() to create the connection. Keyword arguments to this method are passed directly from the class constructor.

You should override this method to convert parameter names, call connect() on the passed DB-API 2.0 interface, and return the connection instance.

connection

The underlying connection object. This property is read-only.

create_connection(**kwargs)

Creates a connection.

If you’re implementing a wrapper not conforming to DB-API 2.0, you should implement this method to override the default behavior, which depends on the API.

Keyword arguments to this method are passed directly from the class constructor.

Returns:A DB-API 2.0 Connection object.
cursor()

Return a new Cursor Object using the connection.

This method exists to conform to DB-API 2.0.

delete(*args, **kwargs)

Build and execute a DELETE statement.

execute_statement(statement_builder, args, kwargs)

Build a statement, and execute it on the connection.

This method provides implementation of statement construction and execution. Call this method like this when you implement a statement builder in custom database subclasses:

def select(self, *args, **kwargs):
    return self.execute_statement(self.engine.select, args, kwargs)

You generally don’t need to call this method directly as a user, but use one of the wrapper functions like the above instead.

Return type:RecordCollection
insert(*args, **kwargs)

Build and execute an INSERT statement.

is_open()

Whether the connection is open.

Return type:bool
rollback()

Rollback pending transaction.

This method exists to conform to DB-API 2.0. Behavior of calling this method on a database not supporting transactions is undefined.

select(*args, **kwargs)

Build and execute a SELECT statement.

update(*args, **kwargs)

Build and execute an UPDATE statement.

Connecting to Unsupported Databases

To connect to a database not yet supported by SQLian, you need to implement your own Database subclass, and optionally register it to SQLian if you want to use it with connect().

Implement a Database

Most functionalities are available by subclassing Database, but you need to declare and do a few things to interface with the underlying DB-API 2.0 module:

  • Declare dbapi2_module_name on the class. This should be a dotted import path to the DB-API 2.0 module, i.e. the thing you put after the import keyword.
  • Declare engine_class on the class. This should be Engine or its subclass. Use the basic Engine if your database support the standard SQL syntax, and you don’t need vendor-specific commands (e.g. MySQL’s REPLACE). See documenttion on engines for detailed explaination on how to build custom engine classes.
  • Override Database.connect() to instruct the class how to call connect on the underlying DB-API module.
Register the Database (Optional)

Use register() to let connect() recognize your database’s URL scheme.

sqlian.register(scheme, klass, replaces_existing=False)

Register a database type to be usable by connect().

After registering a database class, it will be instantiatable via connect() by specifying the appropriate scheme.

Parameters:
  • scheme – The scheme to register this class under.
  • klass – The database class to register.
  • replaces_existing – If True, replaces the existing if there is already a database registered under this scheme. When False, try to prevent this by raising an DuplicateScheme error.
class sqlian.DuplicateScheme(scheme, klass)

Raised when register()-ing a database under an existing scheme.

Engines

class sqlian.Engine

Engine that emits ANSI-compliant SQL.

Records

class sqlian.Record(keys, values)

A single row of data from a database.

You typically don’t need to create instances of this class, but interact with instances returned by a sqlian.standard.Database query.

__eq__(other)

Test record equality.

Two records are equal if their keys and values both match. Ordering matters.

__getattr__(key)

Access content in the record.

This works like the string indexing of __getitem__() to provide a simpler syntax under common usage. Always prefer __getitem__() (the square bracket syntax) if you want to access columns with a variable.

__getitem__(key)

Access content in the record.

Records support both numeric (sequence-like) and string (mapping-like) indexing.

Slicing is not supported. Use the values() method, which returns a slicible object.

__len__()

How many columns there are in this row.

get(key, default=None)

Get an item in the record, return default on failure.

This works similarly with the standard mapping interface, but also supports numeric indexing.

items()

Returns an iterable of 2-tuples containing keys and values.

This works similarly with the standard mapping interface.

keys()

Returns a sequence containing keys (column names) in this row.

This works similarly with the standard mapping interface.

values()

Returns a sequence containing values in this row.

This works similarly with the standard mapping interface.

class sqlian.RecordCollection(record_generator)

A sequence of records.

Record collections are backed by record generators. Results are fetched on demand. This class conforms to the standard sequence interface, and can be seamlessly treated as such.

classmethod from_cursor(cursor)

Create a RecordCollection from a DB-API 2.0 cursor.

This method automatically extract useful information for DB-API 2.0 to generate records. Discrepencies in various interfaces are generally taken care of by this method, and you should use it instead of the basic constructor when returning records for a database query.

SQL Builders

Inspirations

Few ideas in SQLian is original. Special thanks to the following projects and their contributors:

MoSQL by Mosky Liu
for the idea of building SQL from Python constructs, and coming up with how most of the function calls should look like.
Records by Kenneth Reitz
for the record API.
SQLAlchemy by Michael Bayer
for the database connector API.
DJ-Database-URL by Kenneth Reitz
for the urlparse-based database URL parsing logic.

Content Index