SQLian: handles SQL so you don’t need to¶
SQLian is an all-in-one library that “shephards” you through interaction with SQL. Unlike an ORM (like the ones come with Django, SQLAlchemy, etc.), it does not try to hide the fact you’re writing SQL, but on the other hand still:
- Frees you from handling pesky SQL syntax oddities, so you can better debug your SQL like your Python code.
- Provides a unified interface to connect to different database implementations.
- Automatic cursor handling anda better way to interact with the data returned by the SQL database.
The Basics¶
Connect to a database:
import sqlian
db = sqlian.connect('postgresql://...')
and perform a query:
rows = db.select(
'name', 'occupation',
from_='person',
where={'main_language': 'Python'},
)
Now you can access the data directly:
>>> rows[0]
<Record {"name": "Mosky", "occupation": "Pinkoi"}>
or iterate over them:
for r in rows:
print('{} works at {}'.format(r.name, r.occupation))
Interested? Read on!
Table of Contents¶
Installation Guide¶
I highly recommend you check out Pipenv by Kenneth Reitz to handle you project dependencies. With Pipenv, you can insall SQLian simply with:
$ pipenv install sqlian
The “non-modern” way to install SQLian is from the PyPI, through Pip:
$ pip install sqlian
The source code is also available at GitHub. You can download release packages directly on it, or use Pip and Git to install the in-development snapshot:
$ pip install git+https://github.com/uranusjr/sqlian.git
Or even just clone and install manually yourself:
$ git clone https://github.com/uranusjr/sqlian.git
$ cd sqlian
$ python setup.py install
Quickstart¶
SQLian is composed of three main parts:
- Databases represent database connections.
- Statement builders take native objects and convert them to a SQL command string. The built command are then passed to the associated database connection to be executed.
- Records are returned by database queries. They offer a clean, nice interface to interact with the data retrieved from the database.
Let’s do a quick walk through on them one by one.
Connecting to a database¶
SQLian uses the 12factor-inspired database URL syntax to describe a database. This syntax is compatible with popular tools, including DJ-Database-URL, SQLAlchemy, and everything that builds on top of them. Which means, like, everything?
As an example, let’s connect to a PostgreSQL database:
import sqlian
db = sqlian.connect('postgresql://user:pa55@localhost/contactbook')
SQLian has some database support built-in. Some databases require extra
dependencies to actually connect to, like psycopg2
for PostgreSQL. You
can also build your own database support, but we’ll save that discussion for
later.
The connect()
function returns a Database
instance, which
conforms to the DB-API 2.0 specification (PEP 249), so you can get to work
directly if you know your way around. But there’s a better way to do it.
Issuing commands¶
The Database
instance provides a rich set of “statement builders” that
format and execute SQL for you, and make it easier to convert native Python
objects for SQL usage.
Inserting data:
db.insert('person', values={
'name': 'Mosky',
'occupation': 'Pinkoi',
'main_language': 'Python',
})
This roughly translates to:
INSERT INTO "person" ("name", "occupation", "main_language")
VALUES ('Mosky', 'Pinkoi', 'Python')
but saves you from dealing with column and value clauses and all those
%(name)s
stuff.
You can still use column name–value sequences if you wish to:
db.insert(
'person',
columns=('name', 'occupation', 'main_language'),
values=[
('Tim', 'GilaCloud', 'Python'),
('Adam', 'Pinkoi', 'JavaScript'),
],
)
Did I mention you can insert multiple rows at one go? Yeah, you can.
It’s also easy to update data:
db.update('person', where={'name': 'Adam'}, set={'main_language': 'CSS'})
Notice the key ordering does not matter.
You’d guess how deletion works by now, so let’s add a little twist:
db.delete('person', where={'occupation !=': 'Pinkoi'})
The builders automatically parse trailing operators and do the right thing.
Handling results¶
Some statements produce data. For every such query, SQLian returns an iterable
object RecordCollection
so you can handle them.
>>> rows = db.select(sqlian.star, from_='person')
>>> rows
<RecordCollection (pending)>
Accessing the content in any way automatically resolves it:
>>> rows[0]
<Record {"name": "Mosky", "occupation": "Pinkoi", "main_language": "Python"}>
>>> rows
<RecordCollection (1+ rows, pending)>
>>> for row in rows:
... print(row)
<Record {"name": "Mosky", "occupation": "Pinkoi", "main_language": "Python"}>
<Record {"name": "Adam", "occupation": "Pinkoi", "main_language": "CSS"}>
>>> rows
<RecordCollection (2 rows)>
A Record
can be accessed like a sequence, mapping, or even object:
>>> row = rows[0]
>>> row[0]
'Mosky'
>>> row['occupation']
Pinkoi
>>> row.main_language
Python
Database Conenctions¶
SQLian provide an interface to connect to relational database backends through
the database API. The main way to do this is by passing a database URL to the
connect()
function, but it is also possible to explicitly create
Database
instances for various backends.
The Main Interface¶
-
sqlian.
connect
(url)¶ Create a database connection.
The database URL takes the form:
scheme://username:password@host:port/database
Multiple parts of the form can be omitted if not present. Common examples:
postgresql:///db
connects to the local PostgreSQL instance via Un*x sockets without authentication, to the databasedb
. Psycopg2, the default PostgreSQL driver, is used.mysql+pymysql://localhost/db
connects to thedb
database in the MySQL instance at localhost, using no authentication. The PyMySQL driver is used.sqlite:///db.sqlite3
connects to the SQLite3 file at relative pathdb.sqlite3
. The third slash is to seperate path from the host. Use four slashes if you need to specify an absolute path. The default driver (built-in sqlite3) is used.
Parameters: url – URL of the database to connect to. Returns: A Database
instance with open connection.
-
class
sqlian.
Database
(**kwargs)¶ A database connection.
This class provides a wrapper to a DB-API 2.0 Connection instance, offering additional SQL-building methods alongside with the standard API.
Keyord arguments passed to the constructor are used to create the underlying Connection instance. The implementor is responsible for converting them for the underlying DB-API 2.0 interface.
Instances of this class implement the context manager interface. The instance itself is assigned to the as expression, and the connection is closed when the context manager exists, committing done automatically if there are no exceptions.
Parameters: - host – Network location of the database.
- port – Network port to access the database.
- database – Name of the database.
- username – Username to connect to the database.
- password – Password to connect to the database.
- options – Database options as a string-string mapping.
-
close
()¶ Close the connection.
This method exists to conform to DB-API 2.0.
-
commit
()¶ Commit any pending transaction to the database.
This method exists to conform to DB-API 2.0.
-
connect
(dbapi, **kwargs)¶ Connect to the database.
This is called by
create_connection()
to create the connection. Keyword arguments to this method are passed directly from the class constructor.You should override this method to convert parameter names, call
connect()
on the passed DB-API 2.0 interface, and return the connection instance.
-
connection
¶ The underlying connection object. This property is read-only.
-
create_connection
(**kwargs)¶ Creates a connection.
If you’re implementing a wrapper not conforming to DB-API 2.0, you should implement this method to override the default behavior, which depends on the API.
Keyword arguments to this method are passed directly from the class constructor.
Returns: A DB-API 2.0 Connection object.
-
cursor
()¶ Return a new Cursor Object using the connection.
This method exists to conform to DB-API 2.0.
-
delete
(*args, **kwargs)¶ Build and execute a DELETE statement.
-
execute_statement
(statement_builder, args, kwargs)¶ Build a statement, and execute it on the connection.
This method provides implementation of statement construction and execution. Call this method like this when you implement a statement builder in custom database subclasses:
def select(self, *args, **kwargs): return self.execute_statement(self.engine.select, args, kwargs)
You generally don’t need to call this method directly as a user, but use one of the wrapper functions like the above instead.
Return type: RecordCollection
-
insert
(*args, **kwargs)¶ Build and execute an INSERT statement.
-
is_open
()¶ Whether the connection is open.
Return type: bool
-
rollback
()¶ Rollback pending transaction.
This method exists to conform to DB-API 2.0. Behavior of calling this method on a database not supporting transactions is undefined.
-
select
(*args, **kwargs)¶ Build and execute a SELECT statement.
-
update
(*args, **kwargs)¶ Build and execute an UPDATE statement.
Connecting to Unsupported Databases¶
To connect to a database not yet supported by SQLian, you need to implement
your own Database subclass, and optionally register it to SQLian if you want
to use it with connect()
.
Implement a Database¶
Most functionalities are available by subclassing Database
, but you
need to declare and do a few things to interface with the underlying DB-API
2.0 module:
- Declare dbapi2_module_name on the class. This should be a dotted import
path to the DB-API 2.0 module, i.e. the thing you put after the
import
keyword. - Declare engine_class on the class. This should be
Engine
or its subclass. Use the basicEngine
if your database support the standard SQL syntax, and you don’t need vendor-specific commands (e.g. MySQL’s REPLACE). See documenttion on engines for detailed explaination on how to build custom engine classes. - Override
Database.connect()
to instruct the class how to call connect on the underlying DB-API module.
Register the Database (Optional)¶
Use register()
to let connect()
recognize your database’s URL
scheme.
-
sqlian.
register
(scheme, klass, replaces_existing=False)¶ Register a database type to be usable by
connect()
.After registering a database class, it will be instantiatable via
connect()
by specifying the appropriate scheme.Parameters: - scheme – The scheme to register this class under.
- klass – The database class to register.
- replaces_existing – If
True
, replaces the existing if there is already a database registered under this scheme. WhenFalse
, try to prevent this by raising anDuplicateScheme
error.
-
class
sqlian.
DuplicateScheme
(scheme, klass)¶ Raised when
register()
-ing a database under an existing scheme.
Records¶
-
class
sqlian.
Record
(keys, values)¶ A single row of data from a database.
You typically don’t need to create instances of this class, but interact with instances returned by a
sqlian.standard.Database
query.-
__eq__
(other)¶ Test record equality.
Two records are equal if their keys and values both match. Ordering matters.
-
__getattr__
(key)¶ Access content in the record.
This works like the string indexing of
__getitem__()
to provide a simpler syntax under common usage. Always prefer__getitem__()
(the square bracket syntax) if you want to access columns with a variable.
-
__getitem__
(key)¶ Access content in the record.
Records support both numeric (sequence-like) and string (mapping-like) indexing.
Slicing is not supported. Use the
values()
method, which returns a slicible object.
-
__len__
()¶ How many columns there are in this row.
-
get
(key, default=None)¶ Get an item in the record, return default on failure.
This works similarly with the standard mapping interface, but also supports numeric indexing.
-
items
()¶ Returns an iterable of 2-tuples containing keys and values.
This works similarly with the standard mapping interface.
-
keys
()¶ Returns a sequence containing keys (column names) in this row.
This works similarly with the standard mapping interface.
-
values
()¶ Returns a sequence containing values in this row.
This works similarly with the standard mapping interface.
-
-
class
sqlian.
RecordCollection
(record_generator)¶ A sequence of records.
Record collections are backed by record generators. Results are fetched on demand. This class conforms to the standard sequence interface, and can be seamlessly treated as such.
-
classmethod
from_cursor
(cursor)¶ Create a
RecordCollection
from a DB-API 2.0 cursor.This method automatically extract useful information for DB-API 2.0 to generate records. Discrepencies in various interfaces are generally taken care of by this method, and you should use it instead of the basic constructor when returning records for a database query.
-
classmethod
SQL Builders¶
Inspirations¶
Few ideas in SQLian is original. Special thanks to the following projects and their contributors:
- MoSQL by Mosky Liu
- for the idea of building SQL from Python constructs, and coming up with how most of the function calls should look like.
- Records by Kenneth Reitz
- for the record API.
- SQLAlchemy by Michael Bayer
- for the database connector API.
- DJ-Database-URL by Kenneth Reitz
- for the urlparse-based database URL parsing logic.