MongoEngine User Documentation¶

MongoEngine 은 MongoDB를 이용하기 위해 파이썬으로 작성된 객체와 Document를 매핑해주는 라이브러리입니다. 간단하게 설치하고 실행할 수 있습니다.

$ pip install -U mongoengine

Tutorial: MongoEngine을 이용하여 tumblelog을 만드는 튜토리얼입니다.
User Guide: documents를 모델링 하는 것부터 파일을 저장하는 것까지 그리고 데이터를 query하는 것 등 MongoEngine에 대한 전체적인 가이드입니다.
API Reference: documents, querysets, fields의 내부에 대한 전체적인 API 문서입니다.
Upgrading: 어떻게 MongoEngine을 업그레이드 하는지에 대한 문서입니다.
Django Support: Django 내에서의 MongoEngine 사용법에 대한 문서입니다.

Community¶

MongoEngine을 사용하는 데에 있어 도움이 필요하다면 MongoEngine 사용자 google 그룹스 를 이용하거나 stackoverflow 를 이용하세요.

Contributing¶

Yes please! 우리는 언제나 추가와 발전을 위한 컨트리뷰션을 기다리고 있습니다.

MongoEngine의 소스는 GitHub 에 있습니다. 그리고 언제나 이 문서, 웹사이트, 내부구조에 대한 컨트리뷰션을 격려하고 있습니다.

컨트리뷰션은 GitHub 에서 포크를 한 다음 풀 리퀘스트를 보내주세요.

Changes¶

MongoEngine에 대한 모든 변경 사항들은 Changelog 에서, 모든 업그레이드 사항들은 Upgrading 에서 확인할 수 있습니다.

Note

항상 upgrade 문서를 읽고 실제 배포 되기 전에 읽고 테스트 해주세요! ;)

Offline Reading¶

pdf 또는 epub 에서 오프라인에서 이 문서를 다운로드할 수 있습니다.

Tutorial¶

이 튜토리얼은 Tumblelog 블로그 어플리케이션을 만들며 Mongoengine 을 설명합니다. Tumblelog은 텍스트, 이미지, 링크, 영상, 오디오 등등의 미디어 컨텐츠를 지원하는 블로그 어플리케이션 입니다. 튜토리얼을 간단하게 하기 위해 우리의 Tumblelog은 텍스트, 이미지, 링크만 지원합니다. 그리고 이 튜토리얼의 목적은 MongoEngine을 설명하는 것이기 때문에 유저인터페이스 보다는 데이터 모델링에 중점을 두고 설명할 것입니다.

Getting started¶

시작하기 전에 MongoDB가 접근 가능한 위치에서 실행되고 있어야 합니다. — 만약 로컬에서 실행되고 있다면 훨씬 수월할 것입니다. 그렇지 않다면 리모트서버에서 실행이 되고 있어야 합니다. 만약 당신의 컴퓨터에 MongoEngine이 설치가 되어있지 않다면 pip를 이용하여 다음과 같이 쉽게 설치할 수 있습니다.

$ pip install mongoengine

MongoEngine을 사용하기 전에 connect() 함수를 사용하여 mongod 의 인스턴스를 연결하는 법을 알려드리겠습니다. 만약에 로컬에서 실행되고 있다면 함수의 매개변수로 MongoDB의 이름만 들어가면 됩니다.:

from mongoengine import *

connect('tumblelog')

MongoDB를 연결할 때 수많은 옵션을 사용할 수 있습니다. 더 자세한 정보는 Connecting to MongoDB 가이드를 확인하세요.

Defining our documents¶

MongoDB는 스키마를 정의할 필요가 없는 스키마리스 데이터베이스입니다. — 필드를 추가하거나 지워도 에러가 발생하지 않습니다. 이는 데이터 모델 변경 등등 많은 것들을 편하게 해주지만 도큐먼트의 스키마를 정의하는 것은 잘못된 타입 또는 빠진 필드 등의 버그를 잡는 데에 도움을 줍니다. 그리고 ORMs 처럼 유틸리티 메소드를 정의할 수 있습니다.

Tumblelog 어플리케이션에서는 다양한 타입의 정보들을 저장해야 합니다. 먼저 post들을 각각의 유저마다 연결하기 위해 users 컬렉션이 필요합니다. 또한, 다양한 종류의 posts (예: 텍스트, 이미지, 링크)를 데이터베이스에 저장해야 합니다. Tumblelog에서 post마다 Tags 를 달면 특정 tag를 검색했을 때 글을 찾기 쉽게 해줍니다. 마지막으로 comments 를 추가하면 멋진 Tumblelog가 만들어집니다. 다른 많은 도큐먼트들과 연결되어 있는 users 를 먼저 정의해봅시다.

Users¶

관계형 데이터베이스에서 ORM을 쓰는 것 처럼 User 가 어떠한 필드를 가지고 있는지, 어떠한 타입의 데이터가 들어갈 것인지를 정의해야 합니다.:

class User(Document):
    email = StringField(required=True)
    first_name = StringField(max_length=50)
    last_name = StringField(max_length=50)

위 코드는 일반적인 ORM에서 table의 구조를 정의하는 것과 비슷해 보입니다. 하지만 가장 중요한 차이점은 스키마가 MongoDB에 전달되거나 정의되지 않는다는 것입니다. — 이것은 추후 수정 사항을 쉽게 관리하기 위해 어플리케이션 레벨에서 적용이 됩니다. User 도큐먼트는 MongoDB의 table이 아닌 collection 에 저장이 될 것입니다.

Posts, Comments and Tags¶

이제 남은 정보들을 어떻게 저장할 것인지 생각해봅시다. 만약 관계형 데이터베이스를 사용했다면 아마 posts table과 comments table 그리고 tags table을 만들었을 것입니다. 그리고 comments와 각각의 posts를 연결하기 위해 comments table의 속성에 posts table의 foreign key를 넣었을 것입니다. 그리고 post table과 tag table을 연결하기 위해 many-to-many relationship을 제공했을 것입니다. 그다음에 각각의 데이터 타입들(텍스트, 이미지, 링크)을 저장하는 문제를 해결해야 합니다. 이 문제를 해결하는 데에는 다양한 방법이 있지만, 각각의 방법마다 단점이 있어서 완벽한 해결책은 없습니다.

Posts¶

행복하게도 MongoDB는 관계형 데이터베이스가 아니기 때문에 위에 말했던 방법을 사용하지 않습니다. MongoDB의 schemaless 환경이 더 멋진 해결책을 제공해주기 때문입니다. 우리는 모든 종류의 posts를 한 collection 에 저장하고 각각의 종류의 posts를 위한 필드를 정의합니다. 만약에 나중에 비디오 posts를 추가하고 싶더라도 collection의 전체 내용을 수정할 필요가 없습니다. 단지 비디오 posts를 지원할 새로운 필드를 만들기만 하면 됩니다. 이것은 객체지향의 상속 원칙에 멋지게 알맞습니다. 우리는 Post 를 부모 클래스. TextPost, ImagePost 그리고 LinkPost 를 Post 의 자식 클래스로 생각하면 됩니다. 사실 MongoEngine은 이 것을 창조적은 모델링의 한 종류로 지원합니다. — 상속이 가능하게 설정하기 위해서는 meta 안에 있는 allow_inheritance 를 True로 설정하기만 하면 됩니다.:

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)

    meta = {'allow_inheritance': True}

class TextPost(Post):
    content = StringField()

class ImagePost(Post):
    image_path = StringField()

class LinkPost(Post):
    link_url = StringField()

ReferenceField 를 이용하여 게시글 작성자의 Reference를 저장했습니다. 이는 전통적인 ORM의 foreign key 필드와 비슷합니다. 그리고 데이터를 저장할 때 자동으로 참조가 되고 데이터를 불러올 때 역참조가 됩니다.

Tags¶

이제 Post 모델을 이해했으니 여기에 tags를 어떻게 붙일 수 있을까요? MongoDB는 자체적으로 아이템들의 리스트를 저장할 수 있습니다. 각각 포스트의 태그들의 리스트를 Post 모델에 저장할 수 있습니다. 효율성과 간편성을 위해서 태그들을 collection에 reference로 분할하여 저장하지 않고 post에 string으로 저장하겠습니다. 특별히 tags는 매우 짧기 때문에(주로 도큐먼트의 id보다도 짧습니다.) 제한하지 않아도 데이터베이스의 크기에 크게 영향을 주지 않습니다. 수정된 Post 클래스를 확인해봅시다.:

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)
    tags = ListField(StringField(max_length=30))

Post의 tags를 저장하기 위해 사용한 ListField 객체는 첫 번째 인자로 필드 객체를 받습니다. — 이것은 어떤 종류의 필드의 리스트도 저장할 수 있다는 것입니다. (리스트를 포함해서 말입니다.)

Note

:class:`Post`를 상속받았기 때문에 각각의 post 종류마다 다 수정해줄 필요가 없습니다.

Comments¶

comment는 전형적으로 한 post에 연결이 되어 있습니다. 관계형 데이터베이스에서는 post와 그 post의 comments를 보여주기 위해 데이터베이스에서 post를 검색한 후 해당 post의 댓글을 다시 query를 했습니다. 관계형 데이터베이스를 사용하지 않는 이상 굳이 comments를 연결되어 있는 posts와 따로 저장할 이유가 없습니다. MongoDB를 사용하면 comments를 post document에 embedded documents 의 리스트로 직접 저장할 수 있습니다. embedded documet는 다른 평범한 document와 다른 취급을 받지 않습니다. 단지 데이터베이스에서 자신만의 collection을 가지지 못할 뿐입니다. MngoEngine에서 embedded document의 구조를 평범한 document처럼 utility 메소드를 사용하여 정의할 수 있습니다.:

class Comment(EmbeddedDocument):
    content = StringField()
    name = StringField(max_length=120)

post document에서 comment documents의 리스트를 저장할 수 있습니다.:

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User)
    tags = ListField(StringField(max_length=30))
    comments = ListField(EmbeddedDocumentField(Comment))

Handling deletions of references¶

ReferenceField 객체는 참조되는 객체가 지워질 때의 삭제 규칙을 제어하기 위해 reverse_delete_rule 키워드를 가집니다. user가 지워질 때 모든 posts를 삭제하기 위해서 규칙을 설정해야 합니다.:

class Post(Document):
    title = StringField(max_length=120, required=True)
    author = ReferenceField(User, reverse_delete_rule=CASCADE)
    tags = ListField(StringField(max_length=30))
    comments = ListField(EmbeddedDocumentField(Comment))

더 자세한 정보는 ReferenceField 를 확인하세요.

Note

현재 Map필드와 Dict필드는 삭제된 참조에 대한 제어 방법이 지원되지 않습니다.

Adding data to our Tumblelog¶

지금까지 document가 어떤 구조로 저장될 것인지 정의했습니다. 그러면 데이터베이스에 몇 개의 documents를 추가해 봅시다! 먼저 User 객체를 만들어 봅시다.:

ross = User(email='ross@example.com', first_name='Ross', last_name='Lawley').save()

Note

user를 속성을 통해서 정의할 수도 있습니다.:

ross = User(email='ross@example.com')
ross.first_name = 'Ross'
ross.last_name = 'Lawley'
ross.save()

위에서 ross 이라는 user를 만든 것처럼 john 이라는 user도 만들어 봅시다.

users를 데이터베이스에 넣었으니 두 개의 post를 추가해봅시다.:

post1 = TextPost(title='Fun with MongoEngine', author=john)
post1.content = 'Took a look at MongoEngine today, looks pretty cool.'
post1.tags = ['mongodb', 'mongoengine']
post1.save()

post2 = LinkPost(title='MongoEngine Documentation', author=ross)
post2.link_url = 'http://docs.mongoengine.com/'
post2.tags = ['mongoengine']
post2.save()

Note

이미 save() 메소드를 사용하여 저장된 객체에서 필드를 수정했더라도 save() 메소드를 다시 호출하면 새로운 내용으로 업데이트 됩니다.

Accessing our data¶

데이터베이스에 저장된 두 개의 posts를 어떻게 보여줄 수 있을까요? 각각의 document class (i.e. Document 를 직접적이든 간접적이든 상속받은 class) 는 해당 class와 연관된 데이터베이스의 collection 내에 있는 documents에 접근할 때 사용되는 objects 속성을 가지고 있습니다. 그럼 우리 posts의 제목을 가져와봅시다.:

for post in Post.objects:
    print(post.title)

Retrieving type-specific information¶

이것은 posts의 제목을 한 줄에 하나씩 출력할 것입니다. 하지만 만약 TextPost의 특정 종류의 데이터 (link_url, content, etc.)를 접근하고자 하면 어떻게 해야 할까요? 간단하게 Post 의 자식 클래스의 objects 속성을 이용하는 방법이 있습니다.:

for post in TextPost.objects:
    print(post.content)

TextPost의 objects 속성을 사용하면 TextPost`를 사용하여 생성된 documents들만 반환됩니다. 여기에는 일반적인 규칙이 있습니다.: :class:`~mongoengine.Document`의 서브 클래스의 :attr:`objects 속성은 해당 서브 클래스 또는 해당 서브 클래스의 서브 클래스들을 통해 생성된 documents만 반환합니다.

그러면 특정 타입에 부합하는 posts만을 어떻게 보여줄 수 있을까요? 각각의 서브 클래스마다 objects 속성을 사용하는 것보다 더 좋은 방법이 있습니다. Post 에서 objects 속성을 사용하면 반환되는 값들은 Post 의 인스턴스가 아니라 Post 의 서브 클래스의 인스턴스입니다. 실제로 어떻게 작동하는지 아래 코드를 봅시다.:

for post in Post.objects:
    print(post.title)
    print('=' * len(post.title))

    if isinstance(post, TextPost):
        print(post.content)

    if isinstance(post, LinkPost):
        print('Link: {}'.format(post.link_url))

위 코드는 각각의 posts 제목을 출력하고 텍스트 post라면 post의 content를 출력하고 링크 post라면 “Link: <url>”를 출력하게 됩니다.

Searching our posts by tag¶

Document class의 objects 속성은 QuerySet 객체입니다. 이것은 데이터를 요구할 때마다 데이터베이스에 게으른 query를 합니다. 또한, 필터링하여 반환되는 값들을 줄일 수도 있습니다. 그럼 tag가 “mongodb”인 posts만 반환이 되도록 query를 바로잡아봅시다.:

for post in Post.objects(tags='mongodb'):
    print(post.title)

QuerySet 에 있는 메소드들은 반환되는 결과를 다르게 할 수 있습니다. 예를 들어, objects 속성에 있는 first() 메소드는 query에 첫 번째로 매치되는 하나의 document를 반환합니다. 집합에 관련된 함수 또한 QuerySet 객체에서 자주 사용됩니다.:

num_posts = Post.objects(tags='mongodb').count()
print('Found {} posts with tag "mongodb"'.format(num_posts))

Learning more about MongoEngine¶

만약 이 튜토리얼을 완료했다면 당신은 매우 훌륭한 출발을 했습니다! 잘하셨습니다 :)

MongoEngine을 배우는 여정의 다음 발자국은 MongoEngine과 MongoDB를 심도 있게 사용하는 법을 배울 수 있는 full user guide 입니다.

User Guide¶

Installing MongoEngine¶

To use MongoEngine, you will need to download MongoDB and ensure it is running in an accessible location. You will also need PyMongo to use MongoEngine, but if you install MongoEngine using setuptools, then the dependencies will be handled for you.

MongoEngine is available on PyPI, so you can use pip:

$ pip install mongoengine

Alternatively, if you don’t have setuptools installed, download it from PyPi and run

$ python setup.py install

To use the bleeding-edge version of MongoEngine, you can get the source from GitHub and install it as above:

$ git clone git://github.com/mongoengine/mongoengine
$ cd mongoengine
$ python setup.py install

Connecting to MongoDB¶

실행되고 있는 mongod`의 인스턴스에 연결하기 위해서는 :func:`~mongoengine.connect 함수를 사용해야 합니다. 이 함수의 첫 번째 인자는 연결할 데이터베이스의 이름입니다.:

from mongoengine import connect
connect('project1')

기본적으로 MongoEngine은 mongod`의 인스턴스가 **localhost**에서 **27017** 포트에서 실행이 되고있다고 추측합니다. 만약 MongoDB가 다른 곳에서 실행이 되고 있다면 :func:`~mongoengine.connect:: 함수의 인자로 :attr:`host`와 :attr:`port`를 넘겨주어야 합니다.:

connect('project1', host='192.168.1.35', port=12345)

만약 데이터베이스가 인증을 필요로한다면 connect():: 함수의 인자로 :attr:`username`과 :attr:`password`를 넘겨주어야 합니다.:

connect('project1', username='webapp', password='pwd123')

URI 형식의 연결 또한 지원합니다. – 단지 connect():: 함수의 host 인자에 URI를 넘겨주면 됩니다.:

connect('project1', host='mongodb://localhost/database_name')

Note

URI 스트링에 있는 데이터베이스, 유저네임 그리고 패스워드는 connect():의 인자들을 덮어씁니다.:

connect(
    db='test',
    username='user',
    password='12345',
    host='mongodb://admin:qwerty@localhost/production'
)

위 코드는 admin``이라는 유저네임과 ``qwerty 패스워드로 production 데이터베이스에 접속하게 됩니다.

To connect to a running instance of mongod, use the connect() function. The first argument is the name of the database to connect to:

from mongoengine import connect
connect('project1')

By default, MongoEngine assumes that the mongod instance is running on localhost on port 27017. If MongoDB is running elsewhere, you should provide the host and port arguments to connect():

connect('project1', host='192.168.1.35', port=12345)

If the database requires authentication, username and password arguments should be provided:

connect('project1', username='webapp', password='pwd123')

URI style connections are also supported – just supply the URI as the host to connect():

connect('project1', host='mongodb://localhost/database_name')

Note

Database, username and password from URI string overrides corresponding parameters in connect():

connect(
    db='test',
    username='user',
    password='12345',
    host='mongodb://admin:qwerty@localhost/production'
)

will establish connection to production database using admin username and qwerty password.

Replica Sets¶

MongoEngine supports connecting to replica sets:

from mongoengine import connect

# Regular connect
connect('dbname', replicaset='rs-name')

# MongoDB URI-style connect
connect(host='mongodb://localhost/dbname?replicaSet=rs-name')

Read preferences are supported through the connection or via individual queries by passing the read_preference

Bar.objects().read_preference(ReadPreference.PRIMARY)
Bar.objects(read_preference=ReadPreference.PRIMARY)

Multiple Databases¶

To use multiple databases you can use connect() and provide an alias name for the connection - if no alias is provided then “default” is used.

In the background this uses register_connection() to store the data and you can register all aliases up front if required.

Individual documents can also support multiple databases by providing a db_alias in their meta data. This allows DBRef objects to point across databases and collections. Below is an example schema, using 3 different databases to store data:

class User(Document):
    name = StringField()

    meta = {'db_alias': 'user-db'}

class Book(Document):
    name = StringField()

    meta = {'db_alias': 'book-db'}

class AuthorBooks(Document):
    author = ReferenceField(User)
    book = ReferenceField(Book)

    meta = {'db_alias': 'users-books-db'}

Context Managers¶

Sometimes you may want to switch the database or collection to query against. For example, archiving older data into a separate database for performance reasons or writing functions that dynamically choose collections to write a document to.

Switch Database¶

The switch_db context manager allows you to change the database alias for a given class allowing quick and easy access to the same User document across databases:

from mongoengine.context_managers import switch_db

class User(Document):
    name = StringField()

    meta = {'db_alias': 'user-db'}

with switch_db(User, 'archive-user-db') as User:
    User(name='Ross').save()  # Saves the 'archive-user-db'

Switch Collection¶

The switch_collection context manager allows you to change the collection for a given class allowing quick and easy access to the same Group document across collection:

from mongoengine.context_managers import switch_collection

class Group(Document):
    name = StringField()

Group(name='test').save()  # Saves in the default db

with switch_collection(Group, 'group2000') as Group:
    Group(name='hello Group 2000 collection!').save()  # Saves in group2000 collection

Note

Make sure any aliases have been registered with register_connection() or connect() before using the context manager.

Defining documents¶

In MongoDB, a document is roughly equivalent to a row in an RDBMS. When working with relational databases, rows are stored in tables, which have a strict schema that the rows follow. MongoDB stores documents in collections rather than tables — the principal difference is that no schema is enforced at a database level.

Defining a document’s schema¶

MongoEngine allows you to define schemata for documents as this helps to reduce coding errors, and allows for utility methods to be defined on fields which may be present.

To define a schema for a document, create a class that inherits from Document. Fields are specified by adding field objects as class attributes to the document class:

from mongoengine import *
import datetime

class Page(Document):
    title = StringField(max_length=200, required=True)
    date_modified = DateTimeField(default=datetime.datetime.utcnow)

As BSON (the binary format for storing data in mongodb) is order dependent, documents are serialized based on their field order.

Dynamic document schemas¶

One of the benefits of MongoDB is dynamic schemas for a collection, whilst data should be planned and organised (after all explicit is better than implicit!) there are scenarios where having dynamic / expando style documents is desirable.

DynamicDocument documents work in the same way as Document but any data / attributes set to them will also be saved

from mongoengine import *

class Page(DynamicDocument):
    title = StringField(max_length=200, required=True)

# Create a new page and add tags
>>> page = Page(title='Using MongoEngine')
>>> page.tags = ['mongodb', 'mongoengine']
>>> page.save()

>>> Page.objects(tags='mongoengine').count()
>>> 1

Note

There is one caveat on Dynamic Documents: fields cannot start with _

Dynamic fields are stored in creation order after any declared fields.

Fields¶

By default, fields are not required. To make a field mandatory, set the required keyword argument of a field to True. Fields also may have validation constraints available (such as max_length in the example above). Fields may also take default values, which will be used if a value is not provided. Default values may optionally be a callable, which will be called to retrieve the value (such as in the above example). The field types available are as follows:

Field arguments¶

Each field type can be customized by keyword arguments. The following keyword arguments can be set on all fields:

db_field (Default: None)

The MongoDB field name.

required (Default: False)

If set to True and the field is not set on the document instance, a ValidationError will be raised when the document is validated.

default (Default: None)

A value to use when no value is set for this field.

The definition of default parameters follow the general rules on Python, which means that some care should be taken when dealing with default mutable objects (like in ListField or DictField):

class ExampleFirst(Document):
    # Default an empty list
    values = ListField(IntField(), default=list)

class ExampleSecond(Document):
    # Default a set of values
    values = ListField(IntField(), default=lambda: [1,2,3])

class ExampleDangerous(Document):
    # This can make an .append call to  add values to the default (and all the following objects),
    # instead to just an object
    values = ListField(IntField(), default=[1,2,3])

Note

Unsetting a field with a default value will revert back to the default.

unique (Default: False)

When True, no documents in the collection will have the same value for this field.

unique_with (Default: None)

A field name (or list of field names) that when taken together with this field, will not have two documents in the collection with the same value.

primary_key (Default: False)

When True, use this field as a primary key for the collection. DictField and EmbeddedDocuments both support being the primary key for a document.

Note

If set, this field is also accessible through the pk field.

choices (Default: None)

An iterable (e.g. list, tuple or set) of choices to which the value of this field should be limited.

Can be either be a nested tuples of value (stored in mongo) and a human readable key

SIZE = (('S', 'Small'),
        ('M', 'Medium'),
        ('L', 'Large'),
        ('XL', 'Extra Large'),
        ('XXL', 'Extra Extra Large'))


class Shirt(Document):
    size = StringField(max_length=3, choices=SIZE)

Or a flat iterable just containing values

SIZE = ('S', 'M', 'L', 'XL', 'XXL')

class Shirt(Document):
    size = StringField(max_length=3, choices=SIZE)

**kwargs (Optional)

You can supply additional metadata as arbitrary additional keyword arguments. You can not override existing attributes, however. Common choices include help_text and verbose_name, commonly used by form and widget libraries.

List fields¶

MongoDB allows storing lists of items. To add a list of items to a Document, use the ListField field type. ListField takes another field object as its first argument, which specifies which type elements may be stored within the list:

class Page(Document):
    tags = ListField(StringField(max_length=50))

Embedded documents¶

MongoDB has the ability to embed documents within other documents. Schemata may be defined for these embedded documents, just as they may be for regular documents. To create an embedded document, just define a document as usual, but inherit from EmbeddedDocument rather than Document:

class Comment(EmbeddedDocument):
    content = StringField()

To embed the document within another document, use the EmbeddedDocumentField field type, providing the embedded document class as the first argument:

class Page(Document):
    comments = ListField(EmbeddedDocumentField(Comment))

comment1 = Comment(content='Good work!')
comment2 = Comment(content='Nice article!')
page = Page(comments=[comment1, comment2])

Dictionary Fields¶

Often, an embedded document may be used instead of a dictionary – generally embedded documents are recommended as dictionaries don’t support validation or custom field types. However, sometimes you will not know the structure of what you want to store; in this situation a DictField is appropriate:

class SurveyResponse(Document):
    date = DateTimeField()
    user = ReferenceField(User)
    answers = DictField()

survey_response = SurveyResponse(date=datetime.utcnow(), user=request.user)
response_form = ResponseForm(request.POST)
survey_response.answers = response_form.cleaned_data()
survey_response.save()

Dictionaries can store complex data, other dictionaries, lists, references to other objects, so are the most flexible field type available.

Reference fields¶

References may be stored to other documents in the database using the ReferenceField. Pass in another document class as the first argument to the constructor, then simply assign document objects to the field:

class User(Document):
    name = StringField()

class Page(Document):
    content = StringField()
    author = ReferenceField(User)

john = User(name="John Smith")
john.save()

post = Page(content="Test Page")
post.author = john
post.save()

The User object is automatically turned into a reference behind the scenes, and dereferenced when the Page object is retrieved.

To add a ReferenceField that references the document being defined, use the string 'self' in place of the document class as the argument to ReferenceField‘s constructor. To reference a document that has not yet been defined, use the name of the undefined document as the constructor’s argument:

class Employee(Document):
    name = StringField()
    boss = ReferenceField('self')
    profile_page = ReferenceField('ProfilePage')

class ProfilePage(Document):
    content = StringField()

One to Many with ListFields¶

If you are implementing a one to many relationship via a list of references, then the references are stored as DBRefs and to query you need to pass an instance of the object to the query:

class User(Document):
    name = StringField()

class Page(Document):
    content = StringField()
    authors = ListField(ReferenceField(User))

bob = User(name="Bob Jones").save()
john = User(name="John Smith").save()

Page(content="Test Page", authors=[bob, john]).save()
Page(content="Another Page", authors=[john]).save()

# Find all pages Bob authored
Page.objects(authors__in=[bob])

# Find all pages that both Bob and John have authored
Page.objects(authors__all=[bob, john])

# Remove Bob from the authors for a page.
Page.objects(id='...').update_one(pull__authors=bob)

# Add John to the authors for a page.
Page.objects(id='...').update_one(push__authors=john)

Dealing with deletion of referred documents¶

By default, MongoDB doesn’t check the integrity of your data, so deleting documents that other documents still hold references to will lead to consistency issues. Mongoengine’s ReferenceField adds some functionality to safeguard against these kinds of database integrity problems, providing each reference with a delete rule specification. A delete rule is specified by supplying the reverse_delete_rule attributes on the ReferenceField definition, like this:

class ProfilePage(Document):
    ...
    employee = ReferenceField('Employee', reverse_delete_rule=mongoengine.CASCADE)

The declaration in this example means that when an Employee object is removed, the ProfilePage that references that employee is removed as well. If a whole batch of employees is removed, all profile pages that are linked are removed as well.

Its value can take any of the following constants:

mongoengine.DO_NOTHING: This is the default and won’t do anything. Deletes are fast, but may cause database inconsistency or dangling references.
mongoengine.DENY: Deletion is denied if there still exist references to the object being deleted.
mongoengine.NULLIFY: Any object’s fields still referring to the object being deleted are removed (using MongoDB’s “unset” operation), effectively nullifying the relationship.
mongoengine.CASCADE: Any object containing fields that are referring to the object being deleted are deleted first.
mongoengine.PULL: Removes the reference to the object (using MongoDB’s “pull” operation) from any object’s fields of ListField (ReferenceField).

Warning

A safety note on setting up these delete rules! Since the delete rules are not recorded on the database level by MongoDB itself, but instead at runtime, in-memory, by the MongoEngine module, it is of the upmost importance that the module that declares the relationship is loaded BEFORE the delete is invoked.

If, for example, the Employee object lives in the payroll app, and the ProfilePage in the people app, it is extremely important that the people app is loaded before any employee is removed, because otherwise, MongoEngine could never know this relationship exists.

In Django, be sure to put all apps that have such delete rule declarations in their models.py in the INSTALLED_APPS tuple.

Generic reference fields¶

A second kind of reference field also exists, GenericReferenceField. This allows you to reference any kind of Document, and hence doesn’t take a Document subclass as a constructor argument:

class Link(Document):
    url = StringField()

class Post(Document):
    title = StringField()

class Bookmark(Document):
    bookmark_object = GenericReferenceField()

link = Link(url='http://hmarr.com/mongoengine/')
link.save()

post = Post(title='Using MongoEngine')
post.save()

Bookmark(bookmark_object=link).save()
Bookmark(bookmark_object=post).save()

Note

Using GenericReferenceFields is slightly less efficient than the standard ReferenceFields, so if you will only be referencing one document type, prefer the standard ReferenceField.

Uniqueness constraints¶

MongoEngine allows you to specify that a field should be unique across a collection by providing unique=True to a Field‘s constructor. If you try to save a document that has the same value for a unique field as a document that is already in the database, a NotUniqueError will be raised. You may also specify multi-field uniqueness constraints by using unique_with, which may be either a single field name, or a list or tuple of field names:

class User(Document):
    username = StringField(unique=True)
    first_name = StringField()
    last_name = StringField(unique_with='first_name')

Skipping Document validation on save¶

You can also skip the whole document validation process by setting validate=False when calling the save() method:

class Recipient(Document):
    name = StringField()
    email = EmailField()

recipient = Recipient(name='admin', email='root@localhost')
recipient.save()               # will raise a ValidationError while
recipient.save(validate=False) # won't

Document collections¶

Document classes that inherit directly from Document will have their own collection in the database. The name of the collection is by default the name of the class, converted to lowercase (so in the example above, the collection would be called page). If you need to change the name of the collection (e.g. to use MongoEngine with an existing database), then create a class dictionary attribute called meta on your document, and set collection to the name of the collection that you want your document class to use:

class Page(Document):
    title = StringField(max_length=200, required=True)
    meta = {'collection': 'cmsPage'}

Capped collections¶

A Document may use a Capped Collection by specifying max_documents and max_size in the meta dictionary. max_documents is the maximum number of documents that is allowed to be stored in the collection, and max_size is the maximum size of the collection in bytes. max_size is rounded up to the next multiple of 256 by MongoDB internally and mongoengine before. Use also a multiple of 256 to avoid confusions. If max_size is not specified and max_documents is, max_size defaults to 10485760 bytes (10MB). The following example shows a Log document that will be limited to 1000 entries and 2MB of disk space:

class Log(Document):
    ip_address = StringField()
    meta = {'max_documents': 1000, 'max_size': 2000000}

Indexes¶

You can specify indexes on collections to make querying faster. This is done by creating a list of index specifications called indexes in the meta dictionary, where an index specification may either be a single field name, a tuple containing multiple field names, or a dictionary containing a full index definition.

A direction may be specified on fields by prefixing the field name with a + (for ascending) or a - sign (for descending). Note that direction only matters on multi-field indexes. Text indexes may be specified by prefixing the field name with a $. Hashed indexes may be specified by prefixing the field name with a #:

class Page(Document):
    category = IntField()
    title = StringField()
    rating = StringField()
    created = DateTimeField()
    meta = {
        'indexes': [
            'title',
            '$title',  # text index
            '#title',  # hashed index
            ('title', '-rating'),
            ('category', '_cls'),
            {
                'fields': ['created'],
                'expireAfterSeconds': 3600
            }
        ]
    }

If a dictionary is passed then the following options are available:

fields (Default: None): The fields to index. Specified in the same format as described above.
cls (Default: True): If you have polymorphic models that inherit and have allow_inheritance turned on, you can configure whether the index should have the _cls field added automatically to the start of the index.
sparse (Default: False): Whether the index should be sparse.
unique (Default: False): Whether the index should be unique.
expireAfterSeconds (Optional): Allows you to automatically expire data from a collection by setting the time in seconds to expire the a field.

Note

Inheritance adds extra fields indices see: Document inheritance.

Global index default options¶

There are a few top level defaults for all indexes that can be set:

class Page(Document):
    title = StringField()
    rating = StringField()
    meta = {
        'index_options': {},
        'index_background': True,
        'index_drop_dups': True,
        'index_cls': False
    }

index_options (Optional): Set any default index options - see the full options list
index_background (Optional): Set the default value for if an index should be indexed in the background
index_cls (Optional): A way to turn off a specific index for _cls.
index_drop_dups (Optional): Set the default value for if an index should drop duplicates

Note

Since MongoDB 3.0 drop_dups is not supported anymore. Raises a Warning and has no effect

Compound Indexes and Indexing sub documents¶

Compound indexes can be created by adding the Embedded field or dictionary field name to the index definition.

Sometimes its more efficient to index parts of Embedded / dictionary fields, in this case use ‘dot’ notation to identify the value to index eg: rank.title

Geospatial indexes¶

The best geo index for mongodb is the new “2dsphere”, which has an improved spherical model and provides better performance and more options when querying. The following fields will explicitly add a “2dsphere” index:

PointField

LineStringField

PolygonField

MultiPointField

MultiLineStringField

MultiPolygonField

As “2dsphere” indexes can be part of a compound index, you may not want the automatic index but would prefer a compound index. In this example we turn off auto indexing and explicitly declare a compound index on location and datetime:

class Log(Document):
    location = PointField(auto_index=False)
    datetime = DateTimeField()

    meta = {
        'indexes': [[("location", "2dsphere"), ("datetime", 1)]]
    }

Pre MongoDB 2.4 Geo¶

Note

For MongoDB < 2.4 this is still current, however the new 2dsphere index is a big improvement over the previous 2D model - so upgrading is advised.

Geospatial indexes will be automatically created for all GeoPointFields

It is also possible to explicitly define geospatial indexes. This is useful if you need to define a geospatial index on a subfield of a DictField or a custom field that contains a point. To create a geospatial index you must prefix the field with the * sign.

class Place(Document):
    location = DictField()
    meta = {
        'indexes': [
            '*location.point',
        ],
    }

Time To Live indexes¶

A special index type that allows you to automatically expire data from a collection after a given period. See the official ttl documentation for more information. A common usecase might be session data:

class Session(Document):
    created = DateTimeField(default=datetime.utcnow)
    meta = {
        'indexes': [
            {'fields': ['created'], 'expireAfterSeconds': 3600}
        ]
    }

Warning

TTL indexes happen on the MongoDB server and not in the application code, therefore no signals will be fired on document deletion. If you need signals to be fired on deletion, then you must handle the deletion of Documents in your application code.

Comparing Indexes¶

Use mongoengine.Document.compare_indexes() to compare actual indexes in the database to those that your document definitions define. This is useful for maintenance purposes and ensuring you have the correct indexes for your schema.

Ordering¶

A default ordering can be specified for your QuerySet using the ordering attribute of meta. Ordering will be applied when the QuerySet is created, and can be overridden by subsequent calls to order_by().

from datetime import datetime

class BlogPost(Document):
    title = StringField()
    published_date = DateTimeField()

    meta = {
        'ordering': ['-published_date']
    }

blog_post_1 = BlogPost(title="Blog Post #1")
blog_post_1.published_date = datetime(2010, 1, 5, 0, 0 ,0)

blog_post_2 = BlogPost(title="Blog Post #2")
blog_post_2.published_date = datetime(2010, 1, 6, 0, 0 ,0)

blog_post_3 = BlogPost(title="Blog Post #3")
blog_post_3.published_date = datetime(2010, 1, 7, 0, 0 ,0)

blog_post_1.save()
blog_post_2.save()
blog_post_3.save()

# get the "first" BlogPost using default ordering
# from BlogPost.meta.ordering
latest_post = BlogPost.objects.first()
assert latest_post.title == "Blog Post #3"

# override default ordering, order BlogPosts by "published_date"
first_post = BlogPost.objects.order_by("+published_date").first()
assert first_post.title == "Blog Post #1"

Shard keys¶

If your collection is sharded, then you need to specify the shard key as a tuple, using the shard_key attribute of meta. This ensures that the shard key is sent with the query when calling the save() or update() method on an existing Document instance:

class LogEntry(Document):
    machine = StringField()
    app = StringField()
    timestamp = DateTimeField()
    data = StringField()

    meta = {
        'shard_key': ('machine', 'timestamp',)
    }

Document inheritance¶

To create a specialised type of a Document you have defined, you may subclass it and add any extra fields or methods you may need. As this is new class is not a direct subclass of Document, it will not be stored in its own collection; it will use the same collection as its superclass uses. This allows for more convenient and efficient retrieval of related documents – all you need do is set allow_inheritance to True in the meta data for a document.:

# Stored in a collection named 'page'
class Page(Document):
    title = StringField(max_length=200, required=True)

    meta = {'allow_inheritance': True}

# Also stored in the collection named 'page'
class DatedPage(Page):
    date = DateTimeField()

Note

From 0.8 onwards allow_inheritance defaults to False, meaning you must set it to True to use inheritance.

Working with existing data¶

As MongoEngine no longer defaults to needing _cls, you can quickly and easily get working with existing data. Just define the document to match the expected schema in your database

# Will work with data in an existing collection named 'cmsPage'
class Page(Document):
    title = StringField(max_length=200, required=True)
    meta = {
        'collection': 'cmsPage'
    }

If you have wildly varying schemas then using a DynamicDocument might be more appropriate, instead of defining all possible field types.

If you use Document and the database contains data that isn’t defined then that data will be stored in the document._data dictionary.

Abstract classes¶

If you want to add some extra functionality to a group of Document classes but you don’t need or want the overhead of inheritance you can use the abstract attribute of meta. This won’t turn on Document inheritance but will allow you to keep your code DRY:

class BaseDocument(Document):
    meta = {
        'abstract': True,
    }
    def check_permissions(self):
        ...

class User(BaseDocument):
   ...

Now the User class will have access to the inherited check_permissions method and won’t store any of the extra _cls information.

Documents instances¶

To create a new document object, create an instance of the relevant document class, providing values for its fields as constructor keyword arguments. You may provide values for any of the fields on the document:

>>> page = Page(title="Test Page")
>>> page.title
'Test Page'

You may also assign values to the document’s fields using standard object attribute syntax:

>>> page.title = "Example Page"
>>> page.title
'Example Page'

Saving and deleting documents¶

MongoEngine tracks changes to documents to provide efficient saving. To save the document to the database, call the save() method. If the document does not exist in the database, it will be created. If it does already exist, then any changes will be updated atomically. For example:

>>> page = Page(title="Test Page")
>>> page.save()  # Performs an insert
>>> page.title = "My Page"
>>> page.save()  # Performs an atomic set on the title field.

Note

Changes to documents are tracked and on the whole perform set operations.

list_field.push(0) — sets the resulting list
del(list_field) — unsets whole list

With lists its preferable to use Doc.update(push__list_field=0) as this stops the whole list being updated — stopping any race conditions.

Pre save data validation and cleaning¶

MongoEngine allows you to create custom cleaning rules for your documents when calling save(). By providing a custom clean() method you can do any pre validation / data cleaning.

This might be useful if you want to ensure a default value based on other document values for example:

class Essay(Document):
    status = StringField(choices=('Published', 'Draft'), required=True)
    pub_date = DateTimeField()

    def clean(self):
        """Ensures that only published essays have a `pub_date` and
        automatically sets the pub_date if published and not set"""
        if self.status == 'Draft' and self.pub_date is not None:
            msg = 'Draft entries should not have a publication date.'
            raise ValidationError(msg)
        # Set the pub_date for published items if not set.
        if self.status == 'Published' and self.pub_date is None:
            self.pub_date = datetime.now()

Note

Cleaning is only called if validation is turned on and when calling save().

Cascading Saves¶

If your document contains ReferenceField or GenericReferenceField objects, then by default the save() method will not save any changes to those objects. If you want all references to be saved also, noting each save is a separate query, then passing cascade as True to the save method will cascade any saves.

Deleting documents¶

To delete a document, call the delete() method. Note that this will only work if the document exists in the database and has a valid id.

Document IDs¶

Each document in the database has a unique id. This may be accessed through the id attribute on Document objects. Usually, the id will be generated automatically by the database server when the object is save, meaning that you may only access the id field once a document has been saved:

>>> page = Page(title="Test Page")
>>> page.id
>>> page.save()
>>> page.id
ObjectId('123456789abcdef000000000')

Alternatively, you may define one of your own fields to be the document’s “primary key” by providing primary_key=True as a keyword argument to a field’s constructor. Under the hood, MongoEngine will use this field as the id; in fact id is actually aliased to your primary key field so you may still use id to access the primary key if you want:

>>> class User(Document):
...     email = StringField(primary_key=True)
...     name = StringField()
...
>>> bob = User(email='bob@example.com', name='Bob')
>>> bob.save()
>>> bob.id == bob.email == 'bob@example.com'
True

You can also access the document’s “primary key” using the pk field, it’s an alias to id:

>>> page = Page(title="Another Test Page")
>>> page.save()
>>> page.id == page.pk
True

Note

If you define your own primary key field, the field implicitly becomes required, so a ValidationError will be thrown if you don’t provide it.

Querying the database¶

Document classes have an objects attribute, which is used for accessing the objects in the database associated with the class. The objects attribute is actually a QuerySetManager, which creates and returns a new QuerySet object on access. The QuerySet object may be iterated over to fetch documents from the database:

# Prints out the names of all the users in the database
for user in User.objects:
    print user.name

Note

As of MongoEngine 0.8 the querysets utilise a local cache. So iterating it multiple times will only cause a single query. If this is not the desired behaviour you can call no_cache (version 0.8.3+) to return a non-caching queryset.

Filtering queries¶

The query may be filtered by calling the QuerySet object with field lookup keyword arguments. The keys in the keyword arguments correspond to fields on the Document you are querying:

# This will return a QuerySet that will only iterate over users whose
# 'country' field is set to 'uk'
uk_users = User.objects(country='uk')

Fields on embedded documents may also be referred to using field lookup syntax by using a double-underscore in place of the dot in object attribute access syntax:

# This will return a QuerySet that will only iterate over pages that have
# been written by a user whose 'country' field is set to 'uk'
uk_pages = Page.objects(author__country='uk')

Note

(version 0.9.1+) if your field name is like mongodb operator name (for example type, lte, lt...) and you want to place it at the end of lookup keyword mongoengine automatically prepend $ to it. To avoid this use __ at the end of your lookup keyword. For example if your field name is type and you want to query by this field you must use .objects(user__type__="admin") instead of .objects(user__type="admin")

Query operators¶

Operators other than equality may also be used in queries — just attach the operator name to a key with a double-underscore:

# Only find users whose age is 18 or less
young_users = Users.objects(age__lte=18)

Available operators are as follows:

ne – not equal to
lt – less than
lte – less than or equal to
gt – greater than
gte – greater than or equal to
not – negate a standard check, may be used before other operators (e.g. Q(age__not__mod=5))
in – value is in list (a list of values should be provided)
nin – value is not in list (a list of values should be provided)
mod – value % x == y, where x and y are two provided values
all – every item in list of values provided is in array
size – the size of the array is
exists – value for field exists

String queries¶

The following operators are available as shortcuts to querying with regular expressions:

exact – string field exactly matches value
iexact – string field exactly matches value (case insensitive)
contains – string field contains value
icontains – string field contains value (case insensitive)
startswith – string field starts with value
istartswith – string field starts with value (case insensitive)
endswith – string field ends with value
iendswith – string field ends with value (case insensitive)
match – performs an $elemMatch so you can match an entire document within an array

Geo queries¶

There are a few special operators for performing geographical queries. The following were added in MongoEngine 0.8 for PointField, LineStringField and PolygonField:

geo_within – check if a geometry is within a polygon. For ease of use it accepts either a geojson geometry or just the polygon coordinates eg:

loc.objects(point__geo_within=[[[40, 5], [40, 6], [41, 6], [40, 5]]])
loc.objects(point__geo_within={"type": "Polygon",
                         "coordinates": [[[40, 5], [40, 6], [41, 6], [40, 5]]]})

geo_within_box – simplified geo_within searching with a box eg:

loc.objects(point__geo_within_box=[(-125.0, 35.0), (-100.0, 40.0)])
loc.objects(point__geo_within_box=[<bottom left coordinates>, <upper right coordinates>])

geo_within_polygon – simplified geo_within searching within a simple polygon eg:

loc.objects(point__geo_within_polygon=[[40, 5], [40, 6], [41, 6], [40, 5]])
loc.objects(point__geo_within_polygon=[ [ <x1> , <y1> ] ,
                                        [ <x2> , <y2> ] ,
                                        [ <x3> , <y3> ] ])

geo_within_center – simplified geo_within the flat circle radius of a point eg:

loc.objects(point__geo_within_center=[(-125.0, 35.0), 1])
loc.objects(point__geo_within_center=[ [ <x>, <y> ] , <radius> ])

geo_within_sphere – simplified geo_within the spherical circle radius of a point eg:

loc.objects(point__geo_within_sphere=[(-125.0, 35.0), 1])
loc.objects(point__geo_within_sphere=[ [ <x>, <y> ] , <radius> ])

geo_intersects – selects all locations that intersect with a geometry eg:

# Inferred from provided points lists:
loc.objects(poly__geo_intersects=[40, 6])
loc.objects(poly__geo_intersects=[[40, 5], [40, 6]])
loc.objects(poly__geo_intersects=[[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]])

# With geoJson style objects
loc.objects(poly__geo_intersects={"type": "Point", "coordinates": [40, 6]})
loc.objects(poly__geo_intersects={"type": "LineString",
                                  "coordinates": [[40, 5], [40, 6]]})
loc.objects(poly__geo_intersects={"type": "Polygon",
                                  "coordinates": [[[40, 5], [40, 6], [41, 6], [41, 5], [40, 5]]]})

near – find all the locations near a given point:

loc.objects(point__near=[40, 5])
loc.objects(point__near={"type": "Point", "coordinates": [40, 5]})

You can also set the maximum and/or the minimum distance in meters as well:

loc.objects(point__near=[40, 5], point__max_distance=1000)
loc.objects(point__near=[40, 5], point__min_distance=100)

The older 2D indexes are still supported with the GeoPointField:

within_distance – provide a list containing a point and a maximum distance (e.g. [(41.342, -87.653), 5])
within_spherical_distance – same as above but using the spherical geo model (e.g. [(41.342, -87.653), 5/earth_radius])
near – order the documents by how close they are to a given point
near_sphere – Same as above but using the spherical geo model
within_box – filter documents to those within a given bounding box (e.g. [(35.0, -125.0), (40.0, -100.0)])
within_polygon – filter documents to those within a given polygon (e.g. [(41.91,-87.69), (41.92,-87.68), (41.91,-87.65), (41.89,-87.65)]).

Note

Requires Mongo Server 2.0
max_distance – can be added to your location queries to set a maximum distance.
min_distance – can be added to your location queries to set a minimum distance.

Querying lists¶

On most fields, this syntax will look up documents where the field specified matches the given value exactly, but when the field refers to a ListField, a single item may be provided, in which case lists that contain that item will be matched:

class Page(Document):
    tags = ListField(StringField())

# This will match all pages that have the word 'coding' as an item in the
# 'tags' list
Page.objects(tags='coding')

It is possible to query by position in a list by using a numerical value as a query operator. So if you wanted to find all pages whose first tag was db, you could use the following query:

Page.objects(tags__0='db')

If you only want to fetch part of a list eg: you want to paginate a list, then the slice operator is required:

# comments - skip 5, limit 10
Page.objects.fields(slice__comments=[5, 10])

For updating documents, if you don’t know the position in a list, you can use the $ positional operator

Post.objects(comments__by="joe").update(**{'inc__comments__$__votes': 1})

However, this doesn’t map well to the syntax so you can also use a capital S instead

Post.objects(comments__by="joe").update(inc__comments__S__votes=1)

Note

Due to Mongo, currently the $ operator only applies to the first matched item in the query.

Raw queries¶

It is possible to provide a raw PyMongo query as a query parameter, which will be integrated directly into the query. This is done using the __raw__ keyword argument:

Page.objects(__raw__={'tags': 'coding'})

New in version 0.4.

Limiting and skipping results¶

Just as with traditional ORMs, you may limit the number of results returned or skip a number or results in you query. limit() and skip() and methods are available on QuerySet objects, but the array-slicing syntax is preferred for achieving this:

# Only the first 5 people
users = User.objects[:5]

# All except for the first 5 people
users = User.objects[5:]

# 5 users, starting from the 11th user found
users = User.objects[10:15]

You may also index the query to retrieve a single result. If an item at that index does not exists, an IndexError will be raised. A shortcut for retrieving the first result and returning None if no result exists is provided (first()):

>>> # Make sure there are no users
>>> User.drop_collection()
>>> User.objects[0]
IndexError: list index out of range
>>> User.objects.first() == None
True
>>> User(name='Test User').save()
>>> User.objects[0] == User.objects.first()
True

Retrieving unique results¶

To retrieve a result that should be unique in the collection, use get(). This will raise DoesNotExist if no document matches the query, and MultipleObjectsReturned if more than one document matched the query. These exceptions are merged into your document definitions eg: MyDoc.DoesNotExist

A variation of this method, get_or_create() existed, but it was unsafe. It could not be made safe, because there are no transactions in mongoDB. Other approaches should be investigated, to ensure you don’t accidentally duplicate data when using something similar to this method. Therefore it was deprecated in 0.8 and removed in 0.10.

Default Document queries¶

By default, the objects objects attribute on a document returns a QuerySet that doesn’t filter the collection – it returns all objects. This may be changed by defining a method on a document that modifies a queryset. The method should accept two arguments – doc_cls and queryset. The first argument is the Document class that the method is defined on (in this sense, the method is more like a classmethod() than a regular method), and the second argument is the initial queryset. The method needs to be decorated with queryset_manager() in order for it to be recognised.

class BlogPost(Document):
    title = StringField()
    date = DateTimeField()

    @queryset_manager
    def objects(doc_cls, queryset):
        # This may actually also be done by defining a default ordering for
        # the document, but this illustrates the use of manager methods
        return queryset.order_by('-date')

You don’t need to call your method objects – you may define as many custom manager methods as you like:

class BlogPost(Document):
    title = StringField()
    published = BooleanField()

    @queryset_manager
    def live_posts(doc_cls, queryset):
        return queryset.filter(published=True)

BlogPost(title='test1', published=False).save()
BlogPost(title='test2', published=True).save()
assert len(BlogPost.objects) == 2
assert len(BlogPost.live_posts()) == 1

Custom QuerySets¶

Should you want to add custom methods for interacting with or filtering documents, extending the QuerySet class may be the way to go. To use a custom QuerySet class on a document, set queryset_class to the custom class in a Document‘s meta dictionary:

class AwesomerQuerySet(QuerySet):

    def get_awesome(self):
        return self.filter(awesome=True)

class Page(Document):
    meta = {'queryset_class': AwesomerQuerySet}

# To call:
Page.objects.get_awesome()

New in version 0.4.

Aggregation¶

MongoDB provides some aggregation methods out of the box, but there are not as many as you typically get with an RDBMS. MongoEngine provides a wrapper around the built-in methods and provides some of its own, which are implemented as Javascript code that is executed on the database server.

Counting results¶

Just as with limiting and skipping results, there is a method on a QuerySet object – count():

num_users = User.objects.count()

You could technically use len(User.objects) to get the same result, but it would be significantly slower than count(). When you execute a server-side count query, you let MongoDB do the heavy lifting and you receive a single integer over the wire. Meanwhile, len() retrieves all the results, places them in a local cache, and finally counts them. If we compare the performance of the two operations, len() is much slower than count().

Further aggregation¶

You may sum over the values of a specific field on documents using sum():

yearly_expense = Employee.objects.sum('salary')

Note

If the field isn’t present on a document, that document will be ignored from the sum.

To get the average (mean) of a field on a collection of documents, use average():

mean_age = User.objects.average('age')

As MongoDB provides native lists, MongoEngine provides a helper method to get a dictionary of the frequencies of items in lists across an entire collection – item_frequencies(). An example of its use would be generating “tag-clouds”:

class Article(Document):
    tag = ListField(StringField())

# After adding some tagged articles...
tag_freqs = Article.objects.item_frequencies('tag', normalize=True)

from operator import itemgetter
top_tags = sorted(tag_freqs.items(), key=itemgetter(1), reverse=True)[:10]

Query efficiency and performance¶

There are a couple of methods to improve efficiency when querying, reducing the information returned by the query or efficient dereferencing .

Retrieving a subset of fields¶

Sometimes a subset of fields on a Document is required, and for efficiency only these should be retrieved from the database. This issue is especially important for MongoDB, as fields may often be extremely large (e.g. a ListField of EmbeddedDocuments, which represent the comments on a blog post. To select only a subset of fields, use only(), specifying the fields you want to retrieve as its arguments. Note that if fields that are not downloaded are accessed, their default value (or None if no default value is provided) will be given:

>>> class Film(Document):
...     title = StringField()
...     year = IntField()
...     rating = IntField(default=3)
...
>>> Film(title='The Shawshank Redemption', year=1994, rating=5).save()
>>> f = Film.objects.only('title').first()
>>> f.title
'The Shawshank Redemption'
>>> f.year   # None
>>> f.rating # default value
3

Note

The exclude() is the opposite of only() if you want to exclude a field.

If you later need the missing fields, just call reload() on your document.

Turning off dereferencing¶

Sometimes for performance reasons you don’t want to automatically dereference data. To turn off dereferencing of the results of a query use no_dereference() on the queryset like so:

post = Post.objects.no_dereference().first()
assert(isinstance(post.author, ObjectId))

You can also turn off all dereferencing for a fixed period by using the no_dereference context manager:

with no_dereference(Post) as Post:
    post = Post.objects.first()
    assert(isinstance(post.author, ObjectId))

# Outside the context manager dereferencing occurs.
assert(isinstance(post.author, User))

Advanced queries¶

Sometimes calling a QuerySet object with keyword arguments can’t fully express the query you want to use – for example if you need to combine a number of constraints using and and or. This is made possible in MongoEngine through the Q class. A Q object represents part of a query, and can be initialised using the same keyword-argument syntax you use to query documents. To build a complex query, you may combine Q objects using the & (and) and | (or) operators. To use a Q object, pass it in as the first positional argument to Document.objects when you filter it by calling it with keyword arguments:

from mongoengine.queryset.visitor import Q

# Get published posts
Post.objects(Q(published=True) | Q(publish_date__lte=datetime.now()))

# Get top posts
Post.objects((Q(featured=True) & Q(hits__gte=1000)) | Q(hits__gte=5000))

Warning

You have to use bitwise operators. You cannot use or, and to combine queries as Q(a=a) or Q(b=b) is not the same as Q(a=a) | Q(b=b). As Q(a=a) equates to true Q(a=a) or Q(b=b) is the same as Q(a=a).

Atomic updates¶

Documents may be updated atomically by using the update_one(), update() and modify() methods on a QuerySet or modify() and save() (with save_condition argument) on a Document. There are several different “modifiers” that you may use with these methods:

set – set a particular value
unset – delete a particular value (since MongoDB v1.3)
inc – increment a value by a given amount
dec – decrement a value by a given amount
push – append a value to a list
push_all – append several values to a list
pop – remove the first or last element of a list depending on the value
pull – remove a value from a list
pull_all – remove several values from a list
add_to_set – add value to a list only if its not in the list already

The syntax for atomic updates is similar to the querying syntax, but the modifier comes before the field, not after it:

>>> post = BlogPost(title='Test', page_views=0, tags=['database'])
>>> post.save()
>>> BlogPost.objects(id=post.id).update_one(inc__page_views=1)
>>> post.reload()  # the document has been changed, so we need to reload it
>>> post.page_views
1
>>> BlogPost.objects(id=post.id).update_one(set__title='Example Post')
>>> post.reload()
>>> post.title
'Example Post'
>>> BlogPost.objects(id=post.id).update_one(push__tags='nosql')
>>> post.reload()
>>> post.tags
['database', 'nosql']

Note

If no modifier operator is specified the default will be $set. So the following sentences are identical:

>>> BlogPost.objects(id=post.id).update(title='Example Post')
>>> BlogPost.objects(id=post.id).update(set__title='Example Post')

Note

In version 0.5 the save() runs atomic updates on changed documents by tracking changes to that document.

The positional operator allows you to update list items without knowing the index position, therefore making the update a single atomic operation. As we cannot use the $ syntax in keyword arguments it has been mapped to S:

>>> post = BlogPost(title='Test', page_views=0, tags=['database', 'mongo'])
>>> post.save()
>>> BlogPost.objects(id=post.id, tags='mongo').update(set__tags__S='mongodb')
>>> post.reload()
>>> post.tags
['database', 'mongodb']

From MongoDB version 2.6, push operator supports $position value which allows to push values with index.

>>> post = BlogPost(title="Test", tags=["mongo"])
>>> post.save()
>>> post.update(push__tags__0=["database", "code"])
>>> post.reload()
>>> post.tags
['database', 'code', 'mongo']

Note

Currently only top level lists are handled, future versions of mongodb / pymongo plan to support nested positional operators. See The $ positional operator.

Server-side javascript execution¶

Javascript functions may be written and sent to the server for execution. The result of this is the return value of the Javascript function. This functionality is accessed through the exec_js() method on QuerySet() objects. Pass in a string containing a Javascript function as the first argument.

The remaining positional arguments are names of fields that will be passed into you Javascript function as its arguments. This allows functions to be written that may be executed on any field in a collection (e.g. the sum() method, which accepts the name of the field to sum over as its argument). Note that field names passed in in this manner are automatically translated to the names used on the database (set using the name keyword argument to a field constructor).

Keyword arguments to exec_js() are combined into an object called options, which is available in the Javascript function. This may be used for defining specific parameters for your function.

Some variables are made available in the scope of the Javascript function:

collection – the name of the collection that corresponds to the Document class that is being used; this should be used to get the Collection object from db in Javascript code
query – the query that has been generated by the QuerySet object; this may be passed into the find() method on a Collection object in the Javascript function
options – an object containing the keyword arguments passed into exec_js()

The following example demonstrates the intended usage of exec_js() by defining a function that sums over a field on a document (this functionality is already available through sum() but is shown here for sake of example):

def sum_field(document, field_name, include_negatives=True):
    code = """
    function(sumField) {
        var total = 0.0;
        db[collection].find(query).forEach(function(doc) {
            var val = doc[sumField];
            if (val >= 0.0 || options.includeNegatives) {
                total += val;
            }
        });
        return total;
    }
    """
    options = {'includeNegatives': include_negatives}
    return document.objects.exec_js(code, field_name, **options)

As fields in MongoEngine may use different names in the database (set using the db_field keyword argument to a Field constructor), a mechanism exists for replacing MongoEngine field names with the database field names in Javascript code. When accessing a field on a collection object, use square-bracket notation, and prefix the MongoEngine field name with a tilde. The field name that follows the tilde will be translated to the name used in the database. Note that when referring to fields on embedded documents, the name of the EmbeddedDocumentField, followed by a dot, should be used before the name of the field on the embedded document. The following example shows how the substitutions are made:

class Comment(EmbeddedDocument):
    content = StringField(db_field='body')

class BlogPost(Document):
    title = StringField(db_field='doctitle')
    comments = ListField(EmbeddedDocumentField(Comment), name='cs')

# Returns a list of dictionaries. Each dictionary contains a value named
# "document", which corresponds to the "title" field on a BlogPost, and
# "comment", which corresponds to an individual comment. The substitutions
# made are shown in the comments.
BlogPost.objects.exec_js("""
function() {
    var comments = [];
    db[collection].find(query).forEach(function(doc) {
        // doc[~comments] -> doc["cs"]
        var docComments = doc[~comments];

        for (var i = 0; i < docComments.length; i++) {
            // doc[~comments][i] -> doc["cs"][i]
            var comment = doc[~comments][i];

            comments.push({
                // doc[~title] -> doc["doctitle"]
                'document': doc[~title],

                // comment[~comments.content] -> comment["body"]
                'comment': comment[~comments.content]
            });
        }
    });
    return comments;
}
""")

GridFS¶

New in version 0.4.

Writing¶

GridFS support comes in the form of the FileField field object. This field acts as a file-like object and provides a couple of different ways of inserting and retrieving data. Arbitrary metadata such as content type can also be stored alongside the files. In the following example, a document is created to store details about animals, including a photo:

class Animal(Document):
    genus = StringField()
    family = StringField()
    photo = FileField()

marmot = Animal(genus='Marmota', family='Sciuridae')

marmot_photo = open('marmot.jpg', 'rb')
marmot.photo.put(marmot_photo, content_type = 'image/jpeg')
marmot.save()

Retrieval¶

So using the FileField is just like using any other field. The file can also be retrieved just as easily:

marmot = Animal.objects(genus='Marmota').first()
photo = marmot.photo.read()
content_type = marmot.photo.content_type

Streaming¶

Streaming data into a FileField is achieved in a slightly different manner. First, a new file must be created by calling the new_file() method. Data can then be written using write():

marmot.photo.new_file()
marmot.photo.write('some_image_data')
marmot.photo.write('some_more_image_data')
marmot.photo.close()

marmot.save()

Deletion¶

Deleting stored files is achieved with the delete() method:

marmot.photo.delete()

Warning

The FileField in a Document actually only stores the ID of a file in a separate GridFS collection. This means that deleting a document with a defined FileField does not actually delete the file. You must be careful to delete any files in a Document as above before deleting the Document itself.

Replacing files¶

Files can be replaced with the replace() method. This works just like the put() method so even metadata can (and should) be replaced:

another_marmot = open('another_marmot.png', 'rb')
marmot.photo.replace(another_marmot, content_type='image/png')

Signals¶

New in version 0.5.

Note

Signal support is provided by the excellent blinker library. If you wish to enable signal support this library must be installed, though it is not required for MongoEngine to function.

Overview¶

Signals are found within the mongoengine.signals module. Unless specified signals receive no additional arguments beyond the sender class and document instance. Post-signals are only called if there were no exceptions raised during the processing of their related function.

Available signals include:

pre_init: Called during the creation of a new Document or EmbeddedDocument instance, after the constructor arguments have been collected but before any additional processing has been done to them. (I.e. assignment of default values.) Handlers for this signal are passed the dictionary of arguments using the values keyword argument and may modify this dictionary prior to returning.
post_init: Called after all processing of a new Document or EmbeddedDocument instance has been completed.
pre_save: Called within save() prior to performing any actions.
pre_save_post_validation: Called within save() after validation has taken place but before saving.
post_save: Called within save() after most actions (validation, insert/update, and cascades, but not clearing dirty flags) have completed successfully. Passed the additional boolean keyword argument created to indicate if the save was an insert or an update.
pre_delete: Called within delete() prior to attempting the delete operation.
post_delete: Called within delete() upon successful deletion of the record.
pre_bulk_insert: Called after validation of the documents to insert, but prior to any data being written. In this case, the document argument is replaced by a documents argument representing the list of documents being inserted.
post_bulk_insert: Called after a successful bulk insert operation. As per pre_bulk_insert, the document argument is omitted and replaced with a documents argument. An additional boolean argument, loaded, identifies the contents of documents as either Document instances when True or simply a list of primary key values for the inserted records if False.

Attaching Events¶

After writing a handler function like the following:

import logging
from datetime import datetime

from mongoengine import *
from mongoengine import signals

def update_modified(sender, document):
    document.modified = datetime.utcnow()

You attach the event handler to your Document or EmbeddedDocument subclass:

class Record(Document):
    modified = DateTimeField()

signals.pre_save.connect(update_modified)

While this is not the most elaborate document model, it does demonstrate the concepts involved. As a more complete demonstration you can also define your handlers within your subclass:

class Author(Document):
    name = StringField()

    @classmethod
    def pre_save(cls, sender, document, **kwargs):
        logging.debug("Pre Save: %s" % document.name)

    @classmethod
    def post_save(cls, sender, document, **kwargs):
        logging.debug("Post Save: %s" % document.name)
        if 'created' in kwargs:
            if kwargs['created']:
                logging.debug("Created")
            else:
                logging.debug("Updated")

signals.pre_save.connect(Author.pre_save, sender=Author)
signals.post_save.connect(Author.post_save, sender=Author)

Finally, you can also use this small decorator to quickly create a number of signals and attach them to your Document or EmbeddedDocument subclasses as class decorators:

def handler(event):
    """Signal decorator to allow use of callback functions as class decorators."""

    def decorator(fn):
        def apply(cls):
            event.connect(fn, sender=cls)
            return cls

        fn.apply = apply
        return fn

    return decorator

Using the first example of updating a modification time the code is now much cleaner looking while still allowing manual execution of the callback:

@handler(signals.pre_save)
def update_modified(sender, document):
    document.modified = datetime.utcnow()

@update_modified.apply
class Record(Document):
    modified = DateTimeField()

Text Search¶

After MongoDB 2.4 version, supports search documents by text indexes.

Defining a Document with text index¶

Use the $ prefix to set a text index, Look the declaration:

class News(Document):
    title = StringField()
    content = StringField()
    is_active = BooleanField()

    meta = {'indexes': [
        {'fields': ['$title', "$content"],
         'default_language': 'english',
         'weights': {'title': 10, 'content': 2}
        }
    ]}

Querying¶

Saving a document:

News(title="Using mongodb text search",
     content="Testing text search").save()

News(title="MongoEngine 0.9 released",
     content="Various improvements").save()

Next, start a text search using QuerySet.search_text method:

document = News.objects.search_text('testing').first()
document.title # may be: "Using mongodb text search"

document = News.objects.search_text('released').first()
document.title # may be: "MongoEngine 0.9 released"

Ordering by text score¶

objects = News.objects.search('mongo').order_by('$text_score')

Use mongomock for testing¶

mongomock is a package to do just what the name implies, mocking a mongo database.

To use with mongoengine, simply specify mongomock when connecting with mongoengine:

connect('mongoenginetest', host='mongomock://localhost')
conn = get_connection()

or with an alias:

connect('mongoenginetest', host='mongomock://localhost', alias='testdb')
conn = get_connection('testdb')

API Reference¶

Connecting¶

mongoengine.connect(db=None, alias='default', **kwargs)¶

Connect to the database specified by the ‘db’ argument.

Connection settings may be provided here as well if the database is not running on the default port on localhost. If authentication is needed, provide username and password arguments as well.

Multiple databases are supported by using aliases. Provide a separate alias to connect to a different instance of mongod.

See the docstring for register_connection for more details about all supported kwargs.

Changed in version 0.6: - added multiple database support.

mongoengine.register_connection(alias, db=None, name=None, host=None, port=None, read_preference=Primary(), username=None, password=None, authentication_source=None, authentication_mechanism=None, **kwargs)¶

Add a connection.

Parameters:

alias – the name that will be used to refer to this connection throughout MongoEngine
name – the name of the specific database to use
db – the name of the database to use, for compatibility with connect
host – the host name of the mongod instance to connect to
port – the port that the mongod instance is running on
read_preference – The read preference for the collection ** Added pymongo 2.1
username – username to authenticate with
password – password to authenticate with
authentication_source – database to authenticate against
authentication_mechanism – database authentication mechanisms. By default, use SCRAM-SHA-1 with MongoDB 3.0 and later, MONGODB-CR (MongoDB Challenge Response protocol) for older servers.
is_mock – explicitly use mongomock for this connection (can also be done by using mongomock:// as db host prefix)
kwargs – ad-hoc parameters to be passed into the pymongo driver, for example maxpoolsize, tz_aware, etc. See the documentation for pymongo’s MongoClient for a full list.

Changed in version 0.10.6: - added mongomock support

Documents¶

class mongoengine.Document(*args, **values)¶

The base class used for defining the structure and properties of collections of documents stored in MongoDB. Inherit from this class, and add fields as class attributes to define a document’s structure. Individual documents may then be created by making instances of the Document subclass.

By default, the MongoDB collection used to store documents created using a Document subclass will be the name of the subclass converted to lowercase. A different collection may be specified by providing collection to the meta dictionary in the class definition.

A Document subclass may be itself subclassed, to create a specialised version of the document that will be stored in the same collection. To facilitate this behaviour a _cls field is added to documents (hidden though the MongoEngine interface). To enable this behaviourset allow_inheritance to True in the meta dictionary.

A Document may use a Capped Collection by specifying max_documents and max_size in the meta dictionary. max_documents is the maximum number of documents that is allowed to be stored in the collection, and max_size is the maximum size of the collection in bytes. max_size is rounded up to the next multiple of 256 by MongoDB internally and mongoengine before. Use also a multiple of 256 to avoid confusions. If max_size is not specified and max_documents is, max_size defaults to 10485760 bytes (10MB).

Indexes may be created by specifying indexes in the meta dictionary. The value should be a list of field names or tuples of field names. Index direction may be specified by prefixing the field names with a + or - sign.

Automatic index creation can be disabled by specifying auto_create_index in the meta dictionary. If this is set to False then indexes will not be created by MongoEngine. This is useful in production systems where index creation is performed as part of a deployment system.

By default, _cls will be added to the start of every index (that doesn’t contain a list) if allow_inheritance is True. This can be disabled by either setting cls to False on the specific index or by setting index_cls to False on the meta dictionary for the document.

By default, any extra attribute existing in stored data but not declared in your model will raise a FieldDoesNotExist error. This can be disabled by setting strict to False in the meta dictionary.

Initialise a document or embedded document

Parameters:	__auto_convert – Try and will cast python objects to Object types values – A dictionary of values for the document

objects¶: A QuerySet object that is created lazily on access.

cascade_save(**kwargs)¶: Recursively save any references and generic references on the document.

classmethod compare_indexes()¶: Compares the indexes defined in MongoEngine with the ones existing in the database. Returns any missing/extra indexes.

classmethod create_index(keys, background=False, **kwargs)¶

Creates the given indexes if required.

Parameters:	keys – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering background – Allows index creation in the background

delete(signal_kwargs=None, **write_concern)¶

Delete the Document from the database. This will only take effect if the document has been previously saved.

Parm signal_kwargs:
	(optional) kwargs dictionary to be passed to the signal calls.
Parameters:	write_concern – Extra keyword arguments are passed down which will be used as options for the resultant `getLastError` command. For example, `save(..., write_concern={w: 2, fsync: True}, ...)` will wait until at least two servers have recorded the write and will force an fsync on the primary server.

Changed in version 0.10.7: Add signal_kwargs argument

classmethod drop_collection()¶

Drops the entire collection associated with this Document type from the database.

Raises OperationError if the document has no collection set (i.g. if it is abstract)

Changed in version 0.10.7: OperationError exception raised if no collection available

classmethod ensure_index(key_or_list, drop_dups=False, background=False, **kwargs)¶

Ensure that the given indexes are in place. Deprecated in favour of create_index.

Parameters:	key_or_list – a single index key or a list of index keys (to construct a multi-field index); keys may be prefixed with a + or a - to determine the index ordering background – Allows index creation in the background drop_dups – Was removed/ignored with MongoDB >2.7.5. The value will be removed if PyMongo3+ is used

classmethod ensure_indexes()¶

Checks the document meta data and ensures all the indexes exist.

Global defaults can be set in the meta - see Defining documents

Note

You can disable automatic index creation by setting auto_create_index to False in the documents meta data

classmethod list_indexes()¶: Lists all of the indexes that should be created for given collection. It includes all the indexes from super- and sub-classes.

modify(query=None, **update)¶

Perform an atomic update of the document in the database and reload the document object using updated version.

Returns True if the document has been updated or False if the document in the database doesn’t match the query.

Note

All unsaved changes that have been made to the document are rejected if the method returns True.

Parameters:	query – the update will be performed only if the document in the database matches the query update – Django-style update keyword arguments

my_metaclass¶: alias of TopLevelDocumentMetaclass

pk¶: Get the primary key.

classmethod register_delete_rule(document_cls, field_name, rule)¶: This method registers the delete rules to apply when removing this object.

reload(*fields, **kwargs)¶

Reloads all attributes from the database.

Parameters:	fields – (optional) args list of fields to reload max_depth – (optional) depth of dereferencing to follow

New in version 0.1.2.

Changed in version 0.6: Now chainable

Changed in version 0.9: Can provide specific fields to reload

save(force_insert=False, validate=True, clean=True, write_concern=None, cascade=None, cascade_kwargs=None, _refs=None, save_condition=None, signal_kwargs=None, **kwargs)¶

Save the Document to the database. If the document already exists, it will be updated, otherwise it will be created.

Parameters:

force_insert – only try to create a new document, don’t allow updates of existing documents.
validate – validates the document; set to False to skip.
clean – call the document clean method, requires validate to be True.
write_concern – Extra keyword arguments are passed down to save() OR insert() which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
cascade – Sets the flag for cascading saves. You can set a default by setting “cascade” in the document __meta__
cascade_kwargs – (optional) kwargs dictionary to be passed throw to cascading saves. Implies cascade=True.
_refs – A list of processed references used in cascading saves
save_condition – only perform save if matching record in db satisfies condition(s) (e.g. version number). Raises OperationError if the conditions are not satisfied
signal_kwargs – (optional) kwargs dictionary to be passed to the signal calls.

Changed in version 0.5: In existing documents it only saves changed fields using set / unset. Saves are cascaded and any DBRef objects that have changes are saved as well.

Changed in version 0.6: Added cascading saves

Changed in version 0.8: Cascade saves are optional and default to False. If you want fine grain control then you can turn off using document meta[‘cascade’] = True. Also you can pass different kwargs to the cascade save using cascade_kwargs which overwrites the existing kwargs with custom values.

Changed in version 0.8.5: Optional save_condition that only overwrites existing documents if the condition is satisfied in the current db record.

Changed in version 0.10: OperationError exception raised if save_condition fails.

Changed in version 0.10.1: :class: save_condition failure now raises a SaveConditionError

Changed in version 0.10.7: Add signal_kwargs argument

select_related(max_depth=1)¶: Handles dereferencing of DBRef objects to a maximum depth in order to cut down the number queries to mongodb.

New in version 0.5.

switch_collection(collection_name, keep_created=True)¶

Temporarily switch the collection for a document instance.

Only really useful for archiving off data and calling save():

user = User.objects.get(id=user_id)
user.switch_collection('old-users')
user.save()

Parameters:	collection_name (str) – The database alias to use for saving the document keep_created (bool) – keep self._created value after switching collection, else is reset to True

Context Managers¶

class mongoengine.context_managers.switch_db(cls, db_alias)¶

switch_db alias context manager.

Example

# Register connections
register_connection('default', 'mongoenginetest')
register_connection('testdb-1', 'mongoenginetest2')

class Group(Document):
    name = StringField()

Group(name='test').save()  # Saves in the default db

with switch_db(Group, 'testdb-1') as Group:
    Group(name='hello testdb!').save()  # Saves in testdb-1

Construct the switch_db context manager

Parameters:	cls – the class to change the registered db db_alias – the name of the specific database to use

class mongoengine.context_managers.switch_collection(cls, collection_name)¶

switch_collection alias context manager.

Example

class Group(Document):
    name = StringField()

Group(name='test').save()  # Saves in the default db

with switch_collection(Group, 'group1') as Group:
    Group(name='hello testdb!').save()  # Saves in group1 collection

Construct the switch_collection context manager.

Parameters:	cls – the class to change the registered db collection_name – the name of the collection to use

class mongoengine.context_managers.no_dereference(cls)¶

no_dereference context manager.

Turns off all dereferencing in Documents for the duration of the context manager:

with no_dereference(Group) as Group:
    Group.objects.find()

Construct the no_dereference context manager.

Parameters:	cls – the class to turn dereferencing off on

class mongoengine.context_managers.query_counter¶

Query_counter context manager to get the number of queries.

Construct the query_counter.

Querying¶

class mongoengine.queryset.QuerySet(document, collection)¶

The default queryset, that builds queries and handles a set of results returned from a query.

Wraps a MongoDB cursor, providing Document objects as the results.

__call__(q_obj=None, class_check=True, read_preference=None, **query)¶

Filter the selected documents by calling the QuerySet with a query.

Parameters:	q_obj – a `Q` object to be used in the query; the `QuerySet` is filtered multiple times with different `Q` objects, only the last one will be used class_check – If set to False bypass class name check when querying collection read_preference – if set, overrides connection-level read_preference from ReplicaSetConnection. query – Django-style query keyword arguments

aggregate(*pipeline, **kwargs)¶: Perform a aggregate function based in your queryset params :param pipeline: list of aggregation commands, see: http://docs.mongodb.org/manual/core/aggregation-pipeline/

New in version 0.9.

all()¶: Returns all documents.

all_fields()¶

Include all fields. Reset all previously calls of .only() or .exclude().

post = BlogPost.objects.exclude('comments').all_fields()

New in version 0.5.

as_pymongo()¶

Instead of returning Document instances, return raw values from pymongo.

This method is particularly useful if you don’t need dereferencing and care primarily about the speed of data retrieval.

average(field)¶

Average over the values of the specified field.

Parameters:	field – the field to average over; use dot notation to refer to embedded document fields

batch_size(size)¶

Limit the number of documents returned in a single batch (each batch requires a round trip to the server).

See http://api.mongodb.com/python/current/api/pymongo/cursor.html#pymongo.cursor.Cursor.batch_size for details.

Parameters:	size – desired size of each batch.

clone()¶: Create a copy of the current queryset.

comment(text)¶

Add a comment to the query.

See https://docs.mongodb.com/manual/reference/method/cursor.comment/#cursor.comment for details.

count(with_limit_and_skip=False)¶

Count the selected elements in the query.

Parameters:	(optional) (with_limit_and_skip) – take any `limit()` or `skip()` that has been applied to this cursor into account when getting the count

create(**kwargs)¶: Create new object. Returns the saved object instance.

New in version 0.4.

delete(write_concern=None, _from_doc_delete=False, cascade_refs=None)¶

Delete the documents matched by the query.

Parameters:

write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
_from_doc_delete – True when called from document delete therefore signals will have been triggered so don’t loop.

:returns number of deleted documents

distinct(field)¶

Return a list of distinct values for a given field.

Parameters:	field – the field to select distinct values from

Note

This is a command and won’t take ordering or limit into account.

New in version 0.4.

Changed in version 0.5: - Fixed handling references

Changed in version 0.6: - Improved db_field refrence handling

ensure_index(**kwargs)¶: Deprecated use Document.ensure_index()

exclude(*fields)¶

Opposite to .only(), exclude some document’s fields.

post = BlogPost.objects(...).exclude('comments')

Note

exclude() is chainable and will perform a union :: So with the following it will exclude both: title and author.name:

post = BlogPost.objects.exclude('title').exclude('author.name')

all_fields() will reset any field filters.

Parameters:	fields – fields to exclude

New in version 0.5.

exec_js(code, *fields, **options)¶

Execute a Javascript function on the server. A list of fields may be provided, which will be translated to their correct names and supplied as the arguments to the function. A few extra variables are added to the function’s scope: collection, which is the name of the collection in use; query, which is an object representing the current query; and options, which is an object containing any options specified as keyword arguments.

As fields in MongoEngine may use different names in the database (set using the db_field keyword argument to a Field constructor), a mechanism exists for replacing MongoEngine field names with the database field names in Javascript code. When accessing a field, use square-bracket notation, and prefix the MongoEngine field name with a tilde (~).

Parameters:	code – a string of Javascript code to execute fields – fields that you will be using in your function, which will be passed in to your function as arguments options – options that you want available to the function (accessed in Javascript through the `options` object)

explain(format=False)¶

Return an explain plan record for the QuerySet‘s cursor.

Parameters:	format – format the plan before returning it

fields(_only_called=False, **kwargs)¶

Manipulate how you load this document’s fields. Used by .only() and .exclude() to manipulate which fields to retrieve. If called directly, use a set of kwargs similar to the MongoDB projection document. For example:

Include only a subset of fields:

posts = BlogPost.objects(...).fields(author=1, title=1)

Exclude a specific field:

posts = BlogPost.objects(...).fields(comments=0)

To retrieve a subrange of array elements:

posts = BlogPost.objects(...).fields(slice__comments=5)

Parameters:	kwargs – A set of keyword arguments identifying what to include, exclude, or slice.

New in version 0.5.

filter(*q_objs, **query)¶: An alias of __call__()

first()¶: Retrieve the first object matching the query.

from_json(json_data)¶: Converts json data to unsaved objects

get(*q_objs, **query)¶: Retrieve the the matching object raising MultipleObjectsReturned or DocumentName.MultipleObjectsReturned exception if multiple results and DoesNotExist or DocumentName.DoesNotExist if no results are found.

New in version 0.3.

hint(index=None)¶

Added ‘hint’ support, telling Mongo the proper index to use for the query.

Judicious use of hints can greatly improve query performance. When doing a query on multiple fields (at least one of which is indexed) pass the indexed field as a hint to the query.

Hinting will not do anything if the corresponding index does not exist. The last hint applied to this cursor takes precedence over all others.

New in version 0.5.

in_bulk(object_ids)¶

Retrieve a set of documents by their ids.

Parameters:	object_ids – a list or tuple of `ObjectId`s
Return type:	dict of ObjectIds as keys and collection-specific Document subclasses as values.

New in version 0.3.

insert(doc_or_docs, load_bulk=True, write_concern=None, signal_kwargs=None)¶

bulk insert documents

Parameters:

doc_or_docs – a document or list of documents to be inserted
(optional) (load_bulk) – If True returns the list of document instances
write_concern – Extra keyword arguments are passed down to insert() which will be used as options for the resultant getLastError command. For example, insert(..., {w: 2, fsync: True}) will wait until at least two servers have recorded the write and will force an fsync on each server being written to.

Parm signal_kwargs:

(optional) kwargs dictionary to be passed to the signal calls.

By default returns document instances, set load_bulk to False to return just ObjectIds

New in version 0.5.

Changed in version 0.10.7: Add signal_kwargs argument

item_frequencies(field, normalize=False, map_reduce=True)¶

Returns a dictionary of all items present in a field across the whole queried set of documents, and their corresponding frequency. This is useful for generating tag clouds, or searching documents.

Note

Can only do direct simple mappings and cannot map across ReferenceField or GenericReferenceField for more complex counting a manual map reduce call is required.

If the field is a ListField, the items within each list will be counted individually.

Parameters:	field – the field to use normalize – normalize the results so they add to 1.0 map_reduce – Use map_reduce over exec_js

Changed in version 0.5: defaults to map_reduce and can handle embedded document lookups

limit(n)¶

Limit the number of returned documents to n. This may also be achieved using array-slicing syntax (e.g. User.objects[:5]).

Parameters:	n – the maximum number of objects to return

map_reduce(map_f, reduce_f, output, finalize_f=None, limit=None, scope=None)¶

Perform a map/reduce query using the current query spec and ordering. While map_reduce respects QuerySet chaining, it must be the last call made, as it does not return a maleable QuerySet.

See the test_map_reduce() and test_map_advanced() tests in tests.queryset.QuerySetTest for usage examples.

Parameters:

map_f – map function, as Code or string
reduce_f – reduce function, as Code or string
output – output collection name, if set to ‘inline’ will try to use inline_map_reduce This can also be a dictionary containing output options see: http://docs.mongodb.org/manual/reference/command/mapReduce/#dbcmd.mapReduce
finalize_f – finalize function, an optional function that performs any post-reduction processing.
scope – values to insert into map/reduce global scope. Optional.
limit – number of objects from current query to provide to map/reduce method

Returns an iterator yielding MapReduceDocument.

Note

Map/Reduce changed in server version >= 1.7.4. The PyMongo map_reduce() helper requires PyMongo version >= 1.11.

Changed in version 0.5: - removed keep_temp keyword argument, which was only relevant for MongoDB server versions older than 1.7.4

New in version 0.3.

max_time_ms(ms)¶

Wait ms milliseconds before killing the query on the server

Parameters:	ms – the number of milliseconds before killing the query on the server

modify(upsert=False, full_response=False, remove=False, new=False, **update)¶

Update and return the updated document.

Returns either the document before or after modification based on new parameter. If no documents match the query and upsert is false, returns None. If upserting and new is false, returns None.

If the full_response parameter is True, the return value will be the entire response object from the server, including the ‘ok’ and ‘lastErrorObject’ fields, rather than just the modified document. This is useful mainly because the ‘lastErrorObject’ document holds information about the command’s execution.

Parameters:	upsert – insert if document doesn’t exist (default `False`) full_response – return the entire response object from the server (default `False`, not available for PyMongo 3+) remove – remove rather than updating (default `False`) new – return updated rather than original document (default `False`) update – Django-style update keyword arguments

New in version 0.9.

next()¶: Wrap the result in a Document object.

no_cache()¶: Convert to a non-caching queryset

New in version 0.8.3: Convert to non caching queryset

no_dereference()¶: Turn off any dereferencing for the results of this queryset.

no_sub_classes()¶: Only return instances of this document and not any inherited documents

none()¶: Helper that just returns a list

only(*fields)¶

Load only a subset of this document’s fields.

post = BlogPost.objects(...).only('title', 'author.name')

Note

only() is chainable and will perform a union :: So with the following it will fetch both: title and author.name:

post = BlogPost.objects.only('title').only('author.name')

all_fields() will reset any field filters.

Parameters:	fields – fields to include

New in version 0.3.

Changed in version 0.5: - Added subfield support

order_by(*keys)¶

Order the QuerySet by the keys. The order may be specified by prepending each of the keys by a + or a -. Ascending order is assumed. If no keys are passed, existing ordering is cleared instead.

Parameters:	keys – fields to order the query results by; keys may be prefixed with + or - to determine the ordering direction

read_preference(read_preference)¶

Change the read_preference when querying.

Parameters:	read_preference – override ReplicaSetConnection-level preference.

rewind()¶: Rewind the cursor to its unevaluated state.

New in version 0.3.

scalar(*fields)¶

Instead of returning Document instances, return either a specific value or a tuple of values in order.

Can be used along with no_dereference() to turn off dereferencing.

Note

This effects all results and can be unset by calling scalar without arguments. Calls only automatically.

Parameters:	fields – One or more fields to return instead of a Document.

search_text(text, language=None)¶

Start a text search, using text indexes. Require: MongoDB server version 2.6+.

Parameters:	language – The language that determines the list of stop words for the search and the rules for the stemmer and tokenizer. If not specified, the search uses the default language of the index. For supported languages, see Text Search Languages <http://docs.mongodb.org/manual/reference/text-search-languages/#text-search-languages>.

select_related(max_depth=1)¶: Handles dereferencing of DBRef objects or ObjectId a maximum depth in order to cut down the number queries to mongodb.

New in version 0.5.

skip(n)¶

Skip n documents before returning the results. This may also be achieved using array-slicing syntax (e.g. User.objects[5:]).

Parameters:	n – the number of objects to skip before returning results

slave_okay(enabled)¶

Enable or disable the slave_okay when querying.

Parameters:	enabled – whether or not the slave_okay is enabled

Deprecated since version Ignored: with PyMongo 3+

snapshot(enabled)¶

Enable or disable snapshot mode when querying.

Parameters:	enabled – whether or not snapshot mode is enabled

..versionchanged:: 0.5 - made chainable .. deprecated:: Ignored with PyMongo 3+

sum(field)¶

Sum over the values of the specified field.

Parameters:	field – the field to sum over; use dot notation to refer to embedded document fields

timeout(enabled)¶

Enable or disable the default mongod timeout when querying.

Parameters:	enabled – whether or not the timeout is used

..versionchanged:: 0.5 - made chainable

to_json(*args, **kwargs)¶: Converts a queryset to JSON

update(upsert=False, multi=True, write_concern=None, full_result=False, **update)¶

Perform an atomic update on the fields matched by the query.

Parameters:

upsert – insert if document doesn’t exist (default False)
multi – Update multiple documents.
write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
full_result – Return the full result dictionary rather than just the number updated, e.g. return {'n': 2, 'nModified': 2, 'ok': 1.0, 'updatedExisting': True}.
update – Django-style update keyword arguments

New in version 0.2.

update_one(upsert=False, write_concern=None, **update)¶

Perform an atomic update on the fields of the first document matched by the query.

Parameters:

upsert – insert if document doesn’t exist (default False)
write_concern – Extra keyword arguments are passed down which will be used as options for the resultant getLastError command. For example, save(..., write_concern={w: 2, fsync: True}, ...) will wait until at least two servers have recorded the write and will force an fsync on the primary server.
update – Django-style update keyword arguments

New in version 0.2.

upsert_one(write_concern=None, **update)¶

Overwrite or add the first document matched by the query.

Parameters:	write_concern – Extra keyword arguments are passed down which will be used as options for the resultant `getLastError` command. For example, `save(..., write_concern={w: 2, fsync: True}, ...)` will wait until at least two servers have recorded the write and will force an fsync on the primary server. update – Django-style update keyword arguments

:returns the new or overwritten document

New in version 0.10.2.

using(alias)¶

This method is for controlling which database the QuerySet will be evaluated against if you are using more than one database.

Parameters:	alias – The database alias

New in version 0.9.

values_list(*fields)¶: An alias for scalar

where(where_clause)¶: Filter QuerySet results with a $where clause (a Javascript expression). Performs automatic field name substitution like mongoengine.queryset.Queryset.exec_js().

Note

When using this mode of query, the database will call your function, or evaluate your predicate clause, for each object in the collection.

New in version 0.5.

with_id(object_id)¶

Retrieve the object matching the id provided. Uses object_id only and raises InvalidQueryError if a filter has been applied. Returns None if no document exists with that id.

Parameters:	object_id – the value for the id of the document to look up

Changed in version 0.6: Raises InvalidQueryError if filter has been set

class mongoengine.queryset.QuerySetNoCache(document, collection)¶

A non caching QuerySet

__call__(q_obj=None, class_check=True, read_preference=None, **query)¶

Filter the selected documents by calling the QuerySet with a query.

Parameters:

q_obj – a Q object to be used in the query; the QuerySet is filtered multiple times with different Q objects, only the last one will be used

class_check – If set to False bypass class name check when querying collection

read_preference – if set, overrides connection-level read_preference from ReplicaSetConnection.

query – Django-style query keyword arguments

cache()¶: Convert to a caching queryset

New in version 0.8.3: Convert to caching queryset

mongoengine.queryset.queryset_manager(func)¶: Decorator that allows you to define custom QuerySet managers on Document classes. The manager must be a function that accepts a Document class as its first argument, and a QuerySet as its second argument. The method function should return a QuerySet, probably the same one that was passed in, but modified in some way.

Fields¶

class mongoengine.base.fields.BaseField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

A base class for fields in a MongoDB document. Instances of this class may be added to subclasses of Document to define a document’s schema.

Changed in version 0.5: - added verbose and help text

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.StringField(regex=None, max_length=None, min_length=None, **kwargs)¶: A unicode string field.

class mongoengine.fields.URLField(verify_exists=False, url_regex=None, schemes=None, **kwargs)¶: A field that validates input as an URL.

New in version 0.3.

class mongoengine.fields.EmailField(domain_whitelist=None, allow_utf8_user=False, allow_ip_domain=False, *args, **kwargs)¶

A field that validates input as an email address.

New in version 0.4.

Initialize the EmailField.

Args:

domain_whitelist (list) - list of otherwise invalid domain: names which you’d like to support.
allow_utf8_user (bool) - if True, the user part of the email: address can contain UTF8 characters. False by default.
allow_ip_domain (bool) - if True, the domain part of the email: can be a valid IPv4 or IPv6 address.

class mongoengine.fields.IntField(min_value=None, max_value=None, **kwargs)¶: 32-bit integer field.

class mongoengine.fields.LongField(min_value=None, max_value=None, **kwargs)¶: 64-bit integer field.

class mongoengine.fields.FloatField(min_value=None, max_value=None, **kwargs)¶: Floating point number field.

class mongoengine.fields.DecimalField(min_value=None, max_value=None, force_string=False, precision=2, rounding='ROUND_HALF_UP', **kwargs)¶

Fixed-point decimal number field.

Changed in version 0.8.

New in version 0.3.

Parameters:

min_value – Validation rule for the minimum acceptable value.
max_value – Validation rule for the maximum acceptable value.
force_string – Store as a string.
precision – Number of decimal places to store.
rounding –
The rounding rule from the python decimal library:
- decimal.ROUND_CEILING (towards Infinity)
- decimal.ROUND_DOWN (towards zero)
- decimal.ROUND_FLOOR (towards -Infinity)
- decimal.ROUND_HALF_DOWN (to nearest with ties going towards zero)
- decimal.ROUND_HALF_EVEN (to nearest with ties going to nearest even integer)
- decimal.ROUND_HALF_UP (to nearest with ties going away from zero)
- decimal.ROUND_UP (away from zero)
- decimal.ROUND_05UP (away from zero if last digit after rounding towards zero would have been 0 or 5; otherwise towards zero)
Defaults to: decimal.ROUND_HALF_UP

class mongoengine.fields.BooleanField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

Boolean field type.

New in version 0.1.2.

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.DateTimeField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

Datetime field.

Uses the python-dateutil library if available alternatively use time.strptime to parse the dates. Note: python-dateutil’s parser is fully featured and when installed you can utilise it to convert varying types of date formats into valid python datetime objects.

Note: Microseconds are rounded to the nearest millisecond.: Pre UTC microsecond support is effectively broken. Use ComplexDateTimeField if you need accurate microsecond support.

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.ComplexDateTimeField(separator=', ', **kwargs)¶

ComplexDateTimeField handles microseconds exactly instead of rounding like DateTimeField does.

Derives from a StringField so you can do gte and lte filtering by using lexicographical comparison when filtering / sorting strings.

The stored string has the following format:

YYYY,MM,DD,HH,MM,SS,NNNNNN

Where NNNNNN is the number of microseconds of the represented datetime. The , as the separator can be easily modified by passing the separator keyword when initializing the field.

New in version 0.5.

class mongoengine.fields.EmbeddedDocumentField(document_type, **kwargs)¶: An embedded document field - with a declared document_type. Only valid values are subclasses of EmbeddedDocument.

class mongoengine.fields.GenericEmbeddedDocumentField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

A generic embedded document field - allows any EmbeddedDocument to be stored.

Only valid values are subclasses of EmbeddedDocument.

Note

You can use the choices param to limit the acceptable EmbeddedDocument types

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.DynamicField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

A truly dynamic field type capable of handling different and varying types of data.

Used by DynamicDocument to handle dynamic data

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.ListField(field=None, **kwargs)¶

A list field that wraps a standard field, allowing multiple instances of the field to be used as a list in the database.

If using with ReferenceFields see: One to Many with ListFields

Note

Required means it cannot be empty - as the default for ListFields is []

class mongoengine.fields.EmbeddedDocumentListField(document_type, **kwargs)¶

A ListField designed specially to hold a list of embedded documents to provide additional query helpers.

Note

The only valid list values are subclasses of EmbeddedDocument.

New in version 0.9.

Parameters:	document_type – The type of `EmbeddedDocument` the list will hold. kwargs – Keyword arguments passed directly into the parent `ListField`.

class mongoengine.fields.SortedListField(field, **kwargs)¶: A ListField that sorts the contents of its list before writing to the database in order to ensure that a sorted list is always retrieved.

Warning

There is a potential race condition when handling lists. If you set / save the whole list then other processes trying to save the whole list as well could overwrite changes. The safest way to append to a list is to perform a push operation.

New in version 0.4.

Changed in version 0.6: - added reverse keyword

class mongoengine.fields.DictField(basecls=None, field=None, *args, **kwargs)¶: A dictionary field that wraps a standard Python dictionary. This is similar to an embedded document, but the structure is not defined.

Note

Required means it cannot be empty - as the default for DictFields is {}

New in version 0.3.

Changed in version 0.5: - Can now handle complex / varying types of data

class mongoengine.fields.MapField(field=None, *args, **kwargs)¶: A field that maps a name to a specified field type. Similar to a DictField, except the ‘value’ of each item must match the specified field type.

New in version 0.5.

class mongoengine.fields.ReferenceField(document_type, dbref=False, reverse_delete_rule=0, **kwargs)¶

A reference to a document that will be automatically dereferenced on access (lazily).

Note this means you will get a database I/O access everytime you access this field. This is necessary because the field returns a Document which precise type can depend of the value of the _cls field present in the document in database. In short, using this type of field can lead to poor performances (especially if you access this field only to retrieve it pk field which is already known before dereference). To solve this you should consider using the LazyReferenceField.

Use the reverse_delete_rule to handle what should happen if the document the field is referencing is deleted. EmbeddedDocuments, DictFields and MapFields does not support reverse_delete_rule and an InvalidDocumentError will be raised if trying to set on one of these Document / Field types.

The options are:

DO_NOTHING (0) - don’t do anything (default).

NULLIFY (1) - Updates the reference to null.

CASCADE (2) - Deletes the documents associated with the reference.

DENY (3) - Prevent the deletion of the reference object.

PULL (4) - Pull the reference from a ListField of references

Alternative syntax for registering delete rules (useful when implementing bi-directional delete rules)

class Bar(Document):
    content = StringField()
    foo = ReferenceField('Foo')

Foo.register_delete_rule(Bar, 'foo', NULLIFY)

Changed in version 0.5: added reverse_delete_rule

Initialises the Reference Field.

Parameters:	dbref – Store the reference as `DBRef` or as the `ObjectId`.id . reverse_delete_rule – Determines what to do when the referring object is deleted

Note

A reference to an abstract document type is always stored as a DBRef, regardless of the value of dbref.

class mongoengine.fields.LazyReferenceField(document_type, passthrough=False, dbref=False, reverse_delete_rule=0, **kwargs)¶

A really lazy reference to a document. Unlike the ReferenceField it will not be automatically (lazily) dereferenced on access. Instead, access will return a LazyReference class instance, allowing access to pk or manual dereference by using fetch() method.

New in version 0.15.

Initialises the Reference Field.

Parameters:	dbref – Store the reference as `DBRef` or as the `ObjectId`.id . reverse_delete_rule – Determines what to do when the referring object is deleted passthrough – When trying to access unknown fields, the

LazyReference instance will automatically call fetch() and try to retrive the field on the fetched document. Note this only work getting field (not setting or deleting).

class mongoengine.fields.GenericReferenceField(*args, **kwargs)¶

A reference to any Document subclass that will be automatically dereferenced on access (lazily).

Note this field works the same way as ReferenceField, doing database I/O access the first time it is accessed (even if it’s to access it pk or id field). To solve this you should consider using the GenericLazyReferenceField.

Note

Any documents used as a generic reference must be registered in the document registry. Importing the model will automatically register it.
You can use the choices param to limit the acceptable Document types

New in version 0.3.

class mongoengine.fields.GenericLazyReferenceField(*args, **kwargs)¶

A reference to any Document subclass. Unlike the GenericReferenceField it will not be automatically (lazily) dereferenced on access. Instead, access will return a LazyReference class instance, allowing access to pk or manual dereference by using fetch() method.

Note

Any documents used as a generic reference must be registered in the document registry. Importing the model will automatically register it.
You can use the choices param to limit the acceptable Document types

New in version 0.15.

class mongoengine.fields.CachedReferenceField(document_type, fields=None, auto_sync=True, **kwargs)¶

A referencefield with cache fields to purpose pseudo-joins

New in version 0.9.

Initialises the Cached Reference Field.

Parameters:	fields – A list of fields to be cached in document auto_sync – if True documents are auto updated.

class mongoengine.fields.BinaryField(max_bytes=None, **kwargs)¶: A binary data field.

class mongoengine.fields.FileField(db_alias='default', collection_name='fs', **kwargs)¶: A GridFS storage field.

New in version 0.4.

Changed in version 0.5: added optional size param for read

Changed in version 0.6: added db_alias for multidb support

class mongoengine.fields.ImageField(size=None, thumbnail_size=None, collection_name='images', **kwargs)¶

A Image File storage field.

@size (width, height, force):: max size to store images, if larger will be automatically resized ex: size=(800, 600, True)
@thumbnail (width, height, force):: size to generate a thumbnail

New in version 0.6.

class mongoengine.fields.SequenceField(collection_name=None, db_alias=None, sequence_name=None, value_decorator=None, *args, **kwargs)¶

Provides a sequential counter see:: http://www.mongodb.org/display/DOCS/Object+IDs#ObjectIDs-SequenceNumbers

Note

Although traditional databases often use increasing sequence numbers for primary keys. In MongoDB, the preferred approach is to use Object IDs instead. The concept is that in a very large cluster of machines, it is easier to create an object ID than have global, uniformly increasing sequence numbers.

Parameters:	collection_name – Name of the counter collection (default ‘mongoengine.counters’) sequence_name – Name of the sequence in the collection (default ‘ClassName.counter’) value_decorator – Any callable to use as a counter (default int)

Use any callable as value_decorator to transform calculated counter into any value suitable for your needs, e.g. string or hexadecimal representation of the default integer counter value.

Note

In case the counter is defined in the abstract document, it will be common to all inherited documents and the default sequence name will be the class name of the abstract document.

New in version 0.5.

Changed in version 0.8: added value_decorator

class mongoengine.fields.ObjectIdField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

A field wrapper around MongoDB’s ObjectIds.

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.UUIDField(binary=True, **kwargs)¶

A UUID field.

New in version 0.6.

Store UUID data in the database

Parameters:	binary – if False store as a string.

Changed in version 0.8.0.

Changed in version 0.6.19.

class mongoengine.fields.GeoPointField(db_field=None, name=None, required=False, default=None, unique=False, unique_with=None, primary_key=False, validation=None, choices=None, null=False, sparse=False, **kwargs)¶

A list storing a longitude and latitude coordinate.

Note

this represents a generic point in a 2D plane and a legacy way of representing a geo point. It admits 2d indexes but not “2dsphere” indexes in MongoDB > 2.4 which are more natural for modeling geospatial points. See Geospatial indexes

New in version 0.4.

Parameters:

db_field – The database field to store this field in (defaults to the name of the field)
name – Deprecated - use db_field
required – If the field is required. Whether it has to have a value or not. Defaults to False.
default – (optional) The default value for this field if no value has been set (or if the value has been unset). It can be a callable.
unique – Is the field value unique or not. Defaults to False.
unique_with – (optional) The other field this field should be unique with.
primary_key – Mark this field as the primary key. Defaults to False.
validation – (optional) A callable to validate the value of the field. Generally this is deprecated in favour of the FIELD.validate method
choices – (optional) The valid choices
null – (optional) Is the field value can be null. If no and there is a default value then the default value is set
sparse – (optional) sparse=True combined with unique=True and required=False means that uniqueness won’t be enforced for None values
**kwargs –
(optional) Arbitrary indirection-free metadata for this field can be supplied as additional keyword arguments and accessed as attributes of the field. Must not conflict with any existing attributes. Common metadata includes verbose_name and help_text.

class mongoengine.fields.PointField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing a longitude and latitude coordinate.

The data is represented as:

{'type' : 'Point' ,
 'coordinates' : [x, y]}

You can either pass a dict with the full information or a list to set the value.

Requires mongodb >= 2.4

New in version 0.8.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.LineStringField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing a line of longitude and latitude coordinates.

The data is represented as:

{'type' : 'LineString' ,
 'coordinates' : [[x1, y1], [x1, y1] ... [xn, yn]]}

You can either pass a dict with the full information or a list of points.

Requires mongodb >= 2.4

New in version 0.8.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.PolygonField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing a polygon of longitude and latitude coordinates.

The data is represented as:

{'type' : 'Polygon' ,
 'coordinates' : [[[x1, y1], [x1, y1] ... [xn, yn]],
                  [[x1, y1], [x1, y1] ... [xn, yn]]}

You can either pass a dict with the full information or a list of LineStrings. The first LineString being the outside and the rest being holes.

Requires mongodb >= 2.4

New in version 0.8.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.MultiPointField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing a list of Points.

The data is represented as:

{'type' : 'MultiPoint' ,
 'coordinates' : [[x1, y1], [x2, y2]]}

You can either pass a dict with the full information or a list to set the value.

Requires mongodb >= 2.6

New in version 0.9.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.MultiLineStringField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing a list of LineStrings.

The data is represented as:

{'type' : 'MultiLineString' ,
 'coordinates' : [[[x1, y1], [x1, y1] ... [xn, yn]],
                  [[x1, y1], [x1, y1] ... [xn, yn]]]}

You can either pass a dict with the full information or a list of points.

Requires mongodb >= 2.6

New in version 0.9.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.MultiPolygonField(auto_index=True, *args, **kwargs)¶

A GeoJSON field storing list of Polygons.

The data is represented as:

{'type' : 'MultiPolygon' ,
 'coordinates' : [[
       [[x1, y1], [x1, y1] ... [xn, yn]],
       [[x1, y1], [x1, y1] ... [xn, yn]]
   ], [
       [[x1, y1], [x1, y1] ... [xn, yn]],
       [[x1, y1], [x1, y1] ... [xn, yn]]
   ]
}

You can either pass a dict with the full information or a list of Polygons.

Requires mongodb >= 2.6

New in version 0.9.

Parameters:	auto_index (bool) – Automatically create a ‘2dsphere’ index. Defaults to True.

class mongoengine.fields.GridFSError¶

class mongoengine.fields.GridFSProxy(grid_id=None, key=None, instance=None, db_alias='default', collection_name='fs')¶: Proxy object to handle writing and reading of files to and from GridFS

New in version 0.4.

Changed in version 0.5: - added optional size param to read

Changed in version 0.6: - added collection name param

class mongoengine.fields.ImageGridFsProxy(grid_id=None, key=None, instance=None, db_alias='default', collection_name='fs')¶

Proxy for ImageField

versionadded: 0.6

class mongoengine.fields.ImproperlyConfigured¶

Embedded Document Querying¶

New in version 0.9.

Additional queries for Embedded Documents are available when using the EmbeddedDocumentListField to store a list of embedded documents.

A list of embedded documents is returned as a special list with the following methods:

class mongoengine.base.datastructures.EmbeddedDocumentList(list_items, instance, name)¶

count()¶

The number of embedded documents in the list.

Returns:	The length of the list, equivalent to the result of `len()`.

create(**values)¶

Creates a new embedded document and saves it to the database.

Note

The embedded document changes are not automatically saved to the database after calling this method.

Parameters:	values – A dictionary of values for the embedded document.
Returns:	The new embedded document instance.

delete()¶

Deletes the embedded documents from the database.

Note

The embedded document changes are not automatically saved to the database after calling this method.

Returns:	The number of entries deleted.

exclude(**kwargs)¶

Filters the list by excluding embedded documents with the given keyword arguments.

Parameters:	kwargs – The keyword arguments corresponding to the fields to exclude on. Multiple arguments are treated as if they are ANDed together.
Returns:	A new `EmbeddedDocumentList` containing the non-matching embedded documents.

Raises AttributeError if a given keyword is not a valid field for the embedded document class.

filter(**kwargs)¶

Filters the list by only including embedded documents with the given keyword arguments.

Parameters:	kwargs – The keyword arguments corresponding to the fields to filter on. Multiple arguments are treated as if they are ANDed together.
Returns:	A new `EmbeddedDocumentList` containing the matching embedded documents.

Raises AttributeError if a given keyword is not a valid field for the embedded document class.

first()¶: Return the first embedded document in the list, or None if empty.

get(**kwargs)¶

Retrieves an embedded document determined by the given keyword arguments.

Parameters:	kwargs – The keyword arguments corresponding to the fields to search on. Multiple arguments are treated as if they are ANDed together.
Returns:	The embedded document matched by the given keyword arguments.

Raises DoesNotExist if the arguments used to query an embedded document returns no results. MultipleObjectsReturned if more than one result is returned.

save(*args, **kwargs)¶

Saves the ancestor document.

Parameters:	args – Arguments passed up to the ancestor Document’s save method. kwargs – Keyword arguments passed up to the ancestor Document’s save method.

update(**update)¶

Updates the embedded documents with the given replacement values. This function does not support mongoDB update operators such as inc__.

Note

The embedded document changes are not automatically saved to the database after calling this method.

Parameters:	update – A dictionary of update values to apply to each embedded document.
Returns:	The number of entries updated.

Misc¶

mongoengine.common._import_class(cls_name)¶

Cache mechanism for imports.

Due to complications of circular imports mongoengine needs to do lots of inline imports in functions. This is inefficient as classes are imported repeated throughout the mongoengine code. This is compounded by some recursive functions requiring inline imports.

mongoengine.common provides a single point to import all these classes. Circular imports aren’t an issue as it dynamically imports the class when first needed. Subsequent calls to the _import_class() can then directly retrieve the class from the mongoengine.common._class_registry_cache.

Changelog¶

dev¶

Subfield resolve error in generic_emdedded_document query #1651 #1652
use each modifier only with $position #1673 #1675
Improve LazyReferenceField and GenericLazyReferenceField with nested fields #1704
Fix validation error instance in GenericEmbeddedDocumentField #1067
Update cached fields when fields argument is given #1712
Add a db parameter to register_connection for compatibility with connect

Changes in 0.15.0¶

Add LazyReferenceField and GenericLazyReferenceField to address #1230

Changes in 0.14.1¶

Removed SemiStrictDict and started using a regular dict for BaseDocument._data #1630
Added support for the $position param in the $push operator #1566
Fixed DateTimeField interpreting an empty string as today #1533
Added a missing __ne__ method to the GridFSProxy class #1632
Fixed BaseQuerySet._fields_to_db_fields #1553

Changes in 0.14.0¶

BREAKING CHANGE: Removed the coerce_types param from QuerySet.as_pymongo #1549
POTENTIAL BREAKING CHANGE: Made EmbeddedDocument not hashable by default #1528
Improved code quality #1531, #1540, #1541, #1547

Changes in 0.13.0¶

POTENTIAL BREAKING CHANGE: Added Unicode support to the EmailField, see docs/upgrade.rst for details.

Changes in 0.12.0¶

POTENTIAL BREAKING CHANGE: Fixed limit/skip/hint/batch_size chaining #1476
POTENTIAL BREAKING CHANGE: Changed a public QuerySet.clone_into method to a private QuerySet._clone_into #1476
Fixed the way Document.objects.create works with duplicate IDs #1485
Fixed connecting to a replica set with PyMongo 2.x #1436
Fixed using sets in field choices #1481
Fixed deleting items from a ListField #1318
Fixed an obscure error message when filtering by field__in=non_iterable. #1237
Fixed behavior of a dec update operator #1450
Added a rename update operator #1454
Added validation for the db_field parameter #1448
Fixed the error message displayed when querying an EmbeddedDocumentField by an invalid value #1440
Fixed the error message displayed when validating unicode URLs #1486
Raise an error when trying to save an abstract document #1449

Changes in 0.11.0¶

BREAKING CHANGE: Renamed ConnectionError to MongoEngineConnectionError since the former is a built-in exception name in Python v3.x. #1428
BREAKING CHANGE: Dropped Python 2.6 support. #1428
BREAKING CHANGE: from mongoengine.base import ErrorClass won’t work anymore for any error from mongoengine.errors (e.g. ValidationError). Use from mongoengine.errors import ErrorClass instead. #1428
BREAKING CHANGE: Accessing a broken reference will raise a DoesNotExist error. In the past it used to return None. #1334
Fixed absent rounding for DecimalField when force_string is set. #1103

Changes in 0.10.8¶

Added support for QuerySet.batch_size (#1426)
Fixed query set iteration within iteration #1427
Fixed an issue where specifying a MongoDB URI host would override more information than it should #1421
Added ability to filter the generic reference field by ObjectId and DBRef #1425
Fixed delete cascade for models with a custom primary key field #1247
Added ability to specify an authentication mechanism (e.g. X.509) #1333
Added support for falsey primary keys (e.g. doc.pk = 0) #1354
Fixed QuerySet#sum/average for fields w/ explicit db_field #1417
Fixed filtering by embedded_doc=None #1422
Added support for cursor.comment #1420
Fixed doc.get_<field>_display #1419
Fixed __repr__ method of the StrictDict #1424
Added a deprecation warning for Python 2.6

Changes in 0.10.7¶

Dropped Python 3.2 support #1390
Fixed the bug where dynamic doc has index inside a dict field #1278
Fixed: ListField minus index assignment does not work #1128
Fixed cascade delete mixing among collections #1224
Add signal_kwargs argument to Document.save, Document.delete and BaseQuerySet.insert to be passed to signals calls #1206
Raise OperationError when trying to do a drop_collection on document with no collection set.
count on ListField of EmbeddedDocumentField fails. #1187
Fixed long fields stored as int32 in Python 3. #1253
MapField now handles unicodes keys correctly. #1267
ListField now handles negative indicies correctly. #1270
Fixed AttributeError when initializing EmbeddedDocument with positional args. #681
Fixed no_cursor_timeout error with pymongo 3.0+ #1304
Replaced map-reduce based QuerySet.sum/average with aggregation-based implementations #1336
Fixed support for __ to escape field names that match operators names in update #1351
Fixed BaseDocument#_mark_as_changed #1369
Added support for pickling QuerySet instances. #1397
Fixed connecting to a list of hosts #1389
Fixed a bug where accessing broken references wouldn’t raise a DoesNotExist error #1334
Fixed not being able to specify use_db_field=False on ListField(EmbeddedDocumentField) instances #1218
Improvements to the dictionary fields docs #1383

Changes in 0.10.6¶

Add support for mocking MongoEngine based on mongomock. #1151
Fixed not being able to run tests on Windows. #1153
Allow creation of sparse compound indexes. #1114
count on ListField of EmbeddedDocumentField fails. #1187

Changes in 0.10.5¶

Fix for reloading of strict with special fields. #1156

Changes in 0.10.4¶

SaveConditionError is now importable from the top level package. #1165
upsert_one method added. #1157

Changes in 0.10.3¶

Fix read_preference (it had chaining issues with PyMongo 2.x and it didn’t work at all with PyMongo 3.x) #1042

Changes in 0.10.2¶

Allow shard key to point to a field in an embedded document. #551
Allow arbirary metadata in fields. #1129
ReferenceFields now support abstract document types. #837

Changes in 0.10.1¶

Fix infinite recursion with CASCADE delete rules under specific conditions. #1046
Fix CachedReferenceField bug when loading cached docs as DBRef but failing to save them. #1047
Fix ignored chained options #842
Document save’s save_condition error raises SaveConditionError exception #1070
Fix Document.reload for DynamicDocument. #1050
StrictDict & SemiStrictDict are shadowed at init time. #1105
Fix ListField minus index assignment does not work. #1119
Remove code that marks field as changed when the field has default but not existed in database #1126
Remove test dependencies (nose and rednose) from install dependencies list. #1079
Recursively build query when using elemMatch operator. #1130
Fix instance back references for lists of embedded documents. #1131

Changes in 0.10.0¶

Django support was removed and will be available as a separate extension. #958
Allow to load undeclared field with meta attribute ‘strict’: False #957
Support for PyMongo 3+ #946
Removed get_or_create() deprecated since 0.8.0. #300
Improve Document._created status when switch collection and db #1020
Queryset update doesn’t go through field validation #453
Added support for specifying authentication source as option authSource in URI. #967
Fixed mark_as_changed to handle higher/lower level fields changed. #927
ListField of embedded docs doesn’t set the _instance attribute when iterating over it #914
Support += and *= for ListField #595
Use sets for populating dbrefs to dereference
Fixed unpickled documents replacing the global field’s list. #888
Fixed storage of microseconds in ComplexDateTimeField and unused separator option. #910
Don’t send a “cls” option to ensureIndex (related to https://jira.mongodb.org/browse/SERVER-769)
Fix for updating sorting in SortedListField. #978
Added __ support to escape field name in fields lookup keywords that match operators names #949
Fix for issue where FileField deletion did not free space in GridFS.
No_dereference() not respected on embedded docs containing reference. #517
Document save raise an exception if save_condition fails #1005
Fixes some internal _id handling issue. #961
Updated URL and Email Field regex validators, added schemes argument to URLField validation. #652
Capped collection multiple of 256. #1011
Added BaseQuerySet.aggregate_sum and BaseQuerySet.aggregate_average methods.
Fix for delete with write_concern {‘w’: 0}. #1008
Allow dynamic lookup for more than two parts. #882
Added support for min_distance on geo queries. #831
Allow to add custom metadata to fields #705

Changes in 0.9.0¶

Update FileField when creating a new file #714
Added EmbeddedDocumentListField for Lists of Embedded Documents. #826
ComplexDateTimeField should fall back to None when null=True #864
Request Support for $min, $max Field update operators #863
BaseDict does not follow setdefault #866
Add support for $type operator # 766
Fix tests for pymongo 2.8+ #877
No module named ‘django.utils.importlib’ (Django dev) #872
Field Choices Now Accept Subclasses of Documents
Ensure Indexes before Each Save #812
Generate Unique Indices for Lists of EmbeddedDocuments #358
Sparse fields #515
write_concern not in params of Collection#remove #801
Better BaseDocument equality check when not saved #798
OperationError: Shard Keys are immutable. Tried to update id even though the document is not yet saved #771
with_limit_and_skip for count should default like in pymongo #759
Fix storing value of precision attribute in DecimalField #787
Set attribute to None does not work (at least for fields with default values) #734
Querying by a field defined in a subclass raises InvalidQueryError #744
Add Support For MongoDB 2.6.X’s maxTimeMS #778
abstract shouldn’t be inherited in EmbeddedDocument # 789
Allow specifying the ‘_cls’ as a field for indexes #397
Stop ensure_indexes running on a secondaries unless connection is through mongos #746
Not overriding default values when loading a subset of fields #399
Saving document doesn’t create new fields in existing collection #620
Added Queryset.aggregate wrapper to aggregation framework #703
Added support to show original model fields on to_json calls instead of db_field #697
Added Queryset.search_text to Text indexes searchs #700
Fixed tests for Django 1.7 #696
Follow ReferenceFields in EmbeddedDocuments with select_related #690
Added preliminary support for text indexes #680
Added elemMatch operator as well - match is too obscure #653
Added support for progressive JPEG #486 #548
Allow strings to be used in index creation #675
Fixed EmbeddedDoc weakref proxy issue #592
Fixed nested reference field distinct error #583
Fixed change tracking on nested MapFields #539
Dynamic fields in embedded documents now visible to queryset.only() / qs.exclude() #425 #507
Add authentication_source option to register_connection #178 #464 #573 #580 #590
Implemented equality between Documents and DBRefs #597
Fixed ReferenceField inside nested ListFields dereferencing problem #368
Added the ability to reload specific document fields #100
Added db_alias support and fixes for custom map/reduce output #586
post_save signal now has access to delta information about field changes #594 #589
Don’t query with $orderby for qs.get() #600
Fix id shard key save issue #636
Fixes issue with recursive embedded document errors #557
Fix clear_changed_fields() clearing unsaved documents bug #602
Removing support for Django 1.4.x, pymongo 2.5.x, pymongo 2.6.x.
Removing support for Python < 2.6.6
Fixed $maxDistance location for geoJSON $near queries with MongoDB 2.6+ #664
QuerySet.modify() and Document.modify() methods to provide find_and_modify() like behaviour #677 #773
Added support for the using() method on a queryset #676
PYPY support #673
Connection pooling #674
Avoid to open all documents from cursors in an if stmt #655
Ability to clear the ordering #657
Raise NotUniqueError in Document.update() on pymongo.errors.DuplicateKeyError #626
Slots - memory improvements #625
Fixed incorrectly split a query key when it ends with “_” #619
Geo docs updates #613
Workaround a dateutil bug #608
Conditional save for atomic-style operations #511
Allow dynamic dictionary-style field access #559
Increase email field length to accommodate new TLDs #726
index_cls is ignored when deciding to set _cls as index prefix #733
Make ‘db’ argument to connection optional #737
Allow atomic update for the entire DictField #742
Added MultiPointField, MultiLineField, MultiPolygonField
Fix multiple connections aliases being rewritten #748
Fixed a few instances where reverse_delete_rule was written as reverse_delete_rules. #791
Make in_bulk() respect no_dereference() #775
Handle None from model __str__; Fixes #753 #754
_get_changed_fields fix for embedded documents with id field. #925

Changes in 0.8.7¶

Calling reload on deleted / nonexistent documents raises DoesNotExist (#538)
Stop ensure_indexes running on a secondaries (#555)
Fix circular import issue with django auth (#531) (#545)

Changes in 0.8.6¶

Fix django auth import (#531)

Changes in 0.8.5¶

Fix multi level nested fields getting marked as changed (#523)
Django 1.6 login fix (#522) (#527)
Django 1.6 session fix (#509)
EmbeddedDocument._instance is now set when setting the attribute (#506)
Fixed EmbeddedDocument with ReferenceField equality issue (#502)
Fixed GenericReferenceField serialization order (#499)
Fixed count and none bug (#498)
Fixed bug with .only() and DictField with digit keys (#496)
Added user_permissions to Django User object (#491, #492)
Fix updating Geo Location fields (#488)
Fix handling invalid dict field value (#485)
Added app_label to MongoUser (#484)
Use defaults when host and port are passed as None (#483)
Fixed distinct casting issue with ListField of EmbeddedDocuments (#470)
Fixed Django 1.6 sessions (#454, #480)

Changes in 0.8.4¶

Remove database name necessity in uri connection schema (#452)
Fixed “$pull” semantics for nested ListFields (#447)
Allow fields to be named the same as query operators (#445)
Updated field filter logic - can now exclude subclass fields (#443)
Fixed dereference issue with embedded listfield referencefields (#439)
Fixed slice when using inheritance causing fields to be excluded (#437)
Fixed ._get_db() attribute after a Document.switch_db() (#441)
Dynamic Fields store and recompose Embedded Documents / Documents correctly (#449)
Handle dynamic fieldnames that look like digits (#434)
Added get_user_document and improve mongo_auth module (#423)
Added str representation of GridFSProxy (#424)
Update transform to handle docs erroneously passed to unset (#416)
Fixed indexing - turn off _cls (#414)
Fixed dereference threading issue in ComplexField.__get__ (#412)
Fixed QuerySetNoCache.count() caching (#410)
Don’t follow references in _get_changed_fields (#422, #417)
Allow args and kwargs to be passed through to_json (#420)

Changes in 0.8.3¶

Fixed EmbeddedDocuments with id also storing _id (#402)
Added get_proxy_object helper to filefields (#391)
Added QuerySetNoCache and QuerySet.no_cache() for lower memory consumption (#365)
Fixed sum and average mapreduce dot notation support (#375, #376, #393)
Fixed as_pymongo to return the id (#386)
Document.select_related() now respects db_alias (#377)
Reload uses shard_key if applicable (#384)
Dynamic fields are ordered based on creation and stored in _fields_ordered (#396)

Potential breaking change: http://docs.mongoengine.org/en/latest/upgrade.html#to-0-8-3
Fixed pickling dynamic documents _dynamic_fields (#387)
Fixed ListField setslice and delslice dirty tracking (#390)
Added Django 1.5 PY3 support (#392)
Added match ($elemMatch) support for EmbeddedDocuments (#379)
Fixed weakref being valid after reload (#374)
Fixed queryset.get() respecting no_dereference (#373)
Added full_result kwarg to update (#380)

Changes in 0.8.2¶

Added compare_indexes helper (#361)
Fixed cascading saves which weren’t turned off as planned (#291)
Fixed Datastructures so instances are a Document or EmbeddedDocument (#363)
Improved cascading saves write performance (#361)
Fixed ambiguity and differing behaviour regarding field defaults (#349)
ImageFields now include PIL error messages if invalid error (#353)
Added lock when calling doc.Delete() for when signals have no sender (#350)
Reload forces read preference to be PRIMARY (#355)
Querysets are now lest restrictive when querying duplicate fields (#332, #333)
FileField now honouring db_alias (#341)
Removed customised __set__ change tracking in ComplexBaseField (#344)
Removed unused var in _get_changed_fields (#347)
Added pre_save_post_validation signal (#345)
DateTimeField now auto converts valid datetime isostrings into dates (#343)
DateTimeField now uses dateutil for parsing if available (#343)
Fixed Doc.objects(read_preference=X) not setting read preference (#352)
Django session ttl index expiry fixed (#329)
Fixed pickle.loads (#342)
Documentation fixes

Changes in 0.8.1¶

Fixed Python 2.6 django auth importlib issue (#326)
Fixed pickle unsaved document regression (#327)

Changes in 0.8.0¶

Fixed querying ReferenceField custom_id (#317)
Fixed pickle issues with collections (#316)
Added get_next_value preview for SequenceFields (#319)
Added no_sub_classes context manager and queryset helper (#312)
Querysets now utilises a local cache
Changed __len__ behaviour in the queryset (#247, #311)
Fixed querying string versions of ObjectIds issue with ReferenceField (#307)
Added $setOnInsert support for upserts (#308)
Upserts now possible with just query parameters (#309)
Upserting is the only way to ensure docs are saved correctly (#306)
Fixed register_delete_rule inheritance issue
Fix cloning of sliced querysets (#303)
Fixed update_one write concern (#302)
Updated minimum requirement for pymongo to 2.5
Add support for new geojson fields, indexes and queries (#299)
If values cant be compared mark as changed (#287)
Ensure as_pymongo() and to_json honour only() and exclude() (#293)
Document serialization uses field order to ensure a strict order is set (#296)
DecimalField now stores as float not string (#289)
UUIDField now stores as a binary by default (#292)
Added Custom User Model for Django 1.5 (#285)
Cascading saves now default to off (#291)
ReferenceField now store ObjectId’s by default rather than DBRef (#290)
Added ImageField support for inline replacements (#86)
Added SequenceField.set_next_value(value) helper (#159)
Updated .only() behaviour - now like exclude it is chainable (#202)
Added with_limit_and_skip support to count() (#235)
Objects queryset manager now inherited (#256)
Updated connection to use MongoClient (#262, #274)
Fixed db_alias and inherited Documents (#143)
Documentation update for document errors (#124)
Deprecated get_or_create (#35)
Updated inheritable objects created by upsert now contain _cls (#118)
Added support for creating documents with embedded documents in a single operation (#6)
Added to_json and from_json to Document (#1)
Added to_json and from_json to QuerySet (#131)
Updated index creation now tied to Document class (#102)
Added none() to queryset (#127)
Updated SequenceFields to allow post processing of the calculated counter value (#141)
Added clean method to documents for pre validation data cleaning (#60)
Added support setting for read prefrence at a query level (#157)
Added _instance to EmbeddedDocuments pointing to the parent (#139)
Inheritance is off by default (#122)
Remove _types and just use _cls for inheritance (#148)
Only allow QNode instances to be passed as query objects (#199)
Dynamic fields are now validated on save (#153) (#154)
Added support for multiple slices and made slicing chainable. (#170) (#190) (#191)
Fixed GridFSProxy __getattr__ behaviour (#196)
Fix Django timezone support (#151)
Simplified Q objects, removed QueryTreeTransformerVisitor (#98) (#171)
FileFields now copyable (#198)
Querysets now return clones and are no longer edit in place (#56)
Added support for $maxDistance (#179)
Uses getlasterror to test created on updated saves (#163)
Fixed inheritance and unique index creation (#140)
Fixed reverse delete rule with inheritance (#197)
Fixed validation for GenericReferences which haven’t been dereferenced
Added switch_db context manager (#106)
Added switch_db method to document instances (#106)
Added no_dereference context manager (#82) (#61)
Added switch_collection context manager (#220)
Added switch_collection method to document instances (#220)
Added support for compound primary keys (#149) (#121)
Fixed overriding objects with custom manager (#58)
Added no_dereference method for querysets (#82) (#61)
Undefined data should not override instance methods (#49)
Added Django Group and Permission (#142)
Added Doc class and pk to Validation messages (#69)
Fixed Documents deleted via a queryset don’t call any signals (#105)
Added the “get_decoded” method to the MongoSession class (#216)
Fixed invalid choices error bubbling (#214)
Updated Save so it calls $set and $unset in a single operation (#211)
Fixed inner queryset looping (#204)

Changes in 0.7.10¶

Fix UnicodeEncodeError for dbref (#278)
Allow construction using positional parameters (#268)
Updated EmailField length to support long domains (#243)
Added 64-bit integer support (#251)
Added Django sessions TTL support (#224)
Fixed issue with numerical keys in MapField(EmbeddedDocumentField()) (#240)
Fixed clearing _changed_fields for complex nested embedded documents (#237, #239, #242)
Added “id” back to _data dictionary (#255)
Only mark a field as changed if the value has changed (#258)
Explicitly check for Document instances when dereferencing (#261)
Fixed order_by chaining issue (#265)
Added dereference support for tuples (#250)
Resolve field name to db field name when using distinct(#260, #264, #269)
Added kwargs to doc.save to help interop with django (#223, #270)
Fixed cloning querysets in PY3
Int fields no longer unset in save when changed to 0 (#272)
Fixed ReferenceField query chaining bug fixed (#254)

Changes in 0.7.9¶

Better fix handling for old style _types
Embedded SequenceFields follow collection naming convention

Changes in 0.7.8¶

Fix sequence fields in embedded documents (#166)
Fix query chaining with .order_by() (#176)
Added optional encoding and collection config for Django sessions (#180, #181, #183)
Fixed EmailField so can add extra validation (#173, #174, #187)
Fixed bulk inserts can now handle custom pk’s (#192)
Added as_pymongo method to return raw or cast results from pymongo (#193)

Changes in 0.7.7¶

Fix handling for old style _types

Changes in 0.7.6¶

Unicode fix for repr (#133)
Allow updates with match operators (#144)
Updated URLField - now can have a override the regex (#136)
Allow Django AuthenticationBackends to work with Django user (hmarr/mongoengine#573)
Fixed reload issue with ReferenceField where dbref=False (#138)

Changes in 0.7.5¶

ReferenceFields with dbref=False use ObjectId instead of strings (#134) See ticket for upgrade notes (#134)

Changes in 0.7.4¶

Fixed index inheritance issues - firmed up testcases (#123) (#125)

Changes in 0.7.3¶

Reverted EmbeddedDocuments meta handling - now can turn off inheritance (#119)

Changes in 0.7.2¶

Update index spec generation so its not destructive (#113)

Changes in 0.7.1¶

Fixed index spec inheritance (#111)

Changes in 0.7.0¶

Updated queryset.delete so you can use with skip / limit (#107)
Updated index creation allows kwargs to be passed through refs (#104)
Fixed Q object merge edge case (#109)
Fixed reloading on sharded documents (hmarr/mongoengine#569)
Added NotUniqueError for duplicate keys (#62)
Added custom collection / sequence naming for SequenceFields (#92)
Fixed UnboundLocalError in composite index with pk field (#88)
Updated ReferenceField’s to optionally store ObjectId strings this will become the default in 0.8 (#89)
Added FutureWarning - save will default to cascade=False in 0.8
Added example of indexing embedded document fields (#75)
Fixed ImageField resizing when forcing size (#80)
Add flexibility for fields handling bad data (#78)
Embedded Documents no longer handle meta definitions
Use weakref proxies in base lists / dicts (#74)
Improved queryset filtering (hmarr/mongoengine#554)
Fixed Dynamic Documents and Embedded Documents (hmarr/mongoengine#561)
Fixed abstract classes and shard keys (#64)
Fixed Python 2.5 support
Added Python 3 support (thanks to Laine Heron)

Changes in 0.6.20¶

Added support for distinct and db_alias (#59)
Improved support for chained querysets when constraining the same fields (hmarr/mongoengine#554)
Fixed BinaryField lookup re (#48)

Changes in 0.6.19¶

Added Binary support to UUID (#47)
Fixed MapField lookup for fields without declared lookups (#46)
Fixed BinaryField python value issue (#48)
Fixed SequenceField non numeric value lookup (#41)
Fixed queryset manager issue (#52)
Fixed FileField comparision (hmarr/mongoengine#547)

Changes in 0.6.18¶

Fixed recursion loading bug in _get_changed_fields

Changes in 0.6.17¶

Fixed issue with custom queryset manager expecting explict variable names

Changes in 0.6.16¶

Fixed issue where db_alias wasn’t inherited

Changes in 0.6.15¶

Updated validation error messages
Added support for null / zero / false values in item_frequencies
Fixed cascade save edge case
Fixed geo index creation through reference fields
Added support for args / kwargs when using @queryset_manager
Deref list custom id fix

Changes in 0.6.14¶

Fixed error dict with nested validation
Fixed Int/Float fields and not equals None
Exclude tests from installation
Allow tuples for index meta
Fixed use of str in instance checks
Fixed unicode support in transform update
Added support for add_to_set and each

Changes in 0.6.13¶

Fixed EmbeddedDocument db_field validation issue
Fixed StringField unicode issue
Fixes __repr__ modifying the cursor

Changes in 0.6.12¶

Fixes scalar lookups for primary_key
Fixes error with _delta handling DBRefs

Changes in 0.6.11¶

Fixed inconsistency handling None values field attrs
Fixed map_field embedded db_field issue
Fixed .save() _delta issue with DbRefs
Fixed Django TestCase
Added cmp to Embedded Document
Added PULL reverse_delete_rule
Fixed CASCADE delete bug
Fixed db_field data load error
Fixed recursive save with FileField

Changes in 0.6.10¶

Fixed basedict / baselist to return super(..)
Promoted BaseDynamicField to DynamicField

Changes in 0.6.9¶

Fixed sparse indexes on inherited docs
Removed FileField auto deletion, needs more work maybe 0.7

Changes in 0.6.8¶

Fixed FileField losing reference when no default set
Removed possible race condition from FileField (grid_file)
Added assignment to save, can now do: b = MyDoc(**kwargs).save()
Added support for pull operations on nested EmbeddedDocuments
Added support for choices with GenericReferenceFields
Added support for choices with GenericEmbeddedDocumentFields
Fixed Django 1.4 sessions first save data loss
FileField now automatically delete files on .delete()
Fix for GenericReference to_mongo method
Fixed connection regression
Updated Django User document, now allows inheritance

Changes in 0.6.7¶

Fixed indexing on ‘_id’ or ‘pk’ or ‘id’
Invalid data from the DB now raises a InvalidDocumentError
Cleaned up the Validation Error - docs and code
Added meta auto_create_index so you can disable index creation
Added write concern options to inserts
Fixed typo in meta for index options
Bug fix Read preference now passed correctly
Added support for File like objects for GridFS
Fix for #473 - Dereferencing abstracts

Changes in 0.6.6¶

Django 1.4 fixed (finally)
Added tests for Django

Changes in 0.6.5¶

More Django updates

Changes in 0.6.4¶

Refactored connection / fixed replicasetconnection
Bug fix for unknown connection alias error message
Sessions support Django 1.3 and Django 1.4
Minor fix for ReferenceField

Changes in 0.6.3¶

Updated sessions for Django 1.4
Bug fix for updates where listfields contain embedded documents
Bug fix for collection naming and mixins

Changes in 0.6.2¶

Updated documentation for ReplicaSet connections
Hack round _types issue with SERVER-5247 - querying other arrays may also cause problems.

Changes in 0.6.1¶

Fix for replicaSet connections

Changes in 0.6¶

Added FutureWarning to inherited classes not declaring ‘allow_inheritance’ as the default will change in 0.7
Added support for covered indexes when inheritance is off
No longer always upsert on save for items with a ‘_id’
Error raised if update doesn’t have an operation
DeReferencing is now thread safe
Errors raised if trying to perform a join in a query
Updates can now take __raw__ queries
Added custom 2D index declarations
Added replicaSet connection support
Updated deprecated imports from pymongo (safe for pymongo 2.2)
Added uri support for connections
Added scalar for efficiently returning partial data values (aliased to values_list)
Fixed limit skip bug
Improved Inheritance / Mixin
Added sharding support
Added pymongo 2.1 support
Fixed Abstract documents can now declare indexes
Added db_alias support to individual documents
Fixed GridFS documents can now be pickled
Added Now raises an InvalidDocumentError when declaring multiple fields with the same db_field
Added InvalidQueryError when calling with_id with a filter
Added support for DBRefs in distinct()
Fixed issue saving False booleans
Fixed issue with dynamic documents deltas
Added Reverse Delete Rule support to ListFields - MapFields aren’t supported
Added customisable cascade kwarg options
Fixed Handle None values for non-required fields
Removed Document._get_subclasses() - no longer required
Fixed bug requiring subclasses when not actually needed
Fixed deletion of dynamic data
Added support for the $elementMatch operator
Added reverse option to SortedListFields
Fixed dereferencing - multi directional list dereferencing
Fixed issue creating indexes with recursive embedded documents
Fixed recursive lookup in _unique_with_indexes
Fixed passing ComplexField defaults to constructor for ReferenceFields
Fixed validation of DictField Int keys
Added optional cascade saving
Fixed dereferencing - max_depth now taken into account
Fixed document mutation saving issue
Fixed positional operator when replacing embedded documents
Added Non-Django Style choices back (you can have either)
Fixed __repr__ of a sliced queryset
Added recursive validation error of documents / complex fields
Fixed breaking during queryset iteration
Added pre and post bulk-insert signals
Added ImageField - requires PIL
Fixed Reference Fields can be None in get_or_create / queries
Fixed accessing pk on an embedded document
Fixed calling a queryset after drop_collection now recreates the collection
Add field name to validation exception messages
Added UUID field
Improved efficiency of .get()
Updated ComplexFields so if required they won’t accept empty lists / dicts
Added spec file for rpm-based distributions
Fixed ListField so it doesnt accept strings
Added DynamicDocument and EmbeddedDynamicDocument classes for expando schemas

Changes in v0.5.2¶

A Robust Circular reference bugfix

Changes in v0.5.1¶

Fixed simple circular reference bug

Changes in v0.5¶

Added InvalidDocumentError - so Document core methods can’t be overwritten
Added GenericEmbeddedDocument - so you can embed any type of embeddable document
Added within_polygon support - for those with mongodb 1.9
Updated sum / average to use map_reduce as db.eval doesn’t work in sharded environments
Added where() - filter to allowing users to specify query expressions as Javascript
Added SequenceField - for creating sequential counters
Added update() convenience method to a document
Added cascading saves - so changes to Referenced documents are saved on .save()
Added select_related() support
Added support for the positional operator
Updated geo index checking to be recursive and check in embedded documents
Updated default collection naming convention
Added Document Mixin support
Fixed queryet __repr__ mid iteration
Added hint() support, so can tell Mongo the proper index to use for the query
Fixed issue with inconsistent setting of _cls breaking inherited referencing
Added help_text and verbose_name to fields to help with some form libs
Updated item_frequencies to handle embedded document lookups
Added delta tracking now only sets / unsets explicitly changed fields
Fixed saving so sets updated values rather than overwrites
Added ComplexDateTimeField - Handles datetimes correctly with microseconds
Added ComplexBaseField - for improved flexibility and performance
Added get_FIELD_display() method for easy choice field displaying
Added queryset.slave_okay(enabled) method
Updated queryset.timeout(enabled) and queryset.snapshot(enabled) to be chainable
Added insert method for bulk inserts
Added blinker signal support
Added query_counter context manager for tests
Added map_reduce method item_frequencies and set as default (as db.eval doesn’t work in sharded environments)
Added inline_map_reduce option to map_reduce
Updated connection exception so it provides more info on the cause.
Added searching multiple levels deep in DictField
Added DictField entries containing strings to use matching operators
Added MapField, similar to DictField
Added Abstract Base Classes
Added Custom Objects Managers
Added sliced subfields updating
Added NotRegistered exception if dereferencing Document not in the registry
Added a write concern for save, update, update_one and get_or_create
Added slicing / subarray fetching controls
Fixed various unique index and other index issues
Fixed threaded connection issues
Added spherical geospatial query operators
Updated queryset to handle latest version of pymongo map_reduce now requires an output.
Added Document __hash__, __ne__ for pickling
Added FileField optional size arg for read method
Fixed FileField seek and tell methods for reading files
Added QuerySet.clone to support copying querysets
Fixed item_frequencies when using name thats the same as a native js function
Added reverse delete rules
Fixed issue with unset operation
Fixed Q-object bug
Added QuerySet.all_fields resets previous .only() and .exclude()
Added QuerySet.exclude
Added django style choices
Fixed order and filter issue
Added QuerySet.only subfield support
Added creation_counter to BaseField allowing fields to be sorted in the way the user has specified them
Fixed various errors
Added many tests

Changes in v0.4¶

Added GridFSStorage Django storage backend
Added FileField for GridFS support
New Q-object implementation, which is no longer based on Javascript
Added SortedListField
Added EmailField
Added GeoPointField
Added exact and iexact match operators to QuerySet
Added get_document_or_404 and get_list_or_404 Django shortcuts
Added new query operators for Geo queries
Added not query operator
Added new update operators: pop and add_to_set
Added __raw__ query parameter
Added support for custom querysets
Fixed document inheritance primary key issue
Added support for querying by array element position
Base class can now be defined for DictField
Fixed MRO error that occured on document inheritance
Added QuerySet.distinct, QuerySet.create, QuerySet.snapshot, QuerySet.timeout and QuerySet.all
Subsequent calls to connect() now work
Introduced min_length for StringField
Fixed multi-process connection issue
Other minor fixes

Changes in v0.3¶

Added MapReduce support
Added contains, startswith and endswith query operators (and case-insensitive versions that are prefixed with ‘i’)
Deprecated fields’ name parameter, replaced with db_field
Added QuerySet.only for only retrieving specific fields
Added QuerySet.in_bulk() for bulk querying using ids
QuerySets now have a rewind() method, which is called automatically when the iterator is exhausted, allowing QuerySets to be reused
Added DictField
Added URLField
Added DecimalField
Added BinaryField
Added GenericReferenceField
Added get() and get_or_create() methods to QuerySet
ReferenceFields may now reference the document they are defined on (recursive references) and documents that have not yet been defined
Document objects may now be compared for equality (equal if _ids are equal and documents are of same type)
QuerySet update methods now have an upsert parameter
Added field name substitution for Javascript code (allows the user to use the Python names for fields in JS, which are later substituted for the real field names)
Q objects now support regex querying
Fixed bug where referenced documents within lists weren’t properly dereferenced
ReferenceFields may now be queried using their _id
Fixed bug where EmbeddedDocuments couldn’t be non-polymorphic
queryset_manager functions now accept two arguments – the document class as the first and the queryset as the second
Fixed bug where QuerySet.exec_js ignored Q objects
Other minor fixes

Changes in v0.2.2¶

Fixed bug that prevented indexes from being used on ListFields
Document.filter() added as an alias to Document.__call__()
validate() may now be used on EmbeddedDocuments

Changes in v0.2.1¶

Added a MongoEngine backend for Django sessions
Added force_insert to Document.save()
Improved querying syntax for ListField and EmbeddedDocumentField
Added support for user-defined primary keys (_id in MongoDB)

Changes in v0.2¶

Added Q class for building advanced queries
Added QuerySet methods for atomic updates to documents
Fields may now specify unique=True to enforce uniqueness across a collection
Added option for default document ordering
Fixed bug in index definitions

Changes in v0.1.3¶

Added Django authentication backend
Added Document.meta support for indexes, which are ensured just before querying takes place
A few minor bugfixes

Changes in v0.1.2¶

Query values may be processed before before being used in queries
Made connections lazy
Fixed bug in Document dictionary-style access
Added BooleanField
Added Document.reload() method

Changes in v0.1.1¶

Documents may now use capped collections

Upgrading¶

Development¶

(Fill this out whenever you introduce breaking changes to MongoEngine)

0.14.0¶

This release includes a few bug fixes and a significant code cleanup. The most important change is that QuerySet.as_pymongo no longer supports a coerce_types mode. If you used it in the past, a) please let us know of your use case, b) you’ll need to override as_pymongo to get the desired outcome.

This release also makes the EmbeddedDocument not hashable by default. If you use embedded documents in sets or dictionaries, you might have to override __hash__ and implement a hashing logic specific to your use case. See #1528 for the reason behind this change.

0.13.0¶

This release adds Unicode support to the EmailField and changes its structure significantly. Previously, email addresses containing Unicode characters didn’t work at all. Starting with v0.13.0, domains with Unicode characters are supported out of the box, meaning some emails that previously didn’t pass validation now do. Make sure the rest of your application can accept such email addresses. Additionally, if you subclassed the EmailField in your application and overrode EmailField.EMAIL_REGEX, you will have to adjust your code to override EmailField.USER_REGEX, EmailField.DOMAIN_REGEX, and potentially EmailField.UTF8_USER_REGEX.

0.12.0¶

This release includes various fixes for the BaseQuerySet methods and how they are chained together. Since version 0.10.1 applying limit/skip/hint/batch_size to an already-existing queryset wouldn’t modify the underlying PyMongo cursor. This has been fixed now, so you’ll need to make sure that your code didn’t rely on the broken implementation.

Additionally, a public BaseQuerySet.clone_into has been renamed to a private _clone_into. If you directly used that method in your code, you’ll need to rename its occurrences.

0.11.0¶

This release includes a major rehaul of MongoEngine’s code quality and introduces a few breaking changes. It also touches many different parts of the package and although all the changes have been tested and scrutinized, you’re encouraged to thorougly test the upgrade.

First breaking change involves renaming ConnectionError to MongoEngineConnectionError. If you import or catch this exception, you’ll need to rename it in your code.

Second breaking change drops Python v2.6 support. If you run MongoEngine on that Python version, you’ll need to upgrade it first.

Third breaking change drops an old backward compatibility measure where from mongoengine.base import ErrorClass would work on top of from mongoengine.errors import ErrorClass (where ErrorClass is e.g. ValidationError). If you import any exceptions from mongoengine.base, change it to mongoengine.errors.

0.10.8¶

This version fixed an issue where specifying a MongoDB URI host would override more information than it should. These changes are minor, but they still subtly modify the connection logic and thus you’re encouraged to test your MongoDB connection before shipping v0.10.8 in production.

0.10.7¶

QuerySet.aggregate_sum and QuerySet.aggregate_average are dropped. Use QuerySet.sum and QuerySet.average instead which use the aggreation framework by default from now on.

0.9.0¶

The 0.8.7 package on pypi was corrupted. If upgrading from 0.8.7 to 0.9.0 please follow:

pip uninstall pymongo
pip uninstall mongoengine
pip install pymongo==2.8
pip install mongoengine

0.8.7¶

Calling reload on deleted / nonexistent documents now raises a DoesNotExist exception.

0.8.2 to 0.8.3¶

Minor change that may impact users:

DynamicDocument fields are now stored in creation order after any declared fields. Previously they were stored alphabetically.

0.7 to 0.8¶

There have been numerous backwards breaking changes in 0.8. The reasons for these are to ensure that MongoEngine has sane defaults going forward and that it performs the best it can out of the box. Where possible there have been FutureWarnings to help get you ready for the change, but that hasn’t been possible for the whole of the release.

Warning

Breaking changes - test upgrading on a test system before putting live. There maybe multiple manual steps in migrating and these are best honed on a staging / test system.

Python and PyMongo¶

MongoEngine requires python 2.6 (or above) and pymongo 2.5 (or above)

Data Model¶

Inheritance¶

The inheritance model has changed, we no longer need to store an array of types with the model we can just use the classname in _cls. This means that you will have to update your indexes for each of your inherited classes like so:

# 1. Declaration of the class
class Animal(Document):
    name = StringField()
    meta = {
        'allow_inheritance': True,
        'indexes': ['name']
    }

# 2. Remove _types
collection = Animal._get_collection()
collection.update({}, {"$unset": {"_types": 1}}, multi=True)

# 3. Confirm extra data is removed
count = collection.find({'_types': {"$exists": True}}).count()
assert count == 0

# 4. Remove indexes
info = collection.index_information()
indexes_to_drop = [key for key, value in info.iteritems()
                   if '_types' in dict(value['key'])]
for index in indexes_to_drop:
    collection.drop_index(index)

# 5. Recreate indexes
Animal.ensure_indexes()

Document Definition¶

The default for inheritance has changed - it is now off by default and _cls will not be stored automatically with the class. So if you extend your Document or EmbeddedDocuments you will need to declare allow_inheritance in the meta data like so:

class Animal(Document):
    name = StringField()

    meta = {'allow_inheritance': True}

Previously, if you had data in the database that wasn’t defined in the Document definition, it would set it as an attribute on the document. This is no longer the case and the data is set only in the document._data dictionary:

>>> from mongoengine import *
>>> class Animal(Document):
...    name = StringField()
...
>>> cat = Animal(name="kit", size="small")

# 0.7
>>> cat.size
u'small'

# 0.8
>>> cat.size
Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
AttributeError: 'Animal' object has no attribute 'size'

The Document class has introduced a reserved function clean(), which will be called before saving the document. If your document class happens to have a method with the same name, please try to rename it.

def clean(self):

pass

ReferenceField¶

ReferenceFields now store ObjectIds by default - this is more efficient than DBRefs as we already know what Document types they reference:

# Old code
class Animal(Document):
    name = ReferenceField('self')

# New code to keep dbrefs
class Animal(Document):
    name = ReferenceField('self', dbref=True)

To migrate all the references you need to touch each object and mark it as dirty eg:

# Doc definition
class Person(Document):
    name = StringField()
    parent = ReferenceField('self')
    friends = ListField(ReferenceField('self'))

# Mark all ReferenceFields as dirty and save
for p in Person.objects:
    p._mark_as_changed('parent')
    p._mark_as_changed('friends')
    p.save()

An example test migration for ReferenceFields is available on github.

Note

Internally mongoengine handles ReferenceFields the same, so they are converted to DBRef on loading and ObjectIds or DBRefs depending on settings on storage.

UUIDField¶

UUIDFields now default to storing binary values:

# Old code
class Animal(Document):
    uuid = UUIDField()

# New code
class Animal(Document):
    uuid = UUIDField(binary=False)

To migrate all the uuids you need to touch each object and mark it as dirty eg:

# Doc definition
class Animal(Document):
    uuid = UUIDField()

# Mark all UUIDFields as dirty and save
for a in Animal.objects:
    a._mark_as_changed('uuid')
    a.save()

An example test migration for UUIDFields is available on github.

DecimalField¶

DecimalFields now store floats - previously it was storing strings and that made it impossible to do comparisons when querying correctly.:

# Old code
class Person(Document):
    balance = DecimalField()

# New code
class Person(Document):
    balance = DecimalField(force_string=True)

To migrate all the DecimalFields you need to touch each object and mark it as dirty eg:

# Doc definition
class Person(Document):
    balance = DecimalField()

# Mark all DecimalField's as dirty and save
for p in Person.objects:
    p._mark_as_changed('balance')
    p.save()

Note

DecimalFields have also been improved with the addition of precision and rounding. See DecimalField for more information.

An example test migration for DecimalFields is available on github.

Cascading Saves¶

To improve performance document saves will no longer automatically cascade. Any changes to a Document’s references will either have to be saved manually or you will have to explicitly tell it to cascade on save:

# At the class level:
class Person(Document):
    meta = {'cascade': True}

# Or on save:
my_document.save(cascade=True)

Storage¶

Document and Embedded Documents are now serialized based on declared field order. Previously, the data was passed to mongodb as a dictionary and which meant that order wasn’t guaranteed - so things like $addToSet operations on EmbeddedDocument could potentially fail in unexpected ways.

If this impacts you, you may want to rewrite the objects using the doc.mark_as_dirty('field') pattern described above. If you are using a compound primary key then you will need to ensure the order is fixed and match your EmbeddedDocument to that order.

Querysets¶

Attack of the clones¶

Querysets now return clones and should no longer be considered editable in place. This brings us in line with how Django’s querysets work and removes a long running gotcha. If you edit your querysets inplace you will have to update your code like so:

# Old code:
mammals = Animal.objects(type="mammal")
mammals.filter(order="Carnivora")       # Returns a cloned queryset that isn't assigned to anything - so this will break in 0.8
[m for m in mammals]                    # This will return all mammals in 0.8 as the 2nd filter returned a new queryset

# Update example a) assign queryset after a change:
mammals = Animal.objects(type="mammal")
carnivores = mammals.filter(order="Carnivora") # Reassign the new queryset so filter can be applied
[m for m in carnivores]                        # This will return all carnivores

# Update example b) chain the queryset:
mammals = Animal.objects(type="mammal").filter(order="Carnivora")  # The final queryset is assgined to mammals
[m for m in mammals]                                               # This will return all carnivores

Len iterates the queryset¶

If you ever did len(queryset) it previously did a count() under the covers, this caused some unusual issues. As len(queryset) is most often used by list(queryset) we now cache the queryset results and use that for the length.

This isn’t as performant as a count() and if you aren’t iterating the queryset you should upgrade to use count:

# Old code
len(Animal.objects(type="mammal"))

# New code
Animal.objects(type="mammal").count()

.only() now inline with .exclude()¶

The behaviour of .only() was highly ambiguous, now it works in mirror fashion to .exclude(). Chaining .only() calls will increase the fields required:

# Old code
Animal.objects().only(['type', 'name']).only('name', 'order')  # Would have returned just `name`

# New code
Animal.objects().only('name')

# Note:
Animal.objects().only(['name']).only('order')  # Now returns `name` *and* `order`

Client¶

PyMongo 2.4 came with a new connection client; MongoClient and started the depreciation of the old Connection. MongoEngine now uses the latest MongoClient for connections. By default operations were safe but if you turned them off or used the connection directly this will impact your queries.

Querysets¶

Safe¶

safe has been depreciated in the new MongoClient connection. Please use write_concern instead. As safe always defaulted as True normally no code change is required. To disable confirmation of the write just pass {“w”: 0} eg:

# Old
Animal(name="Dinasour").save(safe=False)

# new code:
Animal(name="Dinasour").save(write_concern={"w": 0})

Write Concern¶

write_options has been replaced with write_concern to bring it inline with pymongo. To upgrade simply rename any instances where you used the write_option keyword to write_concern like so:

# Old code:
Animal(name="Dinasour").save(write_options={"w": 2})

# new code:
Animal(name="Dinasour").save(write_concern={"w": 2})

Indexes¶

Index methods are no longer tied to querysets but rather to the document class. Although QuerySet._ensure_indexes and QuerySet.ensure_index still exist. They should be replaced with ensure_indexes() / ensure_index().

SequenceFields¶

SequenceField now inherits from BaseField to allow flexible storage of the calculated value. As such MIN and MAX settings are no longer handled.

0.6 to 0.7¶

Cascade saves¶

Saves will raise a FutureWarning if they cascade and cascade hasn’t been set to True. This is because in 0.8 it will default to False. If you require cascading saves then either set it in the meta or pass via save eg

# At the class level:
class Person(Document):
    meta = {'cascade': True}

# Or in code:
my_document.save(cascade=True)

Note

Remember: cascading saves do not cascade through lists.

ReferenceFields¶

ReferenceFields now can store references as ObjectId strings instead of DBRefs. This will become the default in 0.8 and if dbref is not set a FutureWarning will be raised.

To explicitly continue to use DBRefs change the dbref flag to True

class Person(Document):
    groups = ListField(ReferenceField(Group, dbref=True))

To migrate to using strings instead of DBRefs you will have to manually migrate

# Step 1 - Migrate the model definition
class Group(Document):
    author = ReferenceField(User, dbref=False)
    members = ListField(ReferenceField(User, dbref=False))

# Step 2 - Migrate the data
for g in Group.objects():
    g.author = g.author
    g.members = g.members
    g.save()

item_frequencies¶

In the 0.6 series we added support for null / zero / false values in item_frequencies. A side effect was to return keys in the value they are stored in rather than as string representations. Your code may need to be updated to handle native types rather than strings keys for the results of item frequency queries.

BinaryFields¶

Binary fields have been updated so that they are native binary types. If you previously were doing str comparisons with binary field values you will have to update and wrap the value in a str.

0.5 to 0.6¶

Embedded Documents - if you had a pk field you will have to rename it from _id to pk as pk is no longer a property of Embedded Documents.

Reverse Delete Rules in Embedded Documents, MapFields and DictFields now throw an InvalidDocument error as they aren’t currently supported.

Document._get_subclasses - Is no longer used and the class method has been removed.

Document.objects.with_id - now raises an InvalidQueryError if used with a filter.

FutureWarning - A future warning has been added to all inherited classes that don’t define allow_inheritance in their meta.

You may need to update pyMongo to 2.0 for use with Sharding.

0.4 to 0.5¶

There have been the following backwards incompatibilities from 0.4 to 0.5. The main areas of changed are: choices in fields, map_reduce and collection names.

Choice options:¶

Are now expected to be an iterable of tuples, with the first element in each tuple being the actual value to be stored. The second element is the human-readable name for the option.

PyMongo / MongoDB¶

map reduce now requires pymongo 1.11+- The pymongo merge_output and reduce_output parameters, have been depreciated.

More methods now use map_reduce as db.eval is not supported for sharding as such the following have been changed:

sum()

average()

item_frequencies()

Default collection naming¶

Previously it was just lowercase, it’s now much more pythonic and readable as it’s lowercase and underscores, previously

class MyAceDocument(Document):
    pass

MyAceDocument._meta['collection'] == myacedocument

In 0.5 this will change to

class MyAceDocument(Document):
    pass

MyAceDocument._get_collection_name() == my_ace_document

To upgrade use a Mixin class to set meta like so

class BaseMixin(object):
    meta = {
        'collection': lambda c: c.__name__.lower()
    }

class MyAceDocument(Document, BaseMixin):
    pass

MyAceDocument._get_collection_name() == "myacedocument"

Alternatively, you can rename your collections eg

from mongoengine.connection import _get_db
from mongoengine.base import _document_registry

def rename_collections():
    db = _get_db()

    failure = False

    collection_names = [d._get_collection_name()
                        for d in _document_registry.values()]

    for new_style_name in collection_names:
        if not new_style_name:  # embedded documents don't have collections
            continue
        old_style_name = new_style_name.replace('_', '')

        if old_style_name == new_style_name:
            continue  # Nothing to do

        existing = db.collection_names()
        if old_style_name in existing:
            if new_style_name in existing:
                failure = True
                print "FAILED to rename: %s to %s (already exists)" % (
                    old_style_name, new_style_name)
            else:
                db[old_style_name].rename(new_style_name)
                print "Renamed:  %s to %s" % (old_style_name,
                                              new_style_name)

    if failure:
        print "Upgrading  collection names failed"
    else:
        print "Upgraded collection names"

mongodb 1.8 > 2.0 +¶

It’s been reported that indexes may need to be recreated to the newer version of indexes. To do this drop indexes and call ensure_indexes on each model.

Django Support¶

Note

Django support has been split from the main MongoEngine repository. The legacy Django extension may be found bundled with the 0.9 release of MongoEngine.

Help Wanted!¶

The MongoEngine team is looking for help contributing and maintaining a new Django extension for MongoEngine! If you have Django experience and would like to help contribute to the project, please get in touch on the mailing list or by simply contributing on GitHub.

MongoEngine User Documentation¶

Community¶

Contributing¶

Changes¶

Offline Reading¶

Tutorial¶

Getting started¶

Defining our documents¶

Users¶

Posts, Comments and Tags¶

Posts¶

Tags¶

Comments¶

Handling deletions of references¶

Adding data to our Tumblelog¶

Accessing our data¶

Retrieving type-specific information¶

Searching our posts by tag¶

Learning more about MongoEngine¶

User Guide¶

Installing MongoEngine¶

Connecting to MongoDB¶

Replica Sets¶

Multiple Databases¶

Context Managers¶

Switch Database¶

Switch Collection¶

Defining documents¶

Defining a document’s schema¶

Dynamic document schemas¶

Fields¶

Field arguments¶

List fields¶

Embedded documents¶

Dictionary Fields¶

Reference fields¶

Uniqueness constraints¶

Skipping Document validation on save¶

Document collections¶

Capped collections¶

Indexes¶

Global index default options¶

Compound Indexes and Indexing sub documents¶

Geospatial indexes¶

Time To Live indexes¶

Comparing Indexes¶

Ordering¶

Shard keys¶

Document inheritance¶

Working with existing data¶

Abstract classes¶

Documents instances¶

Saving and deleting documents¶

Pre save data validation and cleaning¶

Cascading Saves¶

Deleting documents¶

Document IDs¶

Querying the database¶

Filtering queries¶

Query operators¶

String queries¶

Geo queries¶

Querying lists¶

Raw queries¶

Limiting and skipping results¶

Retrieving unique results¶

Default Document queries¶

Custom QuerySets¶

Aggregation¶

Counting results¶

Further aggregation¶

Query efficiency and performance¶

Retrieving a subset of fields¶

Getting related data¶

Turning off dereferencing¶

Advanced queries¶

Atomic updates¶

Server-side javascript execution¶

GridFS¶

Writing¶