Settings¶

To use boto3 set:

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto3.S3Boto3Storage'

To use the boto version of the backend set:

DEFAULT_FILE_STORAGE = 'storages.backends.s3boto.S3BotoStorage'

To allow django-admin.py collectstatic to automatically put your static files in your bucket set the following in your settings.py:

STATICFILES_STORAGE = 'storages.backends.s3boto.S3Boto3Storage'

Available are numerous settings. It should be especially noted the following:

AWS_ACCESS_KEY_ID

Your Amazon Web Services access key, as a string.

AWS_SECRET_ACCESS_KEY

Your Amazon Web Services secret access key, as a string.

AWS_STORAGE_BUCKET_NAME

Your Amazon Web Services storage bucket name, as a string.

AWS_DEFAULT_ACL (optional)

If set to private changes uploaded file’s Access Control List from the default permission public-read to give owner full control and remove read access from everyone else.

AWS_AUTO_CREATE_BUCKET (optional)

If set to True the bucket specified in AWS_STORAGE_BUCKET_NAME is automatically created.

AWS_HEADERS (optional)

If you’d like to set headers sent with each file of the storage:

# see http://developer.yahoo.com/performance/rules.html#expires
AWS_HEADERS = {
    'Expires': 'Thu, 15 Apr 2010 20:00:00 GMT',
    'Cache-Control': 'max-age=86400',
}

AWS_QUERYSTRING_AUTH (optional; default is True)

Setting AWS_QUERYSTRING_AUTH to False removes query parameter authentication from generated URLs. This can be useful if your S3 buckets are public.

AWS_QUERYSTRING_EXPIRE (optional; default is 3600 seconds)

The number of seconds that a generated URL with query parameter authentication is valid for.

AWS_S3_ENCRYPTION (optional; default is False)

Enable server-side file encryption while at rest, by setting encrypt_key parameter to True. More info available here: http://boto.cloudhackers.com/en/latest/ref/s3.html

AWS_S3_FILE_OVERWRITE (optional: default is True)

By default files with the same name will overwrite each other. Set this to False to have extra characters appended.

AWS_LOCATION (optional: default is ‘’)

A path prefix that will be prepended to all uploads

AWS_S3_CUSTOM_DOMAIN = 'cdn.mydomain.com'

Storage¶

Standard file access options are available, and work as expected:

>>> default_storage.exists('storage_test')
False
>>> file = default_storage.open('storage_test', 'w')
>>> file.write('storage contents')
>>> file.close()

>>> default_storage.exists('storage_test')
True
>>> file = default_storage.open('storage_test', 'r')
>>> file.read()
'storage contents'
>>> file.close()

>>> default_storage.delete('storage_test')
>>> default_storage.exists('storage_test')
False

Model¶

An object without a file has limited functionality:

>>> obj1 = MyStorage()
>>> obj1.normal
<FieldFile: None>
>>> obj1.normal.size
Traceback (most recent call last):
...
ValueError: The 'normal' attribute has no file associated with it.

Saving a file enables full functionality:

>>> obj1.normal.save('django_test.txt', ContentFile('content'))
>>> obj1.normal
<FieldFile: tests/django_test.txt>
>>> obj1.normal.size
7
>>> obj1.normal.read()
'content'

Files can be read in a little at a time, if necessary:

>>> obj1.normal.open()
>>> obj1.normal.read(3)
'con'
>>> obj1.normal.read()
'tent'
>>> '-'.join(obj1.normal.chunks(chunk_size=2))
'co-nt-en-t'

Save another file with the same name:

>>> obj2 = MyStorage()
>>> obj2.normal.save('django_test.txt', ContentFile('more content'))
>>> obj2.normal
<FieldFile: tests/django_test_.txt>
>>> obj2.normal.size
12

Push the objects into the cache to make sure they pickle properly:

>>> cache.set('obj1', obj1)
>>> cache.set('obj2', obj2)
>>> cache.get('obj2').normal
<FieldFile: tests/django_test_.txt>

Deleting an object deletes the file it uses, if there are no other objects still using that file:

>>> obj2.delete()
>>> obj2.normal.save('django_test.txt', ContentFile('more content'))
>>> obj2.normal
<FieldFile: tests/django_test_.txt>

Default values allow an object to access a single file:

>>> obj3 = MyStorage.objects.create()
>>> obj3.default
<FieldFile: tests/default.txt>
>>> obj3.default.read()
'default content'

But it shouldn’t be deleted, even if there are no more objects using it:

>>> obj3.delete()
>>> obj3 = MyStorage()
>>> obj3.default.read()
'default content'

Verify the fix for #5655, making sure the directory is only determined once:

>>> obj4 = MyStorage()
>>> obj4.random.save('random_file', ContentFile('random content'))
>>> obj4.random
<FieldFile: .../random_file>

Clean up the temporary files:

>>> obj1.normal.delete()
>>> obj2.normal.delete()
>>> obj3.default.delete()
>>> obj4.random.delete()

LIBCLOUD_PROVIDERS¶

This setting is required to configure connections to cloud storage providers. Each entry corresponds to a single ‘bucket’ of storage. You can have multiple buckets for a single service provider (e.g., multiple S3 buckets), and you can define buckets at multiple providers. For example, the following configuration defines 3 providers: two buckets (bucket-1 and bucket-2) on a US-based Amazon S3 store, and a third bucket (bucket-3) on Google:

LIBCLOUD_PROVIDERS = {
    'amazon_1': {
        'type': 'libcloud.storage.types.Provider.S3_US_STANDARD_HOST',
        'user': '<your username here>',
        'key': '<your key here>',
        'bucket': 'bucket-1',
    },
    'amazon_2': {
        'type': 'libcloud.storage.types.Provider.S3_US_STANDARD_HOST',
        'user': '<your username here>',
        'key': '<your key here>',
        'bucket': 'bucket-2',
    },
    'google': {
        'type': 'libcloud.storage.types.Provider.GOOGLE_STORAGE',
        'user': '<Your Google APIv1 username>',
        'key': '<Your Google APIv1 Key>',
        'bucket': 'bucket-3',
    },
}

The values for the type, user and key arguments will vary depending on your storage provider:

Amazon S3:

type: libcloud.storage.types.Provider.S3_US_STANDARD_HOST,

user: Your AWS access key ID

key: Your AWS secret access key

If you want to use a availability zone other than the US default, you can use one of S3_US_WEST_HOST, S3_US_WEST_OREGON_HOST, S3_EU_WEST_HOST, S3_AP_SOUTHEAST_HOST, or S3_AP_NORTHEAST_HOST instead of S3_US_STANDARD_HOST.

Google Cloud Storage:

type: libcloud.storage.types.Provider.GOOGLE_STORAGE,

user: Your Google APIv1 username (20 characters)

key: Your Google APIv1 key

Nimbus.io:

type: libcloud.storage.types.Provider.NIMBUS,

user: Your Nimbus.io user ID

key: Your Nimbus.io access key

Ninefold Cloud Storage:

type: libcloud.storage.types.Provider.NINEFOLD,

user: Your Atmos Access Token

key: Your Atmos Shared Secret

Rackspace Cloudfiles:

type: libcloud.storage.types.Provider.CLOUDFIULES_US or libcloud.storage.types.Provider.CLOUDFIULES_UK,

user: Your Rackspace user ID

key: Your Rackspace access key

You can specify any bucket name you want; however, the bucket must exist before you can start using it. If you need to create the bucket, you can use the storage API. For example, to create bucket-1 from our previous example:

>>> from storages.backends.apache_libcloud import LibCloudStorage
>>> store = LibCloudStorage('amazon_1')
>>> store.driver.create_container('bucket-1')

DEFAULT_LIBCLOUD_PROVIDER¶

Once you have defined your Libcloud providers, you have the option of setting one provider as the default provider of Libcloud storage. This is done setting DEFAULT_LIBCLOUD_PROVIDER to the key in LIBCLOUD_PROVIDER that you want to use as the default provider. For example, if you want the amazon-1 provider to be the default provider, use:

DEFAULT_LIBCLOUD_PROVIDER = 'amazon-1'

If DEFAULT_LIBCLOUD_PROVIDER isn’t set, the Libcloud backend will assume that the default storage backend is named default. Therefore, you can avoid settings DEFAULT_LIBCLOUD_PROVIDER by simply naming one of your Libcloud providers default:

LIBCLOUD_PROVIDERS = {
    'default': {
        'type': ...
    },
}

DEFAULT_FILE_STORAGE¶

If you want your Libcloud storage to be the default Django file store, you can set:

DEFAULT_FILE_STORAGE = 'storages.backends.apache_libcloud.LibCloudStorage'

Your default Libcloud provider will be used as the file store.