
A small library for extracting rich content from urls.
https://github.com/coleifer/micawber
what does it do?¶
micawber supplies a few methods for retrieving rich metadata about a variety of links, such as links to youtube videos. micawber also provides functions for parsing blocks of text and html and replacing links to videos with rich embedded content.
examples¶
here is a quick example:
import micawber
# load up rules for some default providers, such as youtube and flickr
providers = micawber.bootstrap_basic()
providers.request('http://www.youtube.com/watch?v=54XHDUOHuzU')
# returns the following dictionary:
{
'author_name': 'pascalbrax',
'author_url': u'http://www.youtube.com/user/pascalbrax'
'height': 344,
'html': u'<iframe width="459" height="344" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>',
'provider_name': 'YouTube',
'provider_url': 'http://www.youtube.com/',
'title': 'Future Crew - Second Reality demo - HD',
'type': u'video',
'thumbnail_height': 360,
'thumbnail_url': u'http://i2.ytimg.com/vi/54XHDUOHuzU/hqdefault.jpg',
'thumbnail_width': 480,
'url': 'http://www.youtube.com/watch?v=54XHDUOHuzU',
'width': 459,
'version': '1.0',
}
providers.parse_text('this is a test:\nhttp://www.youtube.com/watch?v=54XHDUOHuzU')
# returns the following string:
this is a test:
<iframe width="459" height="344" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>
providers.parse_html('<p>http://www.youtube.com/watch?v=54XHDUOHuzU</p>')
# returns the following html:
<p><iframe width="459" height="344" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
check out the getting started for more examples
integration with web frameworks¶
Contents:
Installation¶
First, you need to install micawber
There are a couple of ways:
Installing with pip¶
pip install micawber
or
pip install -e git+https://github.com/coleifer/micawber.git#egg=micawber
Installing via git¶
git clone https://github.com/coleifer/micawber.git
cd micawber
python setup.py test
sudo python setup.py install
Adding to your Django Project¶
After installing, adding django-utils to your projects is a snap. Simply add it to your projects’ INSTALLED_APPs and run ‘syncdb’:
# settings.py
INSTALLED_APPS = [
...
'micawber.contrib.mcdjango'
]
Getting Started¶
If you want the dead simple get-me-up-and-running, try the following:
>>> import micawber
>>> providers = micawber.bootstrap_basic() # may take a second
>>> print providers.parse_text('this is a test:\nhttp://www.youtube.com/watch?v=54XHDUOHuzU')
this is a test:
<iframe width="640" height="360" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>
Using django? Add micawber.contrib.mcdjango
to your INSTALLED_APP
, then
in your templates:
{% load micawber_tags %}
{# show a video player for the youtube video #}
{{ "http://www.youtube.com/watch?v=mQEWI1cn7HY"|oembed }}
Using flask? Use the add_oembed_filters
function to register two jinja
template filters, oembed
and extract_oembed
:
from flask import Flask
from micawber.providers import bootstrap_basic
from micawber.contrib.mcflask import add_oembed_filters
app = Flask(__name__)
oembed_providers = bootstrap_basic()
add_oembed_filters(app, oembed_providers)
{# show a video player for the youtube video #}
{{ "http://www.youtube.com/watch?v=mQEWI1cn7HY"|oembed() }}
Overview¶
micawber is rather simple. It is built to use the oembed spec, which is designed for converting URLs into rich, embeddable content. Many popular sites support this, including youtube and flickr. There is also a 3rd-party service called embedly that can convert many types of links into rich content.
micawber was designed to make it easy to integrate with these APIs. There are two concepts to understand when using micawber:
Provider
objects - which describe how to match a URL (based on a regex) to an OEmbed endpoint.ProviderRegistry
objects - which encapsulate a collection or providers and expose methods for parsing text and HTML to convert links into media objects.
Providers¶
Providers are used to convert URLs into rich metadata. They have an endpoint associated with them and can have any number of arbitrary URL parameters (such as API keys) which are used when making API requests.
Example:
from micawber.providers import Provider
youtube = Provider('http://www.youtube.com/oembed')
youtube.request('http://www.youtube.com/watch?v=nda_OSWeyn8')
The above code returns a dictionary containing metadata about the requested video, including the markup for an embeddable player:
{'author_name': u'botmib',
'author_url': u'http://www.youtube.com/user/botmib',
'height': 344,
'html': u'<iframe width="459" height="344" src="http://www.youtube.com/embed/nda_OSWeyn8?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>',
'provider_name': u'YouTube',
'provider_url': u'http://www.youtube.com/',
'thumbnail_height': 360,
'thumbnail_url': u'http://i3.ytimg.com/vi/nda_OSWeyn8/hqdefault.jpg',
'thumbnail_width': 480,
'title': u'Leprechaun in Mobile, Alabama',
'type': u'video',
'url': 'http://www.youtube.com/watch?v=nda_OSWeyn8',
'version': u'1.0',
'width': 459}
More information can be found in the Provider
API docs.
ProviderRegistry¶
The ProviderRegistry
is a way of organizing lists
of providers. URLs can be requested from the registry and if any provider matches
it will be used, otherwise a ProviderException
will be raised.
The ProviderRegistry
also supports an optional simple caching mechanism.
Here is an excerpt from the code from the micawber.providers.bootstrap_basic()
function,
which is handy for grabbing a ProviderRegistry
with a handful of basic providers
pre-populated:
def bootstrap_basic(cache=None, registry=None, **params):
pr = registry or ProviderRegistry(cache)
pr.register('http://\S*?flickr.com/\S*', Provider('http://www.flickr.com/services/oembed/'))
pr.register('http://\S*.youtu(\.be|be\.com)/watch\S*', Provider('http://www.youtube.com/oembed'))
pr.register('http://www.hulu.com/watch/\S*', Provider('http://www.hulu.com/api/oembed.json'))
return pr
As you can see, the register()
method takes
two parameters, a regular expression for valid URLs and a Provider
instance.
You can use helper functions to get a populated registry:
bootstrap_basic()
bootstrap_oembed()
- uses oembed.com’s official providers list.bootstrap_embedly()
bootstrap_noembed()
The bootstrap_oembed
, bootstrap_embedly
, and bootstrap_noembed
functions make a HTTP request to the API server asking for a list of supported
providers, so you may experience some latency when using these helpers. For
most WSGI applications this will not be an issue, but if you’d like to speed it
up I suggest fetching the results, storing them in the db or a file, and then
pulling from there.
More information can be found in the ProviderRegistry
API docs.
Parsing Links¶
Replace URLs with rich media:
parse_text()
, which converts URLs on their own line into a rich media object. Links embedded within blocks of text are converted into clickable links.parse_html()
, which converts URLs within HTML into rich media objects or clickable links, depending on the context in which the URL is found.
A quick example:
import micawber
providers = micawber.bootstrap_basic()
providers.parse_text('this is a test:\nhttp://www.youtube.com/watch?v=54XHDUOHuzU')
This will result in the following output:
this is a test:
<iframe width="459" height="344" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>
You can also parse HTML using the parse_html()
method:
providers.parse_html('<p>http://www.youtube.com/watch?v=54XHDUOHuzU</p>')
# yields the following output:
<p><iframe width="459" height="344" src="http://www.youtube.com/embed/54XHDUOHuzU?fs=1&feature=oembed" frameborder="0" allowfullscreen="allowfullscreen"></iframe></p>
If you would rather extract metadata, there are two functions:
extract()
, which finds all URLs within a block of text and returns a dictionary of metadata for each.extract_html()
, which finds URLs within HTML and returns a dictionary of metadata for each.
The API docs are extensive, so please refer there for a full list of parameters and functions.
Examples¶
micawber comes with a handful of examples showing usage with
Django example¶
The django example is very simple – it illustrates a single view that renders
text inputted by the user by piping it through the oembed()
filter. It also shows the output of the extract_oembed()
filter which returns a 2-tuple of URL -> metadata. There is also an input where
you can experiment with entering HTML.
To run the example:
cd examples/django_ex/
./manage.py runserver
Check out the example source code.
Flask example¶
The flask example is almost identical in terms of functionality to the django example. It
shows a one-file app with a single view that renders
text inputted by the user by piping it through the oembed()
filter. It also shows the output of the extract_oembed()
filter which returns a 2-tuple of URL -> metadata. There is also an input where
you can experiment with entering HTML.
To run the example:
cd examples/flask_ex/
python app.py
Check out the example source code.
Python example¶
The python example is a command-line app that shows the use of the micawber.providers.ProviderRegistry
and micawber.providers.bootstrap_embedly
. It runs a loop asking the user to input
URLs, outputting rich metadata when possible (view http://embed.ly for a full list of providers).
To run the example:
cd examples/python_ex/
python example.py
Check out the example source code.
Flask integration¶
micawber exposes two Jinja template filters for use in your flask templates:
You can add them to your jinja environment by using the helper function:
from flask import Flask
from micawber.providers import bootstrap_basic
from micawber.contrib.mcflask import add_oembed_filters
app = Flask(__name__)
oembed_providers = bootstrap_basic()
add_oembed_filters(app, oembed_providers)
Now you can use the filters in your templates:
{% block content %}
<p>{{ object.body|oembed(html=False, maxwidth=600, maxheight=600) }}</p>
{% endblock %}
Flask filter API¶
The following filters are exposed via the micawber.contrib.mcflask
module:
-
micawber.contrib.mcflask.
oembed
(text, urlize_all=True, html=False, **params)¶ Parse the given text, rendering URLs as rich media
Usage within a Jinja2 template:
{{ blog_entry.body|oembed(urlize_all=False, maxwidth=600) }}
Parameters: - text – the text to be parsed, can be HTML
- urlize_all – boolean indicating whether to convert bare links to clickable ones
- html – boolean indicating whether text is plaintext or markup
- params – any additional keyword arguments, e.g. maxwidth or an api key
Return type: parsed text with rich content embedded
-
micawber.contrib.mcflask.
extract_oembed
(text, html=False, **params)¶ Returns a 2-tuple containing
- a list of all URLs found within the text (if HTML, all URLs that aren’t already links)
- a dictionary of URL to metadata provided by the API endpoint
Note
Not all URLs listed will have matching entries in the dictionary, since there may not be a provider for them.
Parameters: - text – the text to be parsed, can be HTML
- html – boolean indicating whether text is plaintext or markup
- params – any additional keyword arguments, e.g. maxwidth or an api key
Return type: 2-tuple containing a list of all urls and a dictionary of url -> metadata
Adding filters to the Jinja Environment¶
To actually use these filters they must be made available to the application. Use the
following function to do this sometime after initializing your Flask
app:
-
micawber.contrib.mcflask.
add_oembed_filters
(app, providers)¶ Add the
oembed
andextract_oembed
filters to the jinja environmentParameters: - app – a flask application
- providers – a
micawber.providers.ProviderRegistry
instance
Return type: (no return value)
Django integration¶
First be sure you have added micawber.contrib.mcdjango
to INSTALLED_APPS
so that we can use the template filters it defines.
# settings.py
INSTALLED_APPS = [
# ...
'micawber.contrib.mcdjango',
]
micawber provides 4 template filters for converting URLs contained within text or HTML to rich content:
oembed()
for plain textoembed_html()
for htmlextract_oembed()
for extracting url data from plain textextract_oembed_html()
for extracting url data from html
These filters are registered in the micawber_tags
library, which can be
invoked in your templates:
{% load micawber_tags %}
<p>{{ object.body|oembed:"600x600" }}</p>
Each filter accepts one argument and one optional argument, due to django’s template filters being wack.
Piping a string through the oembed
filter (or oembed_html
) will convert
URLs to things like youtube videos into video players. A couple things to
understand about the parsers:
- the plaintext parser (
oembed
) will convert URLs on their own line into full images/video-players/etc. URLs that are interspersed within text will simply be converted into clickable links so as not to disrupt the flow of text. - the HTML parser (
oembed_html
) will convert URLs that are not already links into full images/video-players/etc. URLs within block elements along with other text will be converted into clickable links as this would likely disrupt the flow of text or produce invalid HTML.
Note
You can control how things are rendered – check out the default templates for reference implementations.
Django filter API¶
The following filters are exposed via the micawber.contrib.mcdjango
module:
-
micawber.contrib.mcdjango.
oembed
(text[, width_height=None])¶ Parse the given text, rendering URLs as rich media
Usage within a django template:
{{ blog_entry.body|oembed:"600x600" }}
Parameters: - text – the text to be parsed do not use HTML
- width_height – string containing maximum for width and optionally height, of format “WIDTHxHEIGHT” or “WIDTH”, e.g. “500x500” or “800”
Return type: parsed text with rich content embedded
-
micawber.contrib.mcdjango.
oembed_html
(html[, width_height=None])¶ Exactly the same as above except for usage with html
Usage within a django template:
{{ blog_entry.body|markdown|oembed_html:"600x600" }}
-
micawber.contrib.mcdjango.
extract_oembed
(text[, width_height=None])¶ Parse the given text, returning a list of 2-tuples containing url and metadata about the url.
Usage within a django template:
{% for url, metadata in blog_entry.body|extract_oembed:"600x600" %} <img src="{{ metadata.thumbnail_url }}" /> {% endfor %}
Parameters: - text – the text to be parsed do not use HTML
- width_height – string containing maximum for width and optionally height, of format “WIDTHxHEIGHT” or “WIDTH”, e.g. “500x500” or “800”
Return type: 2-tuples containing the URL and a dictionary of metadata
-
micawber.contrib.mcdjango.
extract_oembed_html
(html[, width_height=None])¶ Exactly the same as above except for usage with html
Extending the filters¶
For simplicity, micawber provides a setting allowing you to create custom template filters. An example use case would be to add a template filter that could embed rich content, but did not automatically “urlize” all links.
Extensions are configured in the settings
module and take the form of a list of
2-tuples containing:
- the name for the custom filter
- a dictionary of keyword arguments to pass in to the
parse
function
MICAWBER_TEMPLATE_EXTENSIONS = [
('oembed_no_urlize', {'urlize_all': False}),
]
Assume this is our template:
{% load micawber_tags %}
DEFAULT:
{{ "http://foo.com/ and http://bar.com/"|oembed }}
CUSTOM:
{{ "http://foo.com/ and http://bar.com/"|oembed_no_urlize }}
Rendering the above template will produce the following output:
DEFAULT:
<a href="http://foo.com/">http://foo.com/</a> and <a href="http://bar.com/">http://bar.com/</a>
CUSTOM:
http://foo.com/ and http://bar.com/
Some examples of keyword arguments to override are:
- providers: a
ProviderRegistry
instance - urlize_all (default
True
): whether to convert all URLs to clickable links - html (default
False
): whether to parse as plaintext or html - handler: function used to render metadata as markup
- block_handler: function used to render inline links with rich metadata
- text_fn: function to use when parsing text
- html_fn: function to use when parsing html
The magic happens in micawber.contrib.mcdjango.extension()
– check
out the source code for more details.
Note
The MICAWBER_EXTENSIONS
setting can also be a string path to
a module and an attribute containing a similar data structure.
Additional settings¶
Providers¶
The most important setting to configure is the module / attribute path to the providers you wish to use. The attribute can either be a ProviderRegistry instance or a callable. The default is:
MICAWBER_PROVIDERS = 'micawber.contrib.mcdjango.providers.bootstrap_basic'
You can use the bootstrap embedly function, but beware this may take a few seconds to load up:
MICAWBER_PROVIDERS = 'micawber.contrib.mcdjango.providers.bootstrap_embedly'
If you want to use the embedly endpoints and have an API key, you can specify that in the settings:
MICAWBER_EMBEDLY_KEY = 'foo'
You can also customize this with your own set of providers. This must be either
- the module path to a
ProviderRegistry
instance - the module path to a callable which returns a
ProviderRegistry
instance
Here is a quick example showing a custom ProviderRegistry
:
# settings.py
MICAWBER_PROVIDERS = 'my_app.micawber_providers.oembed_providers'
# my_app/micawber_providers.py
from django.core.cache import cache
from micawber.providers import Provider, bootstrap_basic
oembed_providers = boostrap_basic(cache)
# add a custom provider
oembed_providers.register('http://example.com/\S*', Provider('http://example.com/oembed/'))
Default settings for requests¶
Because of the limitations of django’s template filters, we do not have the flexibility to pass in multiple arguments to the filters. Default arguments need to be specified in the settings:
MICAWBER_DEFAULT_SETTINGS = {
'key': 'your-embedly-api-key',
'maxwidth': 600,
'maxheight': 600,
}
Trying it out in the python shell¶
>>> from django.template import Template, Context
>>> t = Template('{% load micawber_tags %}{{ "http://www.youtube.com/watch?v=mQEWI1cn7HY"|oembed }}')
>>> t.render(Context())
u'<iframe width="480" height="270" src="http://www.youtube.com/embed/mQEWI1cn7HY?fs=1&feature=oembed" frameborder="0" allowfullscreen></iframe>'
API Documentation¶
Providers¶
-
class
micawber.providers.
Provider
(endpoint, **kwargs)¶ The
Provider
object is responsible for retrieving metadata about a given URL. It implements a method calledrequest()
, which takes a URL and any parameters, which it sends off to an endpoint. The endpoint should return a JSON dictionary containing metadata about the resource, which is returned to the caller.Parameters: - endpoint – the API endpoint which should return information about requested links
- kwargs – any additional url parameters to send to the endpoint on each request, used for providing defaults. An example use-case might be for providing an API key on each request.
-
request
(url, **extra_params)¶ Retrieve information about the given url. By default, will make a HTTP GET request to the endpoint. The url will be sent to the endpoint, along with any parameters specified in the
extra_params
and those parameters specified when the class was instantiated.Will raise a
ProviderException
in the event the URL is not accessible or the API times out.Parameters: - url – URL to retrieve metadata for
- extra_params – additional parameters to pass to the endpoint, for example a maxwidth or an API key.
Return type: a dictionary of JSON data
-
class
micawber.providers.
ProviderRegistry
([cache=None])¶ A registry for encapsulating a group of
Provider
instances, with optional caching support.Handles matching regular expressions to providers. URLs are sent to the registry via its
request()
method, it checks to see if it has a provider that matches the URL, and if so, requests the metadata from the provider instance.Exposes methods for parsing various types of text (including HTML), and either rendering oembed media inline or extracting embeddable links.
Parameters: cache – the cache simply needs to implement two methods, .get(key)
and.set(key, value)
.-
register
(regex, provider)¶ Register the provider with the following regex.
Example:
registry = ProviderRegistry() registry.register( 'http://\S*.youtu(\.be|be\.com)/watch\S*', Provider('http://www.youtube.com/oembed'), )
Parameters: - regex – a regex for matching URLs of a given type
- provider – a
Provider
instance
-
request
(url, **extra_params)¶ Retrieve information about the given url if it matches a regex in the instance’s registry. If no provider matches the URL, a
ProviderException
is thrown, otherwise the URL and parameters are dispatched to the matching provider’sProvider.request()
method.If a cache was specified, the resulting metadata will be cached.
Parameters: - url – URL to retrieve metadata for
- extra_params – additional parameters to pass to the endpoint, for example a maxwidth or an API key.
Return type: a dictionary of JSON data
-
parse_text_full
(text[, urlize_all=True[, handler=full_handler[, urlize_params=None[, **params]]]])¶ Parse a block of text, converting all links by passing them to the given handler. Links contained within a block of text (i.e. not on their own line) will be handled as well.
Example input and output:
IN: 'this is a pic http://example.com/some-pic/' OUT: 'this is a pic <a href="http://example.com/some-pic/"><img src="http://example.com/media/some-pic.jpg" /></a>'
Parameters: - text (str) – a string to parse
- urlize_all (bool) – convert unmatched urls into links
- handler – function to use to convert metadata back into a string representation
- urlize_params (dict) – keyword arguments to be used to construct a link when a provider is not found and urlize is enabled.
- params – any additional parameters to use when requesting metadata, i.e. a maxwidth or maxheight.
-
parse_text
(text[, urlize_all=True[, handler=full_handler[, block_handler=inline_handler[, urlize_params=None[, **params]]]]])¶ Very similar to
parse_text_full()
except URLs on their own line are rendered using the givenhandler
, whereas URLs within blocks of text are passed to theblock_handler
. The default behavior renders full content for URLs on their own line (e.g. a video player), whereas URLs within text are rendered simply as links so as not to disrupt the flow of text.- URLs on their own line are converted into full representations
- URLs within blocks of text are converted into clickable links
Parameters: - text (str) – a string to parse
- urlize_all (bool) – convert unmatched urls into links
- handler – function to use to convert links found on their own line
- block_handler – function to use to convert links found within blocks of text
- urlize_params (dict) – keyword arguments to be used to construct a link when a provider is not found and urlize is enabled.
- params – any additional parameters to use when requesting metadata, i.e. a maxwidth or maxheight.
-
parse_html
(html[, urlize_all=True[, handler=full_handler[, block_handler=inline_handler[, urlize_params=None[, **params]]]]])¶ Parse HTML intelligently, rendering items on their own within block elements as full content (e.g. a video player), whereas URLs within text are passed to the
block_handler
which by default will render a simple link. URLs that are already enclosed within a<a>
tag are skipped over.- URLs that are already within <a> tags are passed over
- URLs on their own in block tags are converted into full representations
- URLs interspersed with text are converted into clickable links
Note
requires BeautifulSoup or beautifulsoup4
Parameters: - html (str) – a string of HTML to parse
- urlize_all (bool) – convert unmatched urls into links
- handler – function to use to convert links found on their own within a block element
- block_handler – function to use to convert links found within blocks of text
- urlize_params (dict) – keyword arguments to be used to construct a link when a provider is not found and urlize is enabled.
- params – any additional parameters to use when requesting metadata, i.e. a maxwidth or maxheight.
-
extract
(text, **params)¶ Extract all URLs from a block of text, and additionally get any metadata for URLs we have providers for.
Parameters: - text (str) – a string to parse
- params – any additional parameters to use when requesting metadata, i.e. a maxwidth or maxheight.
Return type: returns a 2-tuple containing a list of all URLs and a dict keyed by URL containing any metadata. If a provider was not found for a URL it is not listed in the dictionary.
-
extract_html
(html, **params)¶ Extract all URLs from an HTML string, and additionally get any metadata for URLs we have providers for.
extract()
but for HTML.Note
URLs within <a> tags will not be included.
Parameters: - html (str) – a string to parse
- params – any additional parameters to use when requesting metadata, i.e. a maxwidth or maxheight.
Return type: returns a 2-tuple containing a list of all URLs and a dict keyed by URL containing any metadata. If a provider was not found for a URL it is not listed in the dictionary.
-
-
micawber.providers.
bootstrap_basic
([cache=None[, registry=None]])¶ Create a
ProviderRegistry
and register some basic providers, including youtube, flickr, vimeo.Parameters: - cache – an object that implements simple
get
andset
- registry – a
ProviderRegistry
instance, which will be updated with the list of supported providers. If not specified, an emptyProviderRegistry
will be used.
Return type: a
ProviderRegistry
with a handful of providers registered- cache – an object that implements simple
-
micawber.providers.
bootstrap_oembed
([cache=None[, registry=None[, refresh=False[, **kwargs]]])¶ Create a
ProviderRegistry
and register as many providers as are described in the oembed.com providers list.Note
This function makes a request over the internet whenever it is called.
Parameters: - cache – an object that implements simple
get
andset
- registry – a
ProviderRegistry
instance, which will be updated with the list of supported providers. If not specified, an emptyProviderRegistry
will be used. - refresh (bool) – force refreshing the provider data rather than attempting to load it from cache (if cache is used).
- kwargs – any default keyword arguments to use with providers
Return type: a ProviderRegistry with support for noembed
- cache – an object that implements simple
-
micawber.providers.
bootstrap_embedly
([cache=None[, registry=None[, refresh=False[, **kwargs]]])¶ Create a
ProviderRegistry
and register as many providers as are supported by embed.ly. Valid services are fetched from http://api.embed.ly/1/services/python and parsed then registered.Note
This function makes a request over the internet whenever it is called.
Parameters: - cache – an object that implements simple
get
andset
- registry – a
ProviderRegistry
instance, which will be updated with the list of supported providers. If not specified, an emptyProviderRegistry
will be used. - refresh (bool) – force refreshing the provider data rather than attempting to load it from cache (if cache is used).
- kwargs – any default keyword arguments to use with providers, useful for specifying your API key
Return type: a ProviderRegistry with support for embed.ly
# if you have an API key, you can specify that here pr = bootstrap_embedly(key='my-embedly-key') pr.request('http://www.youtube.com/watch?v=54XHDUOHuzU')
- cache – an object that implements simple
-
micawber.providers.
bootstrap_noembed
([cache=None[, registry=None[, refresh=False[, **kwargs]]])¶ Create a
ProviderRegistry
and register as many providers as are supported by noembed.com. Valid services are fetched from http://noembed.com/providers and parsed then registered.Note
This function makes a request over the internet whenever it is called.
Parameters: - cache – an object that implements simple
get
andset
- registry – a
ProviderRegistry
instance, which will be updated with the list of supported providers. If not specified, an emptyProviderRegistry
will be used. - refresh (bool) – force refreshing the provider data rather than attempting to load it from cache (if cache is used).
- kwargs – any default keyword arguments to use with providers, useful for
passing the
nowrap
option to noembed.
Return type: a ProviderRegistry with support for noembed
# if you have an API key, you can specify that here pr = bootstrap_noembed(nowrap=1) pr.request('http://www.youtube.com/watch?v=54XHDUOHuzU')
- cache – an object that implements simple
Cache¶
-
class
micawber.cache.
Cache
¶ A reference implementation for the cache interface used by the
ProviderRegistry
.from micawber import Cache, bootstrap_oembed cache = Cache() # Simple in-memory cache. # Now our oembed provider will cache the responses for each URL we # request, which can provide a significant speedup. pr = bootstrap_oembed(cache=cache)
-
get
(key)¶ Retrieve the key from the cache or
None
if not present
-
set
(key, value)¶ Set the cache key
key
to the givenvalue
.
-
-
class
micawber.cache.
PickleCache
([filename='cache.db'])¶ A cache that uses pickle to store data.
Note
To use this cache class be sure to call
load()
when initializing your cache andsave()
before your app terminates to persist cached data.-
load
()¶ Load the pickled data into memory
-
save
()¶ Store the internal cache to an external file
-
-
class
micawber.cache.
RedisCache
([namespace='micawber'[, timeout=None[, **conn]]])¶ A cache that uses Redis to store data
Note
requires the redis-py library,
pip install redis
Parameters: - namespace – prefix for cache keys
- timeout (int) – expiration timeout in seconds (optional)
- conn – keyword arguments to pass when initializing redis connection