RedditRover - the Reddit Multibot Framework¶

The basic structure¶

RedditRovers plugins have three very important (and a fourth optional) sections:

1. Configuration File
2. The import of the base class.
3. An object that inherits from the base class and implements abstract methods.
4. A function that instantiates the object and returns it.
5. *(An external instantiating to test a plugin on your own.)

1. Config File¶

Every Bot needs a basic set of configuration. By standard, this is stored in config/bot_config.ini. To secure consistency you can use misc/ConfigWizard.py to setup a new configuration, otherwise we will append a section to the current bot configuration:

[MyBotName]
description=This scans for the occurrences of 'reddit'
is_logged_in=False

Note that this config is only needed if you’re using the standard config setup of the base class. More about builtin methods of the baseclass can be found in the code overview.

2. Importing the base class¶

RedditRover checks always if your plugin is inherited from the baseclass. It gives you a set of features and makes maintaining code really simple. First you import the base class:

from core.BaseClass import PluginBase

3. Object from PluginBase¶

Now comes the real part: Every plugin needs to inherit from the base class and has to implement a specific set of methods to function properly. This would look like this:

class MyPlugin(PluginBase):

    def __init__(self, database, handler):
        super().__init__(database, handler, 'MyBotName')

    def execute_comment(self, comment):
        if 'reddit' in comment.body.lower():
            self.logger.info('{} said reddit here: {}'.format(comment.author.name, comment.permalink))

    def execute_link(self, link_submission):
        pass

    def execute_titlepost(self, title_only):
        pass

    def execute_submission(self, submission):
        if 'reddit' in submission.selftext.lower():
            self.logger.info('{} said reddit here: {}'.format(submission.author.name, submission.permalink))

    def update_procedure(self, thing_id, created, lifetime, last_updated, interval):
        pass

    def on_new_message(self, message):
        pass

4. Function for instantiating the plugin¶

Next is an init-call to initialize the plugin, setup whatever you need to and return the object that inherits from BaseClass/PluginBase. This is a design decision to make it easy, run init-tasks and give a fixed return object back. Also you can define if you need access to the database storage for example.

All you need to do is following:

def init(database, handler):
    return MyPlugin(database, handler)

5. Test Block (optional)¶

And at last there is the optional test block. BaseClass/PluginBase features two functions to load a single submission or comment by id to test your bot against real world data and test cases. You can now execute the plugin itself.

if __name__ == '__main__':
    my_plugin = MyPlugin(None, None, 'MyBotName')  # Remember: We don't always need the database.
    my_plugin.test_single_submission('3iyxxt')  # See: https://redd.it/29f2ah
    my_plugin.test_single_comment('cukvign')  # See:

About PRAW objects¶

I cannot teach you how to program or how to use PRAW objects to its fullest, but I can give you a good hint. In general it’s a good advice lookup all steps in the python console or in iPython. A close look at PRAWs objects is helpful too.

>>> from praw import Reddit
>>> r = Reddit(user_agent='Some user agent for you.')
>>> comment = r.get_info(thing_id='t1_cukvign')
>>> submission = r.get_info(thing_id='t3_3iyxxt')
>>> dir(comment)
>>> dir(submission)
>>> comment.author
>>> submission.author

The entire code¶

In case you struggle with assembling the code, here is it as full set:

from core.BaseClass import PluginBase


class MyPlugin(PluginBase):

    def __init__(self, database, handler):
        super().__init__(database, handler, 'MyBotName')

    def execute_comment(self, comment):
        if 'reddit' in comment.body.lower():
            author = ('[unknown]', submission.author.name)[submission.author is True]
            self.logger.info('{} said reddit here: {}'.format(comment.author.name, comment.permalink))

    def execute_link(self, link_submission):
        pass

    def execute_titlepost(self, title_only):
        pass

    def execute_submission(self, submission):
        if 'reddit' in submission.selftext.lower():
            author = ('[unknown]', submission.author.name)[submission.author is True]
            self.logger.info('{} said reddit here: {}'.format(author, submission.permalink))

    def update_procedure(self, thing, created, lifetime, last_updated, interval):
        pass

    def on_new_message(self, message):
        pass


def init(database, handler):
    return MyPlugin(database, handler)


if __name__ == '__main__':
    my_plugin = MyPlugin(None, None)  # Remember: We don't always need the database.
    my_plugin.test_single_submission('3iyxxt')  # See: https://redd.it/29f2ah
    my_plugin.test_single_comment('cukvign')  # See:

Framework Config¶

The framework currently features no real configuration options. They’re hinted at and planned inside config/config.ini, but they have to be implemented first.

Plugin Config¶

Every plugin needs to have a base set of configuration (if you use the standard config options), which have to be present in a particular way:

[EXAMPLE]
description = Here could be your description.
is_logged_in = True
self_ignore = True
username = YourUsername
oauth_file = YourBot_OAuth.ini

If your bot is not logged in, you can ignore the values self_ignore, username and oauth_file.

Other than that you can use any variable in this section as you please, i. e. storing response strings. The normally supplied attribute config in every plugin can be used to load those variables.

What does RedditRover offer?¶

RedditRover is an easy plugin based bot framework for reddit, reading the most recent submissions and comments. It features a rich set of features to keep the bot completely automatic, self aware and consistent between sessions. The framework is documented, easily extensible and features no magic tricks. You can interface with the database however you please and let the framework revisit generated comments and submissions to update them later. It is certainly faster as PRAW, due to the Handler limitations fixed within this bot.

What are use cases of RedditRover?¶

Everything you would want to do with fresh comments and submissions. Fixing links, remind people, react on messages or name calling, cross referencing, scrape data, analyzing users - even generating live statistics is easily feasible. You’re not limited within a plugin what you do or how complicated the execution on your plugin is. The framework just provides you a lot of data live from Reddit.

What assumptions does this framework make?¶

403 Forbidden: If you’re trying to submit content and the API responds with 403 Forbidden, the framework assumes that this account is banned or not approved in that subreddit and will set it on the active ban list, effectively ignoring all submissions or comments from now on.

APIException: deleted link, InvalidSubmissions: Sometimes content gets faster deleted than the bot can react to it (usually a 2-3 second margin) and it is impossible to post on those submissions, the bot will throw a warning and skip it.

HTTPException: Reddits API is accessed a lot and is known to have regular issues. If the bot encounters one of those it will either retry or simply send a warning and skip the submission. The framework catches HTTPExceptions and retries the execution on its own and you can annotate your functions with the retry-decorator and catch specific exceptions, to literally retry after a few seconds.

How does a plugin work?¶

The most important details are discussed on the ‘Writing a Plugin’ <pages/writing_a_plugin>_ page. Again, it is important to inherit from the PluginBase in order to function properly. ExampleBot.py gives a good insight.

Requests, API limitations, slow framework¶

Reddit has strict API limits:

• Not logged in / Auth-cookie sessions: 30 requests per minute per IP
• OAuthed sessions: 60 requests per minute per session

There is a customized RoverHandler to take care exactly of these limitations. However, since PRAW is based on lazy loading you want to take care of how many requests a plugin produces. In a regular case RedditRover caches a thousand individual comments and submission. If your plugin triggers on every submission and comment, you will slow down the framework so intensively, that you won’t be able to react on every instance. All working threads get a sleep until it can dispatch, send and receive a request. To see a detailed log of requests, there is a logfile within /logs/[year]/[month]. If you see a lot oauth-requests you can assume that you have a problem with lazy loading.

My plugin missed a post¶

There is a slight blindness, especially when Reddit is very busy and/or their API has issues. Also some subreddits do get hidden from /r/all. As of recent, this isn’t only a subreddit setting as some communities (/r/4chan for example) get automatically removed from /r/all. Monitoring a specific subreddit or a multireddit is however possible. As soon as the framework doesn’t monitor all subreddits anymore, the blindness reduces to near zero. Current tests have shown that the current miss chance under normal conditions is under 0.005%. This is a problem with Reddit not PRAW or the framework.

Do you accept donations?¶

Nope. This is a training in software architecture and engineering for me and also a contribution to early programmers who would like to see how you’d scrape big data from big communities. However, I’d love to add your contribution to the project.

PluginBase Group¶

The BaseClass is a vital class for the framework. It ensures a given set of features while staying as flexible as possible. It also gives an extended feature set which helps with basic tasks necessary with every bot, i. e. logging in a bot into reddit, providing the standard config or testing against single, manually input threads and comments.

class core.baseclass.PluginBase(database, handler, bot_name, setup_from_config=True)[source]¶

PluginBase is that basis of every plugin, ensures the functionality of a plugin and has already methods for handling a lot of basic functions. The abstract methods in this plugin have to be overwritten, otherwise it won’t import and will not work at all.

Attributes with an asterisk (*) do not have to be set without being logged in

Methods with an hashtag (#) have to be implemented and overwritten by your plugin.

Variables:

DESCRIPTION (str) – user_agent: describes the bot / function / author USERNAME – *The username which should’ve been logged in, will be verified after logging in. BOT_NAME (str) – Give the bot a nice name IS_LOGGED_IN (bool) – Representing if a plugin needs a login or not. SELF_IGNORE (bool) – *Decides if comments from the same username as the plugin are automatically skipped. OA_ACCESS_TOKEN (str) – *Access token for every requests. Gets automatically fetched and refreshed with oa_refresh(force=False) OA_REFRESH_TOKEN (str) – *Refresh token which gets queried the first time a plugin is initialized, otherwise loaded. OA_APP_KEY (str) – *OAuth Application Key, which has to be set in the config. OA_APP_SECRET (str) – *OAuth Secret Key, which has to be set in the config. OA_TOKEN_DURATION (int | float) – *OAuth Token validation timer. Usually set to 59minutes to have a good error margin OA_VALID_UNTIL (int | float) – *Determines how long the OA_ACCESS_TOKEN is valid as timestamp. Gets refreshed by oa_refresh(force=False session (praw.Reddit) – A session which is used to interface with Reddit. logger (logging.Logger) – Logger for this specific plugin. config (ConfigParser) – ConfigParser loaded for that plugin. Can access all other sections and variables as well. database (Database | None) – Session to database, can be None if not needed. handler (RedditRoverHandler) – Specific handler given from the framework to keep API rate limits

integrity_check()[source]¶

Checks if the most important variables are initialized properly.

Returns:	True if possible
Return type:	bool
Raise:	AssertionError

static factory_logger()[source]¶

Returns a Logger named ‘plugin’.

Returns:	Unchilded Logger ‘Plugin’
Return type:	logging.Logger

factory_reddit(login=False)[source]¶

Sets the class attribute ‘session’ to a Reddit session and authenticates with it if the parameter is set.

Parameters:	login (bool) – Decides if you want the session logged in or not.
Raise:	AssertionError

static factory_config()[source]¶

Sets up a standard config-parser to bot_config.ini. Does not have to be used, but it is handy.

Returns:	Set up ConfigParser object, reading /config/bot_config.ini.
Return type:	ConfigParser

_get_keys_manually()[source]¶

Method to get Access and Refresh Keys manually. Has to be run through once when the credentials are set up, writes then the refresh key into the config and logs the session in.

add_comment(thing_id, text)[source]¶

Add a comment to the current thing.

Parameters:	thing_id (str) – Either a comment or a submission id to comment on. text (str) – Comment text
Returns:	praw.objects.Comment from the responded comment.

_oa_refresh(force=False)[source]¶

Main function to refresh OAuth access token.

Parameters:	force (bool) – Forces to refresh the access token

oa_refresh(force=False)[source]¶

Calls _oa_refresh and tries to reset OAuth credentials if it fails several times.

Parameters:	force (bool) – Forces to refresh the access token

get_unread_messages(mark_as_read=True)[source]¶

Runs down all unread messages of this logged in plugin and if wanted, marks them as read. This should always the case, otherwise a verbose bot with many messages has to run down a long list of mails every time the bot gets rebooted.

Parameters:	mark_as_read (bool) – Decides if the all messages get marked as read (speeds up the message reading every time)

standard_ban_procedure(message, subreddit_banning_allowed=True, user_banning_allowed=True)[source]¶

An exemplary method that bans users and subs and then replies them that the bot has banned. Needs a reddit session, oauth and a database pointer to function properly.

Parameters:	message (praw.objects.Message) – a single praw message object subreddit_banning_allowed (bool) – can block out the banning of subreddits user_banning_allowed (bool) – can block out the banning of users

_test_single_thing(thing_id)[source]¶

If you feel confident enough to mangle with thing_ids of Reddit, you can use this method to load a specific thing to test out your bot. The plugin will behave exactly as it would get the same thing from the framework.

Parameters:	thing_id (str) – The direct thing_id from Reddit, which you can get by looking into the permalink.

test_single_submission(submission_id)[source]¶

Use this method to test your plugin manually for single submissions. Behaves like it would in the framework.

Parameters:	submission_id (str) – Needs the id from Reddit, which you can get from the permalink: https://redd.it/3iyxxt

test_single_comment(comment_id)[source]¶

Use this method to test your plugin manually for single comments. Behaves like it would in the framework.

Parameters:	comment_id (str) – Needs the id from Reddit, which you can get from the permalink: https://reddit.com/comments/3iyxxt/_/cukvign

to_update(response_object, lifetime, interval)[source]¶

This method is preferred if you want a submission or comment to be updated. It writes the important information into the database, which later will get queried into the

Parameters:	response_object (praw.objects.Submission | praw.objects.Comment) – PRAW returns on a posted submission or comment the resulting object. lifetime (int) – The exact moment in unixtime in seconds when this object will be invalid (update cycle)
Params interval:
	The interval after what time of updating this should be queried again.

execute_submission(submission)[source]¶

# Function for handling a submission with a textbody (self.post)

Parameters:	submission (praw.objects.Submission) – A submission with a title and textbody.
Returns:	True if the plugin reacted on in, False or None if he didn’t.
Return type:	bool | None

execute_link(link_submission)[source]¶

# Function for handling a link submission.

Parameters:	link_submission (praw.objects.Submission) – A submission with title and an url.
Returns:	True if the plugin reacted on in, False or None if he didn’t.
Return type:	bool | None

execute_titlepost(title_only)[source]¶

# Function for handling a link submission.

Parameters:	title_only (praw.objects.Submission) – A submission with only a title. No textbody nor url.
Returns:	True if the plugin reacted on in, False or None if he didn’t.
Return type:	bool | None

execute_comment(comment)[source]¶

# Function for handling a comment.

Parameters:	comment (praw.objects.Comment) – A single comment. :warn: Comments can have empty text bodies.
Returns:	True if the plugin reacted on in, False or None if he didn’t.
Return type:	bool | None

update_procedure(thing, created, lifetime, last_updated, interval)[source]¶

# Function that gets called from the update thread when a previously saved thread from self.to_update reached its next interval.

Parameters:	thing (praw.objects.Submission | praw.objects.Comment) – concrete loaded Comment or Submission created (float) – unix timestamp when this thing was created. lifetime (float) – unix timestamp until this update-submission expires. last_updated – unix timestamp when it was updated the last time. interval (int) – interval in seconds how often this thing should be updated.

on_new_message(message)[source]¶

# Method gets called when there is a new message for this plugin.

Parameters:	message (praw.objects.Message) – Message Object from PRAW, contains author, title, text for example.

RedditRover Group¶

class core.redditrover.RedditRover[source]¶

Bases: object

Reddit Rover object is the full framework. When initing, it reads all plugins, initializes them and starts loading submissions and comments from Reddit. Varying from your implementation, it will fire those submissions and comments to your bot. Based on the configuration setting the bot will run a maintenance and update procedure, cleaning up the database and rolling over submissions / comments of a plugin which requested to update from there out on.

Variables:

logger (Logger) – Central bot logger. config (ConfigParser) – Holds a full set of configs from the configfile. responders (list) – A list of plugins the bot is running. @TODO: Exit the bot if no plugin is found. multi_thread (MultiThreader) – The MultiThreader instance, which manages daemonic threads. delete_after (int) – All activity older than x seconds will be cleaned up from the database. verbose (bool) – True if heavily verbose, false if not. update_interval (int) – Sets an interval on the update-thread, cleaning the DB, reading messages and running updates catch_http_exception (bool) – True if HTTP exceptions get automatically catched, False if not. mark_as_read (bool) – True if all messages worked through are marked as read. praw_handler (RedditRoverHandler) – Will hold the handler to RateLimit based on OAuth / No-Auth sessions. submission_poller (praw.Reddit) – Anonymous reddit session for submissions. comment_poller (praw.Reddit) – Anonymous reddit session for comments. submissions (praw.helpers.comment_stream) – Generator of recent submissions on Reddit. comments (praw.helpers.comment_stream) – Generaot of recent comments on Reddit.

load_responders()[source]¶

Loads all plugins from ./plugins/, appends them to a list of responders and verifies that they’re properly setup and working for the main bot process.

submission_thread()[source]¶

The submission thread runs down all submission from the specified sub (usually /r/all), then filters out all banned users and subreddits and then fires submissions at your plugins.

comment_thread()[source]¶

The comment thread runs down all comments from the specified sub (usually /r/all), then filters out banned users and subreddits and fires it at your plugins.

comment_submission_worker(thing)[source]¶

Runs through all available plugins, filters them based on that out and calls the right method within a plugin.

Parameters:	thing (praw.objects.Comment | praw.objects.Submission) – Single submission or comment

comment_submission_action(thing, responder)[source]¶

Separated function to run a single submission or comment through a single comment.

Parameters:	thing (praw.objects.Submission | praw.objects.Comment) – single submission or comment responder (PluginBase) – single plugin
Returns:

update_thread()[source]¶

The update-thread does a lot of different tasks. First it loads all threads that have to update and executes the update_procedure of your plugin. Then it loads all unread messages of your plugin, cleans up the database and sleeps for 5 minutes.

update_action(thread, responder)[source]¶

Separated function to map a thing to update and feed it back into a plugin.

Parameters:	thread (tuple) – A tuple containing information from the database. responder (PluginBase) – A single plugin

Database Group¶

This is currently heavily work in progress.

class core.database.Database[source]¶

Bases: object

This object provides a full set of features to interface with a basic session database,

which includes following tables:

• storage: saves the state of the bot and helps against double posting
• update_threads: storage to store thing_ids which have to be updated by your plugin
• modules: persistent module storage
• userbans: a table to ban users from being able to trigger certain plugins
• subbans: a table to ban subreddits from being able to trigger certain plugins

Variables:	logger (logging.Logger) – A database specific database logger. Is currently missing debug-messages for database actions. db (sqlite3.Connection) – A connection to the SQLite database: /config/storage.db cur (sqlite3.Cursor) – Cursor to interface with the database. _meta_push (dict) – Dictionary with helper methods to reduce the amount of requests for meta tables

:ivar _MAX_CACHE = maximum content within the _meta_push dictionary to get pushed into the database. :type _MAX_CACHE: int :vartype _MAX_CACHE: int

database_init()[source]¶

Initialized the database, checks manually (because: why not?) if those tables already exist and if not, creates the necessary tables. You can modify the PRAGMA or add tables however you please, as long as you keep the order of these tables (their columns) intact. Some SQL statements are not completely explicit to be independent on order.

_database_check_if_exists(table_name)[source]¶

Helper method to check if a certain table (by name) exists. Refrain from using it if you’re not adding new tables. :param table_name: Name of the table you want to check if it exists. :type table_name: str :return: Tuple of the table name, empty if it doesn’t exist.

insert_into_storage(thing_id, module)[source]¶

Stores a certain thing (id of comment or submission) into the storage, which is for the session consistency.

Parameters:	thing_id (str) – Unique thing_id from a comment or submission. module (str) – A string naming your plugin.

get_all_storage()[source]¶

Returns all elements inside the bot storage.

Returns:	Tuple with tuples with all storage elements with (thing_id, module_name, timestamp)

retrieve_thing(thing_id, module)[source]¶

Returns a single thing from the storage by thing_id and module name. Mainly used to check if a plugin already answered on a post.

Parameters:	thing_id (str) – Unique thing_id from a comment or submission. module (str) – A string naming your plugin.
Returns:	Tuple with (thing_id, bot_module, timestamp)

delete_from_storage(min_timestamp)[source]¶

Deletes all items which are older than the given timestamp.

Parameters:	min_timestamp (int | float) – Unix timestamp where all entries in storage get deleted if they’re older than that.

select_from_storage(older_than_timestamp)[source]¶

Selects and retrieves all elements in the storage which are older than this timestamp.

Parameters:	older_than_timestamp (int | float) – Unix timestamp of which time everything has to be selected before.
Returns:	Tuples of (thing_id, bot_module, timestamp)

insert_into_update(thing_id, module, lifetime, interval)[source]¶

Inserts a thing_id (from a comment or submission) into the update-table, which later gets retrieved from the update-thread and fired onto the plugin.

Parameters:	thing_id (str) – Unique thing_id from a comment or submission. module (str) – A string naming your plugin. lifetime (float | int) – Lifetime until this item is valid in Unix timestamp. interval (int) – Interval of how often you’d want this to update in seconds.

get_all_update()[source]¶

Returns all elements inside the update_htreads table.

Returns:	Tuple with tuples of (thing_id, module_name, created, lifetime, last_updated, interval)

_select_to_update(module)[source]¶

Selector method to get the cursor selecting all outstanding threads to update for a certain module. Refrain from using it, since it only places the cursor.

Parameters:	module (str) – A string naming your plugin.

get_latest_to_update(module)[source]¶

Returns a single thing_id (from comment or submssion) for a single module.

Parameters:	module (str) – A string naming your plugin.
Returns:	Tuple with tuples of (thing_id, module_name, created, lifetime, last_updated, interval)

get_all_to_update(module)[source]¶

Returns all thing_ids (from a comment or submission) for a module.

Parameters:	module (str) – A string naming your plugin.
Returns:	Tuple with tuples of (thing_id, module_name, created, lifetime, last_updated, interval)

update_timestamp_in_update(thing_id, module)[source]¶

Updates the timestamp when a thing_id was updated last.

Parameters:	thing_id (str) – Unique thing_id from a comment or submission. module (str) – A string naming your plugin.

delete_from_update(thing_id, module)[source]¶

Deletes all thing_ids (from a comment or submission) for a module when it outlived its lifetime.

Parameters:	thing_id (str) – Unique thing_id from a comment or submission. module (str) – A string naming your plugin.

register_module(module)[source]¶

Registers a module if it hasn’t been so far. A module has to be registered to be useable with the rest of the database. :param module: A string naming your plugin. :type module: str

get_all_userbans()[source]¶

Returns all bans stored in the userban table. :return: Tuple of tuples (username, bot_module)

get_all_bans_per_user(username)[source]¶

Returns all bans of a particular user across all plugins. :param username: Author in fulltext in question :type username: str :return: Tuple of tuples (username, bot_module)

check_user_ban(username, module)[source]¶

Checks if a particular user has been banned, first searches per module, then if there is a global ban.

Parameters:	username (str) – Author in fulltext in question module (str) – A string naming your plugin.
Returns:	Boolean if banned or not.

add_userban_per_module(username, module)[source]¶

Bans a user for a certain module.

Parameters:	username (str) – Author in fulltext in question module (str) – A string naming your plugin.

add_userban_globally(username)[source]¶

Bans a user for all modules.

Parameters:	username (str) – Author in fulltext in question

remove_userban_per_module(username, module)[source]¶

Removes a ban from a certain modules.

Parameters:	username (str) – Author in fulltext in question module (str) – A string naming your plugin.

remove_userban_globally(username)[source]¶

Removes all bans for a user. Globally and per module level.

Parameters:	username (str) – Author in fulltext in question

purge_all_user_bans()[source]¶

Removes all bans for all users - no exception, clears the entire table.

get_all_banned_subreddits()[source]¶

Returns all bans stored in the subreddit ban table

get_all_bans_per_subreddit(subreddit)[source]¶

Returns all bans for a particular subreddit :param subreddit: Author in fulltext in question :type subreddit: str

check_subreddit_ban(subreddit, module)[source]¶

Returns if a certain subreddit is banned from a module or across all modules.

Parameters:	subreddit (str) – Author in fulltext in question module (str) – A string naming your plugin.
Returns:	Boolean, True if banned, False if not.

add_subreddit_ban_per_module(subreddit, module)[source]¶

Bans a subreddit from a certain module.

Parameters:	subreddit (str) – Author in fulltext in question module (str) – A string naming your plugin.

add_subreddit_ban_globally(subreddit)[source]¶

Bans a subreddit across all subreddits.

Parameters:	subreddit (str) – Author in fulltext in question

remove_subreddit_ban_per_module(subreddit, module)[source]¶

Removes a subreddit ban for a certain module

Parameters:	subreddit (str) – Author in fulltext in question module (str) – A string naming your plugin.

remove_subreddit_ban_globally(subreddit)[source]¶

Removes a subreddit ban across all modules and globally

Parameters:	subreddit (str) – Author in fulltext in question

purge_all_subreddit_bans()[source]¶

Removes all subreddit bans from the table - no exceptions, clears the table.

_check_if_module_exists(module)[source]¶

Helper method to determine if a module has already been registered. Refrain from using it, hence it is private.

Parameters:	module (str) – A string naming your plugin.
Returns:	Boolean determining if a module already has been registered.
Raises ValueError:
	In case of a module being registered multiple times - which should never happen - the Database object will raise a value error.

_error_if_not_exists(module)[source]¶

Helper method for throwing a concrete error if a module has not been registered, yet tries to write into the database without having a reference.

Parameters:	module (str) – A string naming your plugin.
Raises LookupError:
	If the module doesn’t exist, it raises an error.

get_all_modules()[source]¶

Returns all modules that have been registered so far.

Returns:	Tuple of tuples (_ROWID_, module_name)

clean_up_database(older_than_unixtime)[source]¶

Cleans up the database, meaning that everything older than the session time and all threads that should be updated and outlived their lifetime will be deleted.

Parameters:	older_than_unixtime (int | float) – Unix timestamp from which point entries have to be older than to be deleted.

wipe_module(module)[source]¶

Wipes a module across all tables and all its references.

Parameters:	module (str) – A string naming your plugin.

add_to_stats(id, bot_name, title, username, subreddit, permalink)[source]¶

Adds a row to the stats, see params (is handled by RedditRover).

Parameters:	id (str) – submission or comment id bot_name (str) – Plugin Name title (str) – Title of original submission username (str) – Original Author of responded submission subreddit (str) – Subreddit Name of submission permalink (str) – Permalink to comment or submission the bot has responded upon

get_all_stats()[source]¶

Returns a tuple of tuple, be warned: upvotes_author and upvotes_bot can both be null.

Returns:	Tuple of tuples: (thing_id, module_name, created, title, username, subreddit, upvotes_author, upvotes_bot)

get_total_responses_per_day(timestamp)[source]¶

Gets the total amount of rows for a day. The timestamp has to be in that day to work.

Parameters:	timestamp (int | float) – Unix timestamp of day
Returns:	Tuple with (amount of rows,)

get_karma_loads()[source]¶

Returns a tuple with IDs for karma statistics.

Returns:	Tuple with (id,)

update_karma_count(thing_id, author_upvotes, plugin_upvotes)[source]¶

Updates the karma count for a previously stored response.

Parameters:	thing_id (str) – id of submission a plugin has responded on author_upvotes (int) – Amount of upvotes from the author plugin_upvotes (int) – Amount of upvotes from the plugin

update_karma_count_with_null(thing_id, author_upvotes)[source]¶

Updates only author_upvotes, sometimes plugin responses are already deleted.

Parameters:	thing_id (str) – id of submission a plugin has responded on author_upvotes (int) – Amount of upvotes from the author

add_message(msg_id, bot_module, created, username, title, body)[source]¶

Upon receiving a message, its contents will be stored in a table for statistical purposes and overview of all plugins inboxes.

Parameters:	msg_id (str) – Unique message id from reddit. bot_module (str) – Plugins Name created (int | float) – Unix timestamp of messages arrival username (str) – Original author of the message title (str) – Subject of said message body (str) – Text body of this message.

get_all_messages()[source]¶

Returns all messages in the messages table.

Returns:	Tuple of tuples: (id, module_name, created, title, author, body)

select_day_from_meta(timestamp)[source]¶

Returns a certain day from the meta_stats.

Parameters:	timestamp (int | float) – Unix timestamp from a certain day. Has to be within that day.
Returns:	Tuple of (day, seen_submissions, seen_comment, update_cycles)

add_submission_to_meta(count, force=False)[source]¶

Increases the submission count for this day in a cached fashion.

Parameters:	count (int) – Increases current count by this count. force (bool) – Forces the write out into the database.

add_comment_to_meta(count, force=False)[source]¶

Increases the comment count for this day in a cached fashion.

Parameters:	count (int) – Increases current count by this count. force (bool) – Forces the write out into the database.

add_update_cycle_to_meta(count, force=False)[source]¶

Increases the update cycle count for this day in a cached fashion

Parameters:	count (int) – Increases current count by this count. force (bool) – Forces the write out into the database.

_write_out_meta_push()[source]¶

Writes out the values in the meta cache. Reduces the amount of DB requests by a major amount.

write_out_meta_push(force=False)[source]¶

Checks if the meta cache has to be written - or can be forced.

Parameters:	force (bool) – Forces the write out

_add_submission_to_meta(count, timestamp)[source]¶

Increases the submission count for a day.

Parameters:	count (int) – Amount of which it should be increased. timestamp (int | float) – Timestamp that lies in that day it should be increased to.

_add_comment_to_meta(count, timestamp)[source]¶

Increases the comment count for a day.

Parameters:	count (int) – Amount of which it should be increased. timestamp (int | float) – Timestamp that lies in that day it should be increased to.

_add_update_cycle_to_meta(count, timestamp)[source]¶

Increases the update cycle count for a day.

Parameters:	count (int) – Amount of which it should be increased. timestamp (int | float) – Timestamp that lies in that day it should be increased to.

LogProvider Group¶

core.logprovider.setup_logging(log_level='INFO', console_log_level=None, log_path_format='logs/%Y/%m/%Y-%m-%d.log', web_log_path=None)[source]¶

Thanks to Renol: https://github.com/RenolY2/Renol-IRC-rv2 - This logging handler is quite powerful and nicely formatted. This sets up the main Logger and needed to receive bot and plugin messages. If you’re testing a single plugin it is recommended to execute this.

Parameters:	log_level (str) – Level on which the logger operates console_log_level (str) – Determines the console log level, which is usually the same as log_level log_path_format (str) – Path-Format for /logs/, supports sub folders

class core.logprovider.DailyRotationHandler(pathformat='logs/%Y/%m/%Y-%m-%d.log', utc=False, encoding=None, delay=False)[source]¶

Bases: logging.handlers.BaseRotatingHandler

This handler swaps over logs after a day. Quirky method names result from inheriting.

Variables:	path_format (str) – Path-Format for /logs/, supports sub folders utc (bool) – Timestamp in utc

shouldRollover(record)[source]¶

Checks if a rollover has to be done

Parameters:	record – Logger message
Returns:	Boolean to determine if or if not.
Return type:	bool

doRollover()[source]¶

Does the rollover.

class core.logprovider.MaxFileHandler(filename, max_len=1500, buffer_len=1400, mode='a+', encoding=None, delay=False)[source]¶

Bases: logging.FileHandler

emit(record)[source]¶

MultiThreader Group¶

class core.multithreader.MultiThreader[source]¶

Bases: object

The MultiThreader object has simple syntax to keep many processes in simple threads that are daemonic, therefore will be killed when the main process is killed too.

Variables:	threads (list) – A list of all threads to coordinate them. lock – A Lock that can be acquired from all instances to share that specific thread lock.

go(*args)[source]¶

Main method to get all threads together. First you call go, then join_threads. The arguments for this have to be a list, where [function, arguments]

Parameters:	args – All threads planned to be threaded.

join_threads()[source]¶

Join all threads given by go in an interruptable fashion.

get_lock()[source]¶

Returns the Lock object for this instance and main thread.

Returns:	threading.Lock

Handlers Group¶

class core.handlers.RoverHandler[source]¶

Bases: object

Main handler for RedditRover, keeps track of all requests sent by OAuth and non-Auth sessions to keep the API-Limit from Reddit in place. The general rule: All non-auth have 30 requests per minute per IP, all OAuth session have 60 requests per minutes without an IP limitation.

Variables:

logger (logging.Logger) – Logger for the handler, mainly writes debug messages into the log-file. no_auth (float) – A unix timestamp of the last planned requests. Stacks up and could be in the future (so further requests get sent after that. First come, first serve.) oauth (dict) – A dictionary with {'unique request token': [last_sent, lifetime]} http (requests.Session) – the requesting http session rl_lock (threading.Lock) – Threading Lock to lock up a requesting thread to update the timestamps on all requests, makes it all thread safe and threads stack each other up.

classmethod evict(urls)[source]¶

Method utilized to evict entries for the given urls. By default this method returns False as a cache need not be present.

Parameters:	urls (list) – An iterable containing normalized urls.
Returns:	The number of items removed from the cache.

request(request, proxies, timeout, verify, **_)[source]¶

Cleans up the oauth attribute, then looks up if its an OAuth requests and dispatched the request in the appropriate time. Sleeps the thread for exactly the time until the next request can be sent.

Parameters:	request – A requests.PreparedRequest object containing all the data necessary to perform the request. proxies – A dictionary of proxy settings to be utilized for the request. timeout – Specifies the maximum time that the actual HTTP request can take. verify – Specifies if SSL certificates should be validated.

send_request(request, proxies, timeout, verify)[source]¶

Responsible for dispatching the request and returning the result. Network level exceptions should be raised and only requests.Response should be returned.

**_ should be added to the method call to ignore the extra arguments intended for the cache handler.

Parameters:	request – A requests.PreparedRequest object containing all the data necessary to perform the request. proxies – A dictionary of proxy settings to be utilized for the request. timeout – Specifies the maximum time that the actual HTTP request can take. verify – Specifies if SSL certificates should be validated.

static dispatch_timer(next_possible_dispatch)[source]¶

Method to determine when the next request can be dispatched. :param next_possible_dispatch: Timestamp of the next possible dispatch. :type next_possible_dispatch: int | float :return: int | float

misc.multiple_strings module¶

misc.multiple_strings.multiple_of(value, string_of_single, string_of_multiple, return_with_value=False, before_string='', after_string='')[source]¶

Helps with english. If the word has a form of multiple, you can feed it’s value and it returns you the appropriate form. (Also has some extra features, in case you have a collection of strings.)

Parameters:	value (bool | float | int) – Numerical value string_of_single (str) – String representation of a single thing string_of_multiple (str) – String representation of multiple things return_with_value (bool) – Should it return with the value formatted? before_string (str) – String content before after_string (str) – String content after
Returns:	Complete string representation
Return type:	str

misc.warning_filter module¶

misc.warning_filter.ignore()[source]¶

Suppresses some warnings of praw.

Module contents¶