Welcome to osfclient’s documentation!¶
The osfclient is a python library and a command-line client for up-
and downloading files to and from your Open Science Framework
projects. The Open Science Framework (OSF) is an open source project
which facilitates the open collaboration of researchers on the web, by
sharing data and other research outputs.
As such the OSF hosts large data sets, associated with papers or scientific projects, that can be freely downloaded. The osfclient allows people to store and retrieve large datasets associated to their scientific projects and papers on the OSF via the command line interface. If you are completely new to the OSF you can read the OSF introductory materials.
This is a very new project, it has some rough edges. You can find the code for
the osfclient in the GitHub repository.
User’s Guide¶
This project provides two things: a python library and a command-line program for interacting with files stored in the OSF.
The python library forms the basis for the command-line program. If you want programmatic access to your files use the library, otherwise try out the command-line program.
Below some examples on how to use the command-line program:
# getting help
$ osf -h
$ osf <command> -h
# list all files for a public project
$ osf -p <projectid> ls
# setup a local folder for an existing project
$ osf init
# list all files for a private project
# set $OSF_PASSWORD to provide the password
$ osf -p <projectid> -u yourOSFacount@example.com ls
# fetch all files from a project and store them in `output_directory`
$ osf -p <projectid> clone [output_directory]
# create a new file in an OSF project
$ osf -p <projectid> -u yourOSFacount@example.com upload local/file.txt remote/path.txt
# download a single file from an OSF project
$ osf -p <projectid> fetch remote/path.txt local/file.txt
# upload a single file to an OSF project
$ osf -p <projectid> upload local/path.txt remote/file.txt
# remove a single file from an OSF project
$ osf -p <projectid> remove remote/file.txt
If the project is private you will need to provide authentication
details. You can provide your OSF account name as command-line argument
(see the osf upload example) or set the OSF_USERNAME environment
variable. The password will be retrieved from the OSF_PASSWORD
environment variable or asked directly by the tool.
You can set a default values by using a configuration file in the
current directory. To set the username and project ID create
.osfcli.config:
[osf]
username = yourOSFaccount@example.com
project = 9zpcy
after which you can simply run osf ls to list the contents of the project.
API Reference¶
Client library for the Open Science Framework
-
class
osfclient.OSF(username=None, password=None, token=None)¶ Interact with the Open Science Framework.
This is the main point of contact for interactions with the OSF. Use the methods of this class to find projects, login to the OSF, etc.
-
login(username, password=None, token=None)¶ Login user for protected API calls.
-
password¶
-
project(project_id)¶ Fetch project project_id.
-
username¶
-
osfclient.cli¶
Command line interface to the OSF
These functions implement the functionality of the command-line interface.
-
osfclient.cli.clone(args)¶ Copy all files from all storages of a project.
The output directory defaults to the current directory.
If the project is private you need to specify a username.
-
osfclient.cli.config_from_env(config)¶
-
osfclient.cli.config_from_file()¶
-
osfclient.cli.fetch(args)¶ Fetch an individual file from a project.
The first part of the remote path is interpreted as the name of the storage provider. If there is no match the default (osfstorage) is used.
The local path defaults to the name of the remote file.
If the project is private you need to specify a username.
-
osfclient.cli.init(args)¶ Initialize or edit an existing .osfcli.config file.
-
osfclient.cli.list_(args)¶ List all files from all storages for project.
If the project is private you need to specify a username.
-
osfclient.cli.might_need_auth(f)¶ Decorate a CLI function that might require authentication.
Catches any UnauthorizedException raised, prints a helpful message and then exits.
-
osfclient.cli.remove(args)¶ Remove a file from the project’s storage.
The first part of the remote path is interpreted as the name of the storage provider. If there is no match the default (osfstorage) is used.
-
osfclient.cli.upload(args)¶ Upload a new file to an existing project.
The first part of the remote path is interpreted as the name of the storage provider. If there is no match the default (osfstorage) is used.
If the project is private you need to specify a username.
To upload a whole directory (and all its sub-directories) use the -r command-line option. If your source directory name ends in a / then files will be created directly in the remote directory. If it does not end in a slash an extra sub-directory with the name of the local directory will be created.
To place contents of local directory foo in remote directory bar/foo: $ osf upload -r foo bar To place contents of local directory foo in remote directory bar: $ osf upload -r foo/ bar
osfclient.utils¶
Utility functions
Helpers and other assorted functions.
-
osfclient.utils.file_empty(fp)¶ Determine if a file is empty or not.
-
osfclient.utils.makedirs(path, mode=511, exist_ok=False)¶
-
osfclient.utils.norm_remote_path(path)¶ Normalize path.
All remote paths are absolute.
-
osfclient.utils.split_storage(path, default='osfstorage')¶ Extract storage name from file path.
If a path begins with a known storage provider the name is removed from the path. Otherwise the default storage provider is returned and the path is not modified.
osfclient.models¶
Classes to model entities in the OSF API.
-
class
osfclient.models.File(json, session=None)¶ -
remove()¶ Remove this file from the remote storage.
-
update(fp)¶ Update the remote file from a local file.
Pass in a filepointer fp that has been opened for writing in binary mode.
-
write_to(fp)¶ Write contents of this file to a local file.
Pass in a filepointer fp that has been opened for writing in binary mode.
-
-
class
osfclient.models.Project(json, session=None)¶ -
storage(provider='osfstorage')¶ Return storage provider.
-
storages¶ Iterate over all storages for this projects.
-
-
class
osfclient.models.Storage(json, session=None)¶ -
create_file(path, fp, update=False)¶ Store a new file at path in this storage.
The contents of the file descriptor fp (opened in ‘rb’ mode) will be uploaded to path which is the full path at which to store the file.
To update an existing file set update=True.
-
files¶ Iterate over all files in this storage.
Recursively lists all files in all subfolders.
-