Welcome to CompleteSearch web app’s documentation!¶

This is a Master Project at the University of Freiburg.

Installation¶

The easiest way to install the app is to use CompleteSearch the Docker image (below). Otherwise, all installation steps can be seen here.

Requirements¶

Docker for Linux/macOS/Windows

Installation steps¶

Step 0¶

Install Docker.

Step 1¶

Clone the repository:

git clone https://github.com/anatskiy/docker-completesearch.git
cd docker-completesearch

Build the image:

docker build \
  -t completesearch \
  --build-arg SVN_USERNAME="username" \
  --build-arg SVN_PASSWORD="password" \
  .

Where SVN_USERNAME and SVN_PASSWORD are your credentials for the CompleteSearch svn repository. More information.

Note: You need to escape the username and the password with double quotes “”.

Step 2¶

Run the container:

docker run -d \
  --name completesearch \
  -p 8000:8000 \
  -p 8888:8888 \
  completesearch \
  python3 manage.py runserver -h 0.0.0.0 -p 8000

Usage¶

Open CompleteSearch at http://localhost:8000/

Enter the container:

docker exec -it completesearch /bin/bash

Run a command inside of the container:

docker exec -d completesearch <command>

See the app logs:

docker exec -t completesearch cat app.log

or:

docker logs --tail 100 completesearch

Troubleshooting¶

If you see the error Cannot get facets for ... CompleteSearch server is not responding., try to restart the server:

docker exec -d completesearch make stop start

API Documentation¶

Upload API¶

API operations on uploading files.

upload.views.allowed_file(filename)[source]¶

Check if the uploading file’s type is allowed.

Parameters:	filename – filename
Returns:	result of the check
Return type:	bool

upload.views.define_facets(data)[source]¶

Define facets by their occurrence in the dataset.

Parameters:	data – `DataFrame` with the uploaded dataset
Returns:	field names which will be used as facets
Return type:	list

upload.views.save_uploaded_dataset()[source]¶

POST /save_uploaded_dataset/: Save the ploaded dataset’s settings and start the CompleteSearch server

Parameters:	data – a dictionary with all dataset’s settings, which have been generated by the `upload_file` function.
Returns:	dictionary with the `success` property and an `error` message
Return type:	JSON response

upload.views.upload_file()[source]¶

POST /upload_file/: Upload, validate and process a file, and send it to CompleteSearch.

Parameters:	use_first_row – use the first data row as a header. If the parameter is `False`, the column names will be generated automatically (i.e. Column1, Column2, etc.). file – the uploading file
Returns:	dictionary with dataset settings, e.g. facet/filter fields, which fields to use for the full-text search, etc.
Return type:	JSON response

Search API¶

API operations searching information.

search.views.get_facets()[source]¶

GET /get_facets/?name={name}&q={query}: Return all facets for a given field name and a search query.

Parameters:	name – facet field name q – search query
Returns:	list of facet items for the given field
Return type:	JSON response

search.views.get_facets_list()[source]¶

GET /get_facets_list/: Get the list of facet fields.

Returns:	list of all facet fields
Return type:	JSON response

search.views.process_user_input(query)[source]¶

Process the user input by stripping extra whitespaces, ensuring prefix search by appending asterisks to each search term, and escaping commas and semicolons.

Parameters:	query – search query
Returns:	processed query
Return type:	string

Doctests:

>>> process_user_input('')
''

>>> process_user_input('antonio vivaldi')
'antonio* vivaldi*'

>>> process_user_input('vivaldi, antonio')
'vivaldi","* antonio*'

>>> process_user_input('The . Beatles$')
'The*.Beatles$'

>>> process_user_input(':facet:Preis:* :facet:Preis:2.40')
':facet:Preis:* :facet:Preis:"2.40"'

>>> process_user_input('magic :facet:Preis:* :facet:Preis:2.40')
':facet:Preis:* :facet:Preis:"2.40" magic*'

>>> process_user_input('ge :facet:Autor:* mo*   vi$ | fr | neur.netw')
':facet:Autor:* ge* mo* vi$|fr*|neur*.netw*'

>>> process_user_input('mo| fr | ge .tu')
'mo*|fr*|ge*.tu*'

search.views.search()[source]¶

GET /search/?q={query}&start={start}&hits_per_page={hits_per_page}: Perform search using CompleteSearch.

Parameters:	q – search query start – send hits starting from this one hits_per_page – how many hits return per page
Returns:	list of search hits
Return type:	JSON response

Settings API¶

API operations on configuring the uploaded dataset.

settings.views.configure_dataset()[source]¶

POST /configure_dataset/: Change dataset parameters and regenerate CompleteSearch’s indices.

Parameters:	title_field – which field to use as a hit’s title allow_multiple_items – fields containing multiple terms per column, e.g. several authors per one document within_field_separator – a delimiter which is used in `allow_multiple_items` full_text – which columns should be searched show – which fields should be returned on a hit filter – search in a specific column facets – restrict the search to specific columns and phrases
Returns:	dictionary with the `success` property and an `error` message
Return type:	JSON response

settings.views.delete_dataset()[source]¶

POST /delete_dataset/: Reset app’s settings, delete the uploaded dataset and stop the CompleteSearch server.

Returns:	dictionary with the `success` property
Return type:	JSON response

settings.views.get_settings()[source]¶

GET /get_settings/: Get a dictionary with all dataset settings.

Returns:	dictionary with dataset settings, e.g. facet/filter fields, which fields to use for the full-text search, etc.
Return type:	JSON response