Usagi Documentation¶

Usagi is a data discovery system that crawls metadata from data sources and builds a catalog of data sets. Usagi enables users to search, monitor, and annotate the metadata which helps them discover the needed data to perform analysis. Usagi is part of the BigGorilla <http://biggorilla.org> project which provides tools for data integration and data preparation.

Free software: Apache Software License 2.0

Import databases into Usagi for search¶

A. Write a configuration file:¶

to teach Usagi to access the databases which you want to manage. (In the future, Usagi plans to also take metadata directly to avoid accessing user databases if needed for security reasons.) For example,

[ClassicModels]
data_store: psql
host: localhost
port: 5432
user: classic_user
dbname: classicmodels
password: password

B. Execute the Usagi installer:¶

./usagi-installer --data-store classic_models.cfg

C. Start Solr server and JSON API¶

./solr/solr/bin/solr start
cd api
pkill -KILL api_server.py
python api_server.py &
cd ..

D. Search¶

cd importer
python search.py <your_favourite_keyword>

E. JSON API¶

curl http://localhost:8085/api/search?q=<your_favourite_keyword>

Search API¶

API Server¶

boot¶

python api_server.py

API¶

endpoint¶

GET /api/search

parameters¶

q : query

sample request¶

curl http://54.199.246.169:8085/api/search?q=emp

response¶

{
  "docs":[
    {"title": "Employees"},
    {"title": "Customers"}
  ],
  "hits": 2
}