Welcome to EGADS GUI’s documentation!¶
Introduction¶
The EGADS (EUFAR General Airborne Data-processing Software) core is a Python-based library of processing and file I/O routines designed to help analyze a wide range of airborne atmospheric science data. EGADS purpose is to provide a benchmark for airborne data-processing through its community-provided algorithms, and to act as a reference by providing guidance to researchers with an open-source design and well-documented processing routines.
The GUI (Graphical User Interface) developed for EGADS is a program based on Python and PyQt5 and designed to process/display/plot airborne atmospheric science data. The GUI purpose is to facilitate the use of EGADS for newcomers and to allow the display of data quickly with few clicks.
Python 2.7 is used in development of EGADS due to its compatibility with old libraries (CEDA Nappy). A version based on Python 3.x is explored and will be available once all libraries are compatible with Python 3.x. PyQt5 has been finally chosen as a basis for its compatibility with other software, in particular Matplotlib 2.x.
Note
EGADS GUI is still in beta state and will require a certain knowledge in Python if corrections must be made as quicly as possible.
Installation¶
The latest version of the EGADS GUI can be obtained from https://github.com/EUFAR/egads-gui
Prerequisites¶
The use of the EGADS GUI requires the following packages:
- Python 2.7.10 or newer. Available at https://www.python.org/
- PyQt 5.8 or newer. Available at https://www.riverbankcomputing.com/software/pyqt/download5
- EGADS 0.8.8 or newer. Available at https://pypi.python.org/pypi/egads
- Matplotlib 2.0.0 or newer. Available at https://pypi.python.org/pypi/matplotlib
- Babel 2.3.4 or newer. Available at https://pypi.python.org/pypi/Babel
Note
It is recommended to use Anaconda (for Linux and Windows) - PyQt5 is included by default in the Anaconda distribution. Available at https://www.anaconda.com/download/#download
Compatibility¶
The EGADS GUI has been tested on Linux and Windows. Even if it has not been tested on MacOS, it should be compatible with the Apple system.
Installation¶
The EGADS GUI is actually available as a common Python script, thus it doesn’t need any particular installation. To use it, the package must be downloaded and uncompressed somewhere on the hard drive, and the script executed with the usual command python egads_gui.py
from a terminal launched in the EGADS GUI directory. This version can be executed on any version of Linux, Windows and MacOS as soon as prerequisites are installed and working.
In the future, a stand-alone version will be available, embedded or not the last version of EGADS core.
To learn how to install EGADS, please read the EGADS documentation available at the following places: https://github.com/EUFAR/egads/blob/master/Documentation/ & http://egads.readthedocs.io/
Testing¶
No test system has been introduced at this time.
Log¶
A logging system has been integrated in the EGADS GUI since the version 0.7.0. By default, the output file is available in the GUI directory and the logging level has been set to DEBUG. Options for logging level and location are set in a config_file egads_gui.ini
and can be change directly through the Option window in the GUI.
Actual options to control the logging system are for now:
level
: the logging level (DEBUG
,INFO
,WARNING
,CRITICAL
,ERROR
).path
: the path of the file containing all logs.
Once the logging level has been changed in the Option window (saved and confirmed by clicking on Ok), the logging file will record new messages based on the logging level. On the contrary, if the path is modified, the GUI has to be restarted to take into account the new path.
Description¶
Important information¶
In the following tutorial, all pictures have been made on a Windows 10 system. On the contrary, all commands, paths and files are from a Linux Mageia 6 system.
Exploring EGADS GUI¶
The simplest way to start working with EGADS GUI is to run it from the terminal:
$ cd egads_gui/
$ python egads_gui.py
The EGADS GUI is easy to use and self explaining by the presence of information button (once a button is clicked, a popup will appear with text to explain the purpose of the area actually used) and tool tips.
The EGADS GUI is composed of different windows, designed to display information to the user and accept interactions, and functions linking the different parts of the GUI with EGADS.
Current GUI limitations¶
Actually, the GUI has few limitations based on the development time and the number of person working on this project. First the file formats handled by the GUI. Even if EGADS can handle text files (raw, ASCII and csv), the GUI can handle only NetCDF and NASA Ames file format at this time. Second, matrix variable (that is to say variables based on multiple dimensions, or gridded data) can be loaded in the GUI, but, at this time, only time series can be handled by the plot function. Third, the NASA Ames function can only handle files with a FFI equal to 1001.
The main window¶
The main window of the GUI is what the user will see first once he has launched the software.
The main window is composed of four parts, from up to down:
- the first one is a menu bar with three menus -
File
,Processings
andAbout
- allowing the user to open/save a file, launch a processing on more than one file, quit the software, display the different algorithms imbedded in EGADS, and display information about the software. - the second one is a bar containing 13 icons and giving access to different kind of functions and windows as the plot window or the metadata window.
- the third one is a central widget whose purpose is to welcome the interface dedicated to NetCDF, NASA Ames and text files.
- the last one is a footbar, used to display few information about the file actually loaded, like the name, the weight and the conventions.
The icon bar and the windows¶
13 icons are embedded in the icon bar, 7 of them give access to the GUI functions and 6 of them are directly linked to sub windows.
The function icons¶
The four first icons of the icon bar give access to few basic functions to control a file:
The data icons¶
The seventh, eighth, and ninth icons are here to manipulate data:
- : With this function, the user have the possibility to create a simple variable : a time series composed of 0 or 1, or a suite of values from n_start to n_end, or a matrix with personal values. It can be useful when a small matrix with optical values is needed in the case of few optical algorithms. NOT AVAILABLE AT THIS TIME
- : In EGADS GUI, once a variable is processed through an algorithm, the result is a new variable displayed in the
New variables
tab of the central widget. In this tab, a variable can’t be saved in a file, only variables in theVariable
tab are saved. The user has to migrate the new variable to theVariables
tab. And he can do that with this function. - : The purpose of this function is to delete a variable. If the file is saved after a deletion, obviously the deleted variable won’t appear in the file anymore.
The global attributes window¶
The user has to click on the icon to open the global attributes window.
All global attributes embedded in a NASA Ames or NetCDF files are loaded in this window. The user has the possibility to modify, create and delete them. As EGADS GUI follows the EUFAR Standards & Protocols recommendations, few global attributes are thus mandatory and displayed above others. They can be modified, but they can’t be deleted.
The variable attributes window¶
Accessible by clicking on the icon , the user has to select first a variable from the Variables
tab of the main window.
All attributes linked to the selected variable, coming from a NetCDF or a NASA Ames file, are loaded in this window. As for the global attributes window, the user can create/modify/delete attributes. Following the EUFAR Standards & Protocols recommendations, two attributes can’t be deleted and modified: units
and _FillValue
.
The processing window¶
Accessible by clicking on the icon , a variable can be processed with an algorithm already embedded in EGADS or created by a user.
Here the user has the possibility to select an algorithm from the EGADS algorithm list, select one or more variables from the opened file, add factors or numbers if necessary by the algorithm, name the output and launch the process. Information about the algorithm is displayed in the first tab. The second tab displays information coming from the input section of each algorithm, info buttons are here to give details about each input. The third tab is dedicated to the output(s).
The algorithm creation window¶
One of the main goal of EGADS is to let the user creates his own algorithms. Accessible by clicking on the icon , this window is an automated system to create algorithm from the user inputs.
The window is composed of three tabs. The first tab is dedicated to the input(s). The user will fill the different text boxes to prepare the input variable(s). The second tab is dedicated to the output(s). As the window follows the EGADS convention, few tips have to be used here and will be explained later. The last tab is dedicated to the metadata of the algorithm and, the most important, to the mathematic formula. Once the user click on Save
, an automated task will check every item in the window, in particular units, and will display a warning popup is something went wrong. The algorithm is then saved in the user
sub-folder of the EGADS/Algorithms folder. For help purpose, few info buttons are present.
The system doesn’t test the algorithm created by the user, it is expected to work, and the user has to give his greatest attention when writing the algorithm. If a complex algorithm needs to be written, the use of a text editor and the manual declaration of the algorithm in EGADS are strongly encouraged.
The data display window¶
Accessible by clicking on the icon , the user has to select first a variable from the Variables
tab of the main window.
Here the user can take a look directly in the variable.
The plot window¶
Accessible by clicking on the icon , the user have to select the Variables
tab to enable the window. The purpose of that window is to give the user the possibility to plot all kind of data and to save the corresponding figure.
The plot window is composed of an icon bar including six icons and four different tabs:
- : To save a figure
- : To move the view in the figure
- : To zoom on a selection
- : To reset the view
- : To reset the plot window
- : To quit the plot window
- The first tab is dedicated to the selection of the type of graph. Once the user has selected a simple plot or a multiple plot, and the variables he wants to display, the software will create the figure automatically with default options.
- In the second and third tab, empty before creating the figure, the software gives the possibility to the user to change few options for each curve or for the whole figure.
- The last tab is only dedicated to the options involved when a figure is saved.
The design of the plot window is based on the access to as many options as possible to let the user modify the figure to his heart content, quickly and easily. Obviously, if the user wants to create complex figures, it is strongly encouraged to use EGADS and Matplotlib from a usual python script.
The central widget, example with a NetCDF file¶
Once a NetCDF or NASA Ames file has been opened in the GUI, a new interface object is displayed. It’s a Tab Widget composed of three tabs.
The global attributes tab¶
The purpose of the first tab is to display the main global attributes of a NetCDF or NASA Ames. Concerning NetCDF, the tab is following the EUFAR Standards & Protocols NetCDF convention and few attributes are automatically displayed. Important information about the dataset is usually recorded in the global attributes.
The edit icon for each field gives the user the possibility to modify each global attribute from the tab view. Once an attribute has been modified, the file has to be saved to keep the new attribute. Finally an object in the lower part of the central widget is here to give information about the compatibility of the file, if it is a NetCDF one, with the official EUFAR Standards and Protocols NetCDF convention.
The variables tab¶
The second tab is dedicated to variables and there attributes.
A list of all variables included in the NetCDF/NASA Ames file is displayed in the left side, ordered alphabetically. If a user clicks on a variable, attributes will be displayed on the right side of the tab. As for the global attributes tab, an edit icon is here to let the user modify directly the variable attributes from the tab. Right clicks are also registered to cancel a modification in edit mode. Few of them are not intended to be modify, like the units or the non value, consequently a grey colour is superimposed on them to inform the user.
The new variables tab¶
The purpose of the third tab, not visible by default, is to welcome newly-created variables.
This tab behaves completely as the Variables
tab.
The footbar¶
The purpose of the footbar is to display few information about the file actually loaded, like the name, the size and the conventions.
Tutorial¶
Important information¶
All tutorials proposed here are produced using a NetCDF file as a reference. Apart from the fact that metadata are different, there is almost no difference concerning the interface if using a NASA Ames file. Once the GUI is ready to handle raw, csv and text files, the corresponding tutorials will be added here if certain differences exist compared to NetCDF handling.
How to open a file?¶
- Then apply the filter
NetCDF Files (*.nc)
, select the NetCDF file you want to open in the list, and click onOpen
.
- If the file is loaded as it should, the interface specific to NASA Ames/NetCDF files should be displayed without any warning.
- Another way to open a file is to use the menu
File
and then click onOpen...
.
How to save a file?¶
- To save a file, the user has two possibilities. The first one by clicking on . Its the basic saving function when a file has been modified.
- The second one by clicking on . It’s the usual way to save a file with a new name, or to convert a file from a format to another format.
- When
Saving As
is selected, just enter the name of the new file and the format, then clickSave
to confirm the action.
How to launch batch processing?¶
The processing of multiple files at once has not been implemented yet.
How to modify the global attributes of a NetCDF file?¶
EGADS GUI proposes two ways to modify a global attribute: from the global attributes tab and from the global attributes window. The global attributes tab only shows the most important attributes, in agreement with the EUFAR Standards & Protocols NetCDF convention. The second solution is the only one if the user wants to display other attributes and to create and/or delete global attributes.
From the global attributes tab¶
- To unlock a field and edit the associated attribute, click on . The icon is replaced by a icon to confirm that you are in edit mode.
- Then, modify the attribute as you wish.
- The software should display the modified attribute and the word
modified
in the window title.
From the global attributes window¶
- Here you have the possibility to create, modify and delete global attributes. Click on
Show other attributes
to display other attributes not showed in the window. To modify a global attribute, just click in the associated field and enter your text. Then click onSave
to confirm your modification.
- The software should display the modified attribute and the word
modified
in the window title.
How to modify the attributes of a variable ?¶
EGADS GUI propose two ways to modify a variable attribute: from the variable attributes tab and from the variable attributes window. The variable attributes tab only shows the most important attributes, in agreement with the EUFAR Standards & Protocols NetCDF convention. The second solution is the only one if the user wants to display other attributes and to create and/or delete variable attributes.
From the variable tab¶
- Select the
Variables
tab.
- Select a variable in the left list and click on to unlock the associated field of the attribute you want to modify. The icon is replaced by a icon to confirm that you are in edit mode.
- The software should display the modified attribute and the word
modified
in the window title.
From the variable attributes window¶
- Select the
Variables
tab.
- Here you have the possibility to create, modify and delete variable attributes. Click on
Show other attributes
to display other attributes not showed in the window. To modify a variable attribute, just click in the associated field and enter your text. Then click onSave
to confirm your modification.
- The software should display the modified attribute and the word
modified
in the window title.
How to create a simple variable ?¶
This function has not been implemented yet.
How to process a variable?¶
EGADS GUI gives the user the possibility to use and apply algorithm embedded in EGADS. Because of the limitation of the GUI, actually, it is only possible to execute one algorithm at a time. All processings are done through the processing window.
- To launch the processing window, select the
Variables
tab.
- The processing window is composed of three tabs: the first one to choose the algorithm, the second one to choose the variable(s) processed by the algorithm, and the last one to set the output(s). First, the user has to choose an algorithm by selecting a
Category
and anAlgorithm
. Information are displayed in the lower part of the tab. Then click on theInput(s)
tab.
- Select the variable(s) to be injected in the algorithm. Info buttons are here for the description of the variables involved in the algorithm. Then click on
Output(s)
.
- Choose a name for the output(s). The name(s) will be displayed in the main window after the processing.
- Depending on the size of the input variables and on the complexity of the algorithm, the processing can take time. Once the output(s) is(are) ready, a new tab appears and new variables are stored here.
How to migrate a variable from the new variable tab to the variable tab?¶
To avoid wrong manipulation and for the sake of convenience, a tab dedicated to new variables and called New variables
has been introduced in EGADS GUI. Once a variable is created, it will appear in this new tab. If the user wants to save the new variable(s), he must migrate it/them in the Variables
tab.
- To migrate a newly-created variable, select the variable to be migrated in the
New variables
tab and click on .
- The newly-created variable should disappear from the
New variables
tab (the tab is removed if the list of variable is empty) and appear in theVariables
tab.
How to delete a variable?¶
- For NetCDF file, a warning information is displayed if a variable is deleted for the first time.
- Once a variable is deleted, the user have to save the file, with a different name if it is a NetCDF one, to delete the variable in the file.
How to create an algorithm through the GUI?¶
EGADS GUI offers the possibility to create algorithm from a window, by filling in different kind of fields. If a complex algorithm has to be written, the most suitable way is to use a text editor as Notepad++ (Windows) or Kate (Linux) and modify the template provided in the EGADS algorithm directory.
- The algorithm creation window is composed of three tabs. The first one to create all inputs needed by the algorithm, the second one to create all outputs created by the algorithm, and the last one to prepare all metadata (category, sources, …) and the algorithm formula. First let’s create an input variable. Click on the
+
button and fill in the different fields. Then click on the tabOutput(s)
.
- To create an output variable, click on the
+
button. The output variable(s) is(are) the result of the algorithm. As EGADS and EGADS GUI are coded in Python, the output variable(s) should be returned by the algorithm:return var
. If theUnits
of a result has to be the same as one of the input, the user can enterinput
and the input number (starting at 0):input0
if the output unit has to be the same than the first input variable. It’s the same for theOutput standard name
and theOutput long name
. Once all fields are filled, click on theAlgorithm
tab.
- The algorithm tab has 2 purposes: metadata and algorithm formula. A special attention should be paid to the
Algorithm
field. The formula has to be written in Python 2.7 (importing modules like numpy is possible), involving the inputs and outputs created earlier, and the algorithm should always return the outputs. Once the algorithm is ready, click onSave
to save it in the EGADS user algorithm directory. If there is a problem with one of the units, a warning message will be displayed.
Note
There is no system to check the algorithm result. Once the algorithm is saved, EGADS and its GUI expect the algorithm to be true and stable.
How to display information about an algorithm?¶
When processing data, it is always a good idea to understand how an algorithm works before making use of it. EGADS GUI offers the possibility to display an algorithm in a window.
- To launch the algorithm display window, just click on the
Processing
item in the menu bar, then select a category and click on an algorithm.
- Information about an algorithm can be found in several tabs.
How to display a variable data?¶
The EGADS GUI offers the possibility to display the values of a variable. At that time, it is not possible to modify them.
- First select the
Variables
tab.
- The values are displayed in a table, and few information are given like the units and the name of the variable.
How to plot a variable ?¶
By integrating the Python module Matplotlib, the EGADS GUI gives the user a great tool to plot data and save the result in a graphic file. Actually only time series can be plotted. Gridded data should be integrated quickly.
- To access the plot module, select first the
Variables
tab.
- Then click on . For the first launch, it can take a long time to display the window, as the Matplotlib module has to create the system font database.
- With the plot function, the user can plot a single figure or multiple figures on the same graph.
- For our tutorial, let’s select
Single plot
. Two comboboxes appear. In the first one, select the variable for the X axis, for example the time. In the second one, select a variable for the Y axis. Once it is done, click on the+
button to add the variable of the Y axis to the list of plotted variables. The figure is drawn automatically with default options. By choosing another variable in the Y axis list and clicking on the+
button, you can plot multiple variables on the same figure. Then let’s modify the options of the figure, click onFigure options
- Those options appear only if a figure has been drawn. For multiple figures on the same graph, multiple option sets will appear. Change few option here and click on
Update
to apply the new options to the figure. Then click onPlot options
to change the options dedicated to the variable drawn in the figure.
- Each time a variable is drawn, a new set of plot options will be displayed. It is possible here to change the options directly related to the variable curve, like the colour or the legend. As before, you have to click on
Update
to apply those modifications. And let’s have a look to the result by clicking onPlot window
- If the result is satisfying, click on
Save options
to modify the save settings of the figure.
- Finally choose a name for the file and click on
Save
.