Metadata Publishing Tool Documentation¶
About¶
Metadata Publishing Tool (MPT) is a desktop application that facilitates the publishing of resources and sitemaps in conformance with the ResourceSync Framework Specification.
Metadata Publishing Tool, rspub-gui and rspub-core were developed by Data Archiving and Networked Services (DANS-KNAW) under auspices of the European Holocaust Research Infrastructure (EHRI).
The documentation in this rtd is intended for end users and system administrators.
- Download latest executables from the releases page of the rspub-gui project.
- Source location: https://github.com/EHRI/rspub-gui.
- The GUI is based on rspub-core. See https://github.com/EHRI/rspub-core.
- In case of questions contact the EHRI team.
Introduction¶
Note
Introducing the EHRI use case
“The mission of the European Holocaust Research Infrastructure (EHRI) is to support the Holocaust research community by building a digital infrastructure and facilitating human networks. EHRI provides online access to information about dispersed sources relating to the Holocaust through its Online Portal, and tools and methods that enable researchers and archivists to collaboratively work with such sources.”
In order to collect information on dispersed archives across Europe -and the rest of the world- that hold Holocaust-related collections the Encoded Archival Description (EAD) is used as the format for information dissemination; the ResourceSync Framework may be used as the transportation and synchronization mechanism.
Metadata Publishing Tool is the subject of this documentation. Examples in this introduction and the rest of the documentation will sometimes use the EHRI use case to describe the working of the tool.
ResourceSync¶
The ResourceSync Framework Specification describes a synchronization framework for the web consisting of various capabilities that allow third-party systems to remain synchronized with a server’s evolving resources. More precisely the ResourceSync Framework describes the communication between Source and Destination aimed at synchronizing one or more resources. Communication utilizes http and an extension on the sitemap protocol, an xml-based format for expressing metadata, relevant for synchronization.
Fig. 1. External logistics. The ResourceSync Framework Specification at work. Collection Holding Institutions expose content and ResourceSync metadata on their web servers. The central hub (in this case the EHRI Portal) is actively collecting resources and keeping them in sync with the aid of published sitemaps.
We can say that the ResourceSync Specification is a perfect fit for solving the external logistics when it comes to synchronizing resources between a central Destination and various Sources. Figure 1. depicts the external logistics.
When the resources we are trying to synchronize are not web-resources by them selves but instead stem from information systems, databases or other places within an organization, we are faced with other problems, which we can qualify as related to internal logistics.
Metadata Publishing Tool¶
Metadata Publishing Tool is an application that solves various problems related to the internal logistics:
- How do we collect and import resources from various places within the organization;
- How do we select relevant resources;
- How do we create ResourceSync sitemap metadata on relevant resources;
- How do we export resources and sitemaps to the web server;
- How do we verify that the exposed URL’s are correct and our ResourceSync site ready to be harvested by a Destination.
Fig. 2. Internal logistics. Metadata Publishing Tool at work.
Figure 2. depicts internal logistics and the role of Metadata Publishing Tool. The situation described may be exemplary for Collection Holding Institutions (CHI’s) within the EHRI infrastructure, although different situations may equally be applicable. Metadata Publishing Tool is an application that is deployed on your laptop or local work station. From there you collect and select resources, create the ResourceSync sitemaps, export resources and sitemaps to your web server and verify the exposed URL’s.
Configuration of Metadata Publishing Tool may need the hand and insight of a technically skilled person. Once configured it can be managed by archivists and other content-savvy users that do not necessarily have technical skills.
This documentation¶
This documentation starts with describing how to install Metadata Publishing Tool on various operating Systems in the Installation pages. Each tab or wizard page of the application is the subject of the other chapters. There are Help buttons on each tab or wizard page that link to the relevant chapters in this documentation. Finally there is a Glossary on terms used throughout this documentation. .. and an appendix that handles various use cases that may be applicable to your situation on your institution.
Installation¶
Windows installer¶
Install Metadata Publishing Tool on Windows using a wizard
Attention
If a previous release of Metadata Publishing Tool was installed on your machine, first uninstall this older release. See Uninstall Metadata Publishing Tool.
Download the latest MPT_win_installer.exe from
the release page of the rspub-gui project on gitHub.
Circumvent security¶
After download, double-click the downloaded MPT_win_installer.exe.
Because the software has not been code-signed by a trusted authority, security settings on your machine may object
to executing it right away.
Windows 7
On Windows 7 you may see a sign similar to this.
Security warning on Windows 7
Click Run and proceed to Run the install wizard.
Windows 10
On Windows 10 a blue warning may appear.
Security warning on Windows 10
Click More info on the left side of this blue screen…
Security warning on Windows 10, second screen
Click Run anyway and proceed to Run the install wizard.
Run the install wizard¶
The install wizard has several screens and will guide you through the installation process. On average, you can accept the predefined values.
Welcome message
Welcome message of the install wizard
The wizard opens with a welcome message. Click Next.
Installation folder
Choosing the installation folder
Choosing the installation folder. The default value is C:\Program Files (x86)\EHRI\MetadataPublishingTool.
Choose Browse… if you want a different installation folder, choose Next when done.
Select components
Selecting components
Metadata Publishing Tool has only one component. So leave the component selected and click Next.
License agreement
Inspecting the license
Code and executables of the Metadata Publishing Tool are distributed under the Apache License 2.0. You must accept the license agreement before you may proceed with the install wizard. Click Next when done.
Start Menu shortcuts
Setting shortcuts
Default a shortcut to the MetadataPublishingTool.exe will be placed in the program group EHRI.
Click Next when done.
Ready to install
Ready to install
Click Next to continue. A security warning may again appear, telling you are about to install software from an unknown publisher. Click Yes to proceed.
Installing Metadata Publishing Tool
Progress of installation
The installation process will now begin. An indication of progress will be shown.
Completing the Metadata Publishing Tool Wizard
Completing the installation
After installation has completed, the last screen of the installation wizard is shown. Click Finish to exit the wizard.
Start Metadata Publishing Tool¶
Shortcuts to Metadata Publishing Tool and the MPT Maintenance Tool in the EHRI group
After completing the installation wizard you can find shortcuts to MetadataPublishingTool.exe and the
MPT_MaintenanceTool.exe in the group EHRI of your Start Menu. Alternatively you may find the executables
in the folder C:\Program Files (x86)\EHRI\MetadataPublishingTool (if you did not change the Installation folder
during installation).
Uninstall Metadata Publishing Tool¶
The Metadata Publishing Tool and its components can be removed from your machine by the
MPT Maintenance Tool. Click the shortcut MPT_MaintenanceTool in the EHRI group of your Start Menu or
navigate to C:\Program Files (x86)\EHRI\MetadataPublishingTool and double-click MPT_MaintenanceTool.exe.
The uninstall wizard will begin.
Removing Metadata Publishing Tool
Click Next to continue.
Ready to uninstall
Click Uninstall. After the uninstall process has run click Finish. The folder MetadataPublishingTool and its
contents are now removed from your system. The folder EHRI may still remain with other executables or is empty
if you did not install other programs in there.
Windows folder installation¶
Install Metadata Publishing Tool and its dependencies as separate files in a folder
Install the Metadata Publishing Tool and its dependencies in a folder. Advantages of this kind of installation: faster start-up time.
Download Metadata Publishing Tool¶
Download the latest MetadataPublishingTool.zip from
the release page of the rspub-gui project on gitHub.
Unzip and place anywhere on your file system.
Start Metadata Publishing Tool¶
Open the folder MetadataPublishingTool and
double-click the executable MetadataPublishingTool.exe.
The first time you double-click the downloaded executable you may get a warning: unknown app has been detected. To prevent this:
- right-click the executable
- on the pop-up menu click properties
- on the General tab under Security check the Unlock checkbox
Mac OS installer¶
Install Metadata Publishing Tool on Mac OS using a wizard
Attention
If a previous release of Metadata Publishing Tool was installed on your machine, first uninstall this older release. See Uninstall Metadata Publishing Tool.
Download the latest MPT_mac_installer.dmg from
the release page of the rspub-gui project on gitHub.
After download, double-click the downloaded MPT_mac_installer.dmg. This will mount the MPT_mac_installer image
on your system.
Mounted MPT_mac_installer
Double-click the MPT_mac_installer.app in the mount window.
Circumvent security¶
Because the software has not been code-signed by a trusted authority, security settings on your machine may object to executing it right away.
Security warning on Mac OS
Part of System Preferences, Security & Privacy
Alternatively you may have to lower your security settings temporary. Under Allow apps downloaded from: click Anywhere. After the installation process has completed you may restore your security settings.
Last warning from security
After starting the MPT_mac_installer.app you may receive a last warning. Click Open to start the install
wizard.
Run the install wizard¶
The install wizard has several screens and will guide you through the installation process. On average, you can accept the predefined values.
Welcome message
Welcome message of the install wizard
The wizard opens with a welcome message. Click Continue.
Installation folder
Choosing the installation folder
Choosing the installation folder. The default value is /Applications/MPT.
Choose Browse… if you want a different installation folder, choose Continue when done.
Select components
Selecting components
Metadata Publishing Tool has only one component. So leave the component selected and click Continue.
License agreement
Inspecting the license
Code and executables of the Metadata Publishing Tool are distributed under the Apache License 2.0. You must accept the license agreement before you may proceed with the install wizard. Click Continue when done.
Ready to install
Ready to install
Click Install.
Installing Metadata Publishing Tool
Progress of installation
The installation process will now begin. An indication of progress will be shown.
Completing the Metadata Publishing Tool Wizard
Completing the installation
After installation has completed, the last screen of the installation wizard is shown. Click Done to exit
the wizard. You may now eject the mounted MPT_mac_installer image.
Start Metadata Publishing Tool¶
MetadataPublishingTool.app and MPT_MaintenanceTool.app in the folder MPT in the Applications folder
After completing the installation wizard you can find MetadataPublishingTool.app and the
MPT_MaintenanceTool.app in the folder MPT of your /Applications folder. Alternatively you
can click on the respective icon in your Launchpad.
Uninstall Metadata Publishing Tool¶
The Metadata Publishing Tool and its components can be removed from your machine by the
MPT Maintenance Tool. Click the shortcut MPT_MaintenanceTool in your Launchpad or
navigate to /Applications/MPT and double-click MPT_MaintenanceTool.app.
The uninstall wizard will begin.
Removing Metadata Publishing Tool
Click Continue.
Ready to uninstall
Click Uninstall. After the uninstall process has run click Finish. The folder MPT and its
contents are now removed from your system.
Configuration¶
Configure Metadata Publishing Tool
Screenshot of the configure page
Important
The parameters on the configuration page constitute all variables needed to synchronize a set of resources. Variables are validated and, if valid, saved automatically. Multiple sets of parameters can be saved as named configurations. See The File menu on how to save, load and delete configurations.
Variables on the configuration page are best set with the help of a technically skilled person. Once variables are set and tested to be correct, they can be reused over future synchronization runs.
In the following paragraphs we’ll describe each parameter in detail.
Resource directory¶
The resource directory should be an existing directory on the (local or networked) filesystem. In this base directory resources to be synchronized can be found. The resource directory should be chosen careful, because it influences the composition of the URL to the resources that will be exposed in sitemaps:
local path to the resource: C:\path\to\resources\ehri\path\to\resource.ead
resource directory: C:\path\to\resources\
URL prefix: http://www.example.com/
composed URL to resource: http://www.example.com/ehri/path/to/resource.ead
Important
The URL prefix may have a path segment:
local path to the resource: C:\path\to\resources\ehri\path\to\resource.ead
resource directory: C:\path\to\resources\
URL prefix: http://www.example.com/rs/abc/
composed URL to resource: http://www.example.com/rs/abc/ehri/path/to/resource.ead
The resource directory influences the composed URL:
local path to the resource: C:\path\to\resources\ehri\path\to\resource.ead
resource directory: C:\path\to\resources\ehri\path\
URL prefix: http://www.example.com/rs/abc/
composed URL to resource: http://www.example.com/rs/abc/to/resource.ead
The button Browse will open a file explorer that enables choosing the resource directory.
See also: URL prefix
Attention
When using the EHRI mapping tool, after conversion your EAD documents are in a directory that bears the conversion date as its name. Point the resource directory entry to this dated directory before execution of a synchronization.
Metadata directory¶
The metadata directory is the name of the directory where generated sitemaps are stored. The value of metadata directory may consist of multiple path segments. The metadata directory is always relative to the resource directory:
resource directory: C:\path\to\resources\
metadata directory: metadata
absolute path: C:\path\to\resources\metadata
If the metadata directory does not exist, it will be created during first execution of a synchronization.
Hint
Executing a synchronization will produce one Capability List in the Metadata directory.
Each Capability List defines a set of resources. When using multiple configurations
to define multiple sets of resources from the same resource directory make sure that each Metadata
directory gets a different name. Like md_01, md_02 etc.
Description directory¶
The description directory should be an existing directory on the (local or networked) filesystem.
In this directory the document that describes the entire site, also known as .well-known/resourcesync
or Source Description is expected
or will be created. If the value of description directory is left blank, the document is expected or will be
created in the metadata directory.
The button Browse will open a file explorer that enables choosing the description directory.
Plugin directory¶
The plugin directory is an existing directory on the (local or networked) filesystem. In this directory or its subdirectories a search for plugins will be conducted. At the moment there are slots for plugins of type ResourceGateBuilder. If plugin directory is left blank, no search will be conducted upon execution of a synchronization.
The button Browse will open a file explorer that enables choosing the plugin directory.
URL prefix¶
The URL prefix is the basename of the site, optionally followed by a path segment. It is the first part in the composition of URL’s to resources and sitemaps. The relative path to resources and sitemaps calculated from resource directory is the second part:
local path to the resource: C:\path\to\resources\ehri\path\to\resource.ead
resource directory: C:\path\to\resources\
URL prefix: http://www.example.com/
composed URL to resource: http://www.example.com/ehri/path/to/resource.ead
The URL prefix may have a path segment:
local path to the resource: C:\path\to\resources\ehri\path\to\resource.ead
resource directory: C:\path\to\resources\
URL prefix: http://www.example.com/rs/abc/
composed URL to resource: http://www.example.com/rs/abc/ehri/path/to/resource.ead
See also: Resource directory
Strategy¶
The strategy defines what kind of sitemap documents will be generated when a synchronization is executed. At the moment you can choose between
- New resourcelist strategy - At each synchronization run a completely new Resource List will be generated. If previous Resource Lists or Capability Lists exist in the metadata directory you will be asked if they can be deleted.
- New changelist strategy - Will create a new Change List at each synchronization run. At the start of synchronization, if no Resource List exists in the metadata directory, will conduct the New resourcelist strategy on first execution.
- Incremental changelist strategy - Will increment an existing Change List with the newly found changes. At the start of synchronization, if no Resource List exists in the metadata directory, will conduct the New resourcelist strategy on first execution.
Hint
The strategy can be changed before each successive execution of a synchronization run.
Max. items in one list¶
The maximum amount of items in one list. The sitemap protocol has an unofficial standard on how many items (links to resources) can be in one list. This amount can vary between 1 and 50000.
Zero-fill filename¶
The amount of digits that generated sitemaps should have in their filenames. This value should be between 1 and 10. Example of filenames with zero-fill filename set at 4:
changelist_0000.xml
changelist_0001.xml
changelist_0002.xml
...
changelist_9999.xml
With zero-fill filename set to 4, the amount of Change Lists can grow to 10000. This means you can execute 10000 synchronizations in strategy mode new changelist strategy.
Sitemaps with new lines¶
A completely ‘pretty print’ mechanism for the generated sitemaps is not (yet) available. Mark the checkbox for a more human readable version of sitemaps.
Save sitemaps to disk¶
You can do a trial run of a synchronization. A trial run will report an overview of how many resources are affected, how many changes have been detected etc. but will not produce new sitemaps. Leave the checkbox unchecked for a trial run.
‘well-known’ at server root¶
This reflects the locality of the Source Description on the web server. The prescribed location of this document is in the .well-known directory at the root of the server:
{server root}/.well-known/resourcesync
However, there may be situations where it is not allowed to access the server root directory. Since EHRI makes no use of discovering the site by means of the well-known URI, this document is not strictly necessary. The value for ‘well-known’ at server root is used for computing the URL’s that link Capability Lists to their parent document.
Mark the checkbox if the Source Description is at the server root. If left unchecked the program assumes that the Source Description is in the metadata directory.
Import resources¶
Import files from a server with the Metadata Publishing Tool
Hint
There are several ways to overcome problems with the exchange of files between servers and your local or network drive. One way is to use an scp client, which is described on this page.
Another way is to use DropBox-like solutions. See for instance B2DROP and WebDAV.
Screenshot of the import page
Important
The import functionality is only necessary if your files (EAD files in the case of EHRI) are on a server. If your files are on a local or networked drive you may skip this page of the wizard.
With the import page you can copy files from the remote server to a local or networked drive with the aid of the Secure Copy Protocol (scp).
Parameters on the import page are best set with the help of a technically skilled person. The variables on this page are part of the same named configuration as the one from the Configuration page and are saved automatically. Once parameters on this page are set, all you have to do each time you want to import your remote files to your local environment is press the Start button (and remember your password).
In the following paragraphs we’ll describe the import process in detail.
SCP parameters¶
Hint
Different scp parameters can be set on each configuration. They are saved automatically.
- Server
- The name or IP address of the server.
- Port
- The port on the remote server. Default scp port is 22.
- User
- The username on the remote server.
- Remote path
- The path to the directory on the remote host. All files and folders in the remote directory will be copied recursively to the Local path (see below).
- Local path
The path to the directory on the local or networked drive that you wish to copy to. All files and folders in the directory on the Remote path (see above) will be copied recursively to this directory. Directories on the local path that do not exist will be created.
The button Browse will open a file explorer that enables choosing the local directory.
Running an import¶
After pressing the Start button, the import execution window will open.
Screenshot of the import execution window
The import execution window has three areas for reporting events:
- Main events
- In this area main events of the import process will be reported.
- Resources
- In this area files that are imported are listed.
- Errors
- In this area errors that took place during the import process are reported.
All areas can be enlarged or made smaller by grabbing the horizontal handle bars. Of course, the import execution window itself can also be reshaped.
Press the Run button to start the import process. A dialog appears in which you have to type your password for the remote server.
Tip
While the import process is running you may at any time press the Stop button to interrupt the process.
Screenshot of the import execution window while the import process is running
After the process has ended without errors the complete file and folder tree of the remote path is now on the local path.
Select resources¶
Select files to be synchronized with Metadata Publishing Tool
Screenshot of the select page
Important
On the select page you decide which files will be synchronized. The page offers a choice on how to select:
- Simple select - All files in one directory will be synchronized.
- Advanced select - Multiple files and folders can be included and multiple files and folders can be excluded. This gives the opportunity to precisely select the resources that will be synchronized.
Both selection methods work recursive: files in subdirectories are included.
The variables on this page are part of the same named configuration as the one from the Configuration page and are saved automatically. Different selections can be set and saved for each configuration.
Simple selection¶
Select the checkbox for Simple selection: one directory. Press the Browse button to select the directory that contains the resources that should be synchronized.
Hint
The selected directory should be equal to or a subdirectory of the resource directory that you chose on the Configure page. Only resources that are in the resource directory or one of its subdirectories are subject to synchronization.
Press the Play… button if you want to get an impression of how many and which files are in the chosen directory and its subdirectories. The list and count displayed are estimates; automatically excluded files like hidden files are not excluded in this display, but will be excluded in the synchronization.
Advanced selection¶
Select the checkbox for Advanced: create a selector.
The advanced part of the select page
Important
With advanced selection you can choose multiple files and folders for inclusion and exclusion of synchronization.
For instance if you include the directory C:\my_files\ehri but want to exclude its subdirectory
C:\my_files\ehri\other_files.
Another feature is the ability to import lists of path names. This opens the possibility of doing the selection of what will be synchronized in or by another application.
Press Add directory or Add files respectively if you want to browse to directories or files to be included or excluded. When adding files you can choose multiple files by pressing Shift and Ctrl or Cmd keys while choosing files with your mouse. You can also type or paste path names in the respective boxes. Remove path names by simply selecting and deleting them.
To import lists of path names press the Import entries button. Choose the file that contains the path names in the explorer that opens. Each entry in the file with path name entries should start on a new line.
Hint
Selected directories or files should be in the resource directory that you chose on the Configure page. Only resources that are in the resource directory or in one of its subdirectories are subject to synchronization.
Press the Play… button if you want to get an impression of how many and which files are included and how many and which files are excluded. The lists and counts displayed are estimates; automatically excluded files like hidden files are not excluded in this display, but will be excluded in the synchronization.
The entries in Includes and Excludes are automatically saved with each configuration. Besides that, they can also be saved and loaded from the filesystem by pressing the Save and Open buttons respectively.
Execute a synchronization¶
Synchronize resources with Metadata Publishing Tool
Screenshot of the execute page
Important
On the execute page you can inspect the parameters of the current configuration and start a synchronization.
Inspect parameters¶
The execute page gives an overview of the value of all parameters that are involved with a synchronization. The parameters in boxes with a white background can directly be set on pages Configure and Select; the values in boxes with a blueish-grey background are derived or computed values.
Synchronize resources¶
Press the Run… button on the execute page to start a synchronization run.
Screenshot of the synchronization window
The synchronization window has three areas for reporting events:
- Main events
- In this area main events of the synchronization process will be reported.
- Resources
- In this area resources that are synchronized are listed.
- Errors
- In this area errors that took place during the synchronization process are reported.
All areas can be enlarged or made smaller by grabbing the horizontal handle bars. Of course, the synchronization window itself can also be reshaped.
Press the Run button to start the synchronization process. If the chosen strategy is new resourcelist strategy, a confirmation window will appear asking if it is OK to delete existing sitemaps in de metadata directory.
Tip
A trial run will report the number of resources that are affected and all sitemaps that will be created, without writing them to disk.
While the synchronization process is running you may at any time press the Stop button to interrupt the process.
Screenshot of the synchronization window while the synchronization process is running
When the synchronization process has finished the main events of the synchronization process are reported.
Part of the synchronization window after the synchronization process has finished
Clicking on the paths to the sitemaps will open your local editor for xml-files with the contents of the sitemap. As long as the sitemaps are not exported to your web site the links to the URI’s are of course stale or will not function.
The default resource gate will reject hidden files and files in the metadata directory in case this directory is on the select path.
Export resources and sitemaps¶
Transfer resources and sitemaps to a web server
Hint
There are several ways to overcome problems with the exchange of files between servers and your local or network drive. One way is to use an scp client, which is described on this page.
Another way is to use DropBox-like solutions. See for instance B2DROP and WebDAV.
Screenshot of the export page
Important
The local synchronization will not publish your resources and sitemaps. For that, resources and sitemaps must be made available on a web server. The export page offers two methods to export resources and sitemaps from your local or networked drive to your web server:
- Transfer files with SCP - Uses the Secure Copy Protocol (scp) to transfer files directly to your web server.
- Create a zip file - This method creates a zip file of your resources and sitemaps. You can hand over this zip file to your system administrator who should take care of publishing the contents on the web server.
The export page shows the current configuration, the location of the metadata the export will be based upon and at what date and time the last execution of the synchronization took place.
Detail of the export page
Attention
Do not change parameters on the Configuration page in between a synchronization run and the export. The outcome of the export may be undecided if you do so. Always export resources and sitemaps right after a fresh synchronization run.
Transfer files with SCP¶
Detail of the export page with the SCP parameters
The parameters for export with scp can best be set with the help of a technically skilled person. These parameters are automatically saved with the current configuration.
- Server
- The name or IP address of the web server.
- Port
- The scp port on the web server. Default scp port is 22.
- User
- The username on the web server.
- Document root
- The document root is the folder where the website files for a domain name are stored. With the Apache
HTTP Server for instance this defaults to
/var/www/html.
The Document path, relative to the Document root, is derived from the URL prefix you set on the Configuration page. (The Document path will be equal to the path segment of the URL prefix.)
You have a choice between exporting all resources and export the latest changes.
- Export all resources will include all resources mentioned in the sitemap documents currently in the metadata directory. Useful if you want to completely update the part of your site that hosts resources and sitemaps of the current configuration.
- Export latest changes will only include resources that where affected according to the last synchronization. The default. This will suffice in most occasions.
Press Start to open the scp Transport execution window.
The scp Transport execution window
The scp Transport execution window has three areas for reporting events:
- Main events
- In this area main events of the export process will be reported.
- Resources
- In this area files that are exported are listed.
- Errors
- In this area errors that took place during the export process are reported.
Press the Run button to start the export process. A popup dialog will ask for the password of the user at the web server.
Tip
While the import process is running you may at any time press the Stop button to interrupt the process.
Screenshot of the scp Transport execution window while the export process is running
After the export has finished without errors your resources and sitemaps at the web server are now up to date with the local changes after your latest synchronization.
Partial screenshot of the scp Transport execution window after the export process has finished
Attention
You need to have write access to the Document root/Document path at the remote server. If not, you will have received an error message in the scp Transport execution window.
Also, if your source description is at server root, you need to have write access to the .well-know directory at the Document root of the web server.
Create a zip file¶
Detail of the export page for creating zip files
- Zip filename
- Fill in the name of the zip file that will be created. Use the Browse button to open a file explorer that enables choosing the zip filename.
You have a choice between zipping all resources and zipping only the latest changes.
- Zip all resources will include all resources mentioned in the sitemap documents currently in the metadata directory. Useful if you want to completely update the part of your site that hosts resources and sitemaps of the current configuration.
- Zip latest changes will only include resources that where affected according to the last synchronization. The default. This will suffice in most occasions.
Press Start to open the zip Transport execution window.
The zip Transport execution window
The zip Transport execution window has three areas for reporting events:
- Main events
- In this area main events of the zip process will be reported.
- Resources
- In this area files that are zipped are listed.
- Errors
- In this area errors that took place during the zip process are reported.
Press the Run button to start the zip process. While the zip process is running you may at any time
press the Stop button to interrupt the process.
The zip Transport execution window after the zip process has finished
You now need to hand over the zip file to the system administrator of your web server. She should take care of copying resources and sitemaps to the correct location on the web server.
Attention
No matter what you chose at ‘well-known’ at server root during Configuration,
the source description will always be in the metadata directory in the zip file. Your system
administrator should take care to place it in the correct location on the web server. Either keep it in
the metadata directory (‘well-known’ at server root was set to False) or copy it
to the {Document root}/.well-known/resourcesync
(‘well-known’ at server root was set to True).
Audit¶
Check sitemaps, resources and URL’s on the web server
Screenshot of the audit page
Important
The audit page enables verification of the sitemap and resource URL’s on your web server. After a successful audit, you can rest assure that your ResourceSync site is healthy.
The audit page shows the current configuration, the location of the metadata the audit will be based upon, at what date and time the last execution of the synchronization took place, the web server that is being tested and the Document path for documents of the current configuration.
The audit process¶
You have a choice between auditing all resources and auditing the latest changes.
- Audit all resources will include all resources mentioned in the sitemap documents currently in the metadata directory. Useful if you want to completely audit the part of your site that hosts resources and sitemaps of the current configuration.
- Audit latest changes will only include resources that where affected according to the last synchronization.
All sitemaps will be included in the audit, no matter what choice you made for included resources.
Press Start to open the Audit execution window.
The Audit execution window
The Audit execution window has three areas for reporting events:
- Main events
- In this area main events of the audit process will be reported.
- Resources
- In this area URL’s that are being checked are listed.
- Errors
- In this area errors that took place during the audit process are reported.
Press the Run button to start the audit process.
While the audit process is running you may at any time press the Stop button to interrupt the process.
Partial screenshot of the Audit execution window after the audit process has finished
The audit was successful if the Total errors in the display reports 0 errors.
If the audit was not successful, try to analyze the reported errors and adjust Configuration and/or repeat the Export process.
Miscellaneous¶
Various other items concerning Metadata Publishing Tool
Reporting issues¶
Issues, bugs, crashes, unwanted or unexpected behavior of the MPT application can be reported at gitHub’ rspub-gui issues. If relevant, please include the latest log file.
Log file location¶
The MPT application is creating log files. The log file location for various operating systems can be found in the table below.
| Operating System | Log file location |
|---|---|
| Windows | {user home}\AppData\Local\rspub\logs\rspub.log |
| Mac OS and Linux | {user home}/.config/rspub/logs/rspub.log |
| Other | {user home}/rspub/logs/rspub.log |
Contributing a language¶
The MPT-team appreciates if you can and will contribute a translation of the text on buttons, menus and descriptions in the MPT application. Here is how to.
- Download and install the free Poedit application from https://poedit.net/download.
- The
rspub.pottemplate file contains the original entries in English. Download therspub.pottemplate file.- Go to https://raw.githubusercontent.com/EHRI/rspub-gui/master/i18n/rspub.pot
- Right-click on the page and choose
Save asfrom the popup menu.
- Start Poedit and on the Welcome page click
Create New Translation. - Choose the
rspub.pottemplate file you downloaded in step 2. - Choose language (and region) for the translation you want to contribute.
Start translating. Choose an entry in the top panel. Enter tour translation in the text area below. The Poedit application may give suggestions on the right-most side panel.
Save your translation as a *.po file and contact the EHRI-team on how to send in your translation.
Alternatively you may clone the GitHub repository and issue a pull request for your translation.
If all goes well, your translation will be included in the next version release of the MPT application.
Glossary¶
Glossary of terms used in context with Metadata Publishing Tool
- B2DROP
“B2DROP is a secure and trusted data exchange service for researchers and scientists to keep their research data synchronized and up-to-date and to exchange with other researchers.”
– EUDAT web site
Together with a WebDAV client on a server it is an alternative for using scp for exchanging resources and sitemaps between servers and local drives.
See also
- B2DROP
- EUDAT service
- Mount B2DROP using the WebDAV client
- ownCloud Manual
- Capability List
A Capability List is an xml-document of type sitemap that enlists Resource Lists and Change Lists that contain links to a particular set of resources.
See also
- Capability List
- ResourceSync Framework Specification
- Change List
A Change List is an xml-document of type sitemap that enlists resources that have changed since a previous synchronization.
See also
- Change List
- ResourceSync Framework Specification
- configuration
A named set of parameters that constitute all variables needed to synchronize a set of resources. The first configuration will be saved under the name ‘DEFAULT’. Configurations can be loaded, saved, listed and deleted under the File menu. The parameters that are set under a configuration are automatically saved.
See also
- Configuration
- Configure page
- DANS
Data Archiving and Networked Services. Netherlands Institute for Permanent Access to Digital Research Resources. DANS is an institute of the Dutch Academy KNAW and funding organisation NWO.
- description directory
The description directory is an existing directory on the (local or networked) filesystem. In this directory (a copy of) the document that describes the entire site, also known as
.well-known/resourcesyncor Source Description is expected or will be created. If no description directory is given, the document is expected or will be created in the metadata directory.See also
- Configuration
- Configure page
- Describing the Source
- ResourceSync Framework Specification
- Destination
A system that synchronizes itself with a Source’s resources.
See also
- Definitions
- ResourceSync Framework Specification
- EAD
“Encoded Archival Description (..) is a non-proprietary de facto standard for the encoding of finding aids for use in a networked (online) environment. Finding aids are inventories, indexes, or guides that are created by archival and manuscript repositories to provide information about specific collections. While the finding aids may vary somewhat in style, their common purpose is to provide detailed description of the content and intellectual organization of collections of archival materials. EAD allows the standardization of collection information in finding aids within and across repositories.”
– The Library of Congress, Official Site
See also
- EHRI
European Holocaust Research Infrastructure.
“The mission of the European Holocaust Research Infrastructure (EHRI) is to support the Holocaust research community by building a digital infrastructure and facilitating human networks.”
– About EHRI
- incremental changelist strategy
Will increment an existing Change List with the newly found changes.
See also
- Configuration
- Configure page
- metadata directory
The name of the directory where generated sitemaps are stored. The value of metadata directory may consist of multiple path elements. The metadata directory is always relative to the resource directory.
See also
- Configuration
- Configure page
- MPT
- Metadata Publishing Tool
Metadata Publishing Tool (MPT) is an application for publishing resources in conformance with the ResourceSync Framework Specification. Metadata Publishing Tool was developed by Data Archiving and Networked Services (DANS-KNAW) under auspices of the European Holocaust Research Infrastructure (EHRI).
See also
- rspub-core at gitHub
- The base library under MPT
- rspub-gui at gitHub
- The source code of the graphical user interface under MPT
- new changelist strategy
Will create a new Change List at each synchronization run.
See also
- Configuration
- Configure page
- new resourcelist strategy
At each synchronization run a completely new Resource List will be generated.
See also
- Configuration
- Configure page
- plugin directory
In this directory or its subdirectories a search for plugins will be conducted.
See also
- Configuration
- Configure page
- ResourceGateBuilder
- Documentation on rspub-core
- resource
- In web technology, a data resource is anything that can be accessed with a link. It can for instance be a file, (part of) a database or the output of a program.
- resource directory
The base directory on the (local or networked) filesystem where resources that should be synchronized are stored. The resource directory should be chosen careful, because it influences the composition of the URL to the resource.
See also
- Configuration
- Configure page
- Resource List
A Resource List is an xml-document of type sitemap that enlists available resources on a particular site.
See also
- Resource List
- ResourceSync Framework Specification
- ResourceSync
- ResourceSync Framework Specification
The ResourceSync specification describes a synchronization framework for the web consisting of various capabilities that allow third-party systems to remain synchronized with a server’s evolving resources.
See also
- ResourceSync Framework Specification
- Open Archives Initiative ResourceSync Framework Specification
- scp
Secure Copy Protocol. SCP copies files over a secure, encrypted network connection.
See also
- set of resources
“A collection of resources that is made available for synchronization by a Source. A Source may expose one or more such collections and support distinct ResourceSync capabilities for each. Individual resources may be included in more than one set of resources”
– ResourceSync Framework Specification
See also
- Definitions
- ResourceSync Framework Specification
- sitemap
- sitemap protocol
An XML schema for xml-documents that describe the resources of a site. The ResourceSync Framework Specification makes use of this protocol to express the location of resources and to express changes that affected this set of resources. ResourceSync specifies these sitemaps:
- Source Description enumerates the Capability Lists offered by a Source
- Capability List enumerates all capabilities supported by a Source for a specific set of resources
- Resource List lists and describes the resources that a Source makes available for synchronization
- Resource List Index for grouping multiple Resource Lists
- Resource Dump used to transfer resources from the Source in bulk
- Resource Dump Manifest describes the bulk package’s constituents
- Change List contains a description of changes to a Source’s resources
- Change List Index for grouping multiple Change Lists
- Change Dump used to transfer changed resources from the Source in bulk
- Change Dump Manifest describes the bulk package’s constituents
See also
- Sitemap protocol
- Official site
- Source
A server that hosts resources subject to synchronization.
See also
- Definitions
- ResourceSync Framework Specification
- Source Description
In the context of ResourceSync Framework Specification the document at:
{server root}/.well-known/resourcesync
This document describes the site by listing all Capability Lists that are available from the site. This practice is an extension on the well-known URI scheme, also known as RFC5785.
See also
- ResourceSync Well-Known URI
- ResourceSync Framework Specification
- strategy
The strategy defines what kind of sitemap documents will be generated when a synchronization is executed.
See also
- Configuration
- Configure page
- synchronization
- synchronize
Keeping a set of resources at a Destination the same as the set of resources at a Source.
As seen from the Destination this includes copying the original set of resources from the Source and then keeping up to date with additions to, changes of and deletions from the set of resources at the Source.
As seen from a Source this involves providing the resources and the metadata that describe the changes to this set of resources.
See also
- Source perspective
- ResourceSync Framework Specification
- Destination perspective
- ResourceSync Framework Specification
- Execute a synchronization
- Execute page
- trial run
The execution of a synchronization run that will not write sitemaps to disk.
- URL prefix
The URL prefix is the basename of the site, optionally followed by a path segment.
See also
- Configuration
- Configure page
- WebDAV
“Web Distributed Authoring and Versioning (WebDAV) is an extension of the Hypertext Transfer Protocol (HTTP) that allows clients to perform remote Web content authoring operations. WebDAV is defined in RFC 4918 by a working group of the Internet Engineering Task Force.”
– WikipediaSee also
- B2DROP
- EUDAT service
- Mount B2DROP using the WebDAV client
- ownCloud Manual
- well-known URI
RFC5785. An internet standard that defines a path prefix for “well-known locations”
See also