Skip to content
This repository was archived by the owner on Dec 4, 2019. It is now read-only.

Further Information and Considerations for Content Providers

Jump to bottom
Mark Allen Matney, Jr edited this page Sep 13, 2017 · 20 revisions

High-level architecture

The software for content providers is structured in the following way:

  • source.py - a single Python script with dependencies, the majority of which are installed via Pip
    • the script may optionally be scheduled to be run via cron, for example
  • source.ini - general configuration options for the script
  • source_logging.ini - logging configuration options

High-level description

source.py does the following, in order:

  • Read config (*.ini) files and process command-line arguments
  • Send HTTP GET requests to OAI-PMH data providers
  • Process responses from data providers
  • Write ResourceSync document files (e.g., SourceDescription, CapabilityList, ResourceList, ChangeList) to the local filesystem at the locations specified by command-line arguments
  • Write log file at the location specified by source_logging.ini, and log to STDOUT

Security considerations

Recommendations

Although we've done our best to mitigate any serious security issues and other bugs, the reality is that this software is to a large extent "experimental", its components built and maintained by a relatively small team of developers. Therefore, in production environments with "mission-critical" services, we highly recommend spinning up a dedicated machine on which to run the software and to host the output files (ResourceSync documents).

Dependencies

The software mostly depends on standard, widely-used Python modules available on PyPI. The only exception is the primary dependency (py-resourcesync), developed primarily by software developers at LANL. There are numerous tests written for py-resourcesync, which is open sourced here: https://github.com/resourcesync/py-resourcesync.

Permissions

As the document roots of many popular web servers (e.g., Tomcat, httpd) are owned by users with elevated privileges, the simplest use of this software is to run it with the privileges required to write ResourceSync document files directly to their final locations (i.e., under the server document root).

If the script is unable to be run with elevated privileges, it can be invoked so that the output files are written outside the document root, as long as the output files are subsequently moved to the correct locations under the document root.

The software does not accept incoming HTTP requests or any other kind of user input, and since it resides on the backend can only be invoked by either the system administrator or some scheduling utility such as cron, both who presumably are trustworthy.

Target audience and input (Usage)

The target audience (indented user) of the software is the administrator of a system that is to become a resourcesync hosting server, or the cron scheduling utility. For example usage, see https://github.com/UCLALibrary/resourcesync-oai-pmh/wiki/Use-Case-Recipes; for detailed usage, please download the software, install dependencies, and run python3 source.py --help.

Destination and output

The output of the software is:

  • INFO-level logging to STDOUT
  • DEBUG-level logging to a file specified in the configuration
  • ResourceSync documents (files) created or updated
Clone this wiki locally