The configuration of the Archive Interface service is done in two files. The REST API utilizes CherryPy and review of their configuration documentation is recommended. The service configuration file is a INI formatted file containing configuration for accessing the HSM backend.
CherryPy Configuration File¶
An example of Archive Interface server CherryPy configuration:
[global] log.screen: True log.access_file: 'access.log' log.error_file: 'error.log' server.socket_host: '0.0.0.0' server.socket_port: 8080 [/] request.dispatch: cherrypy.dispatch.MethodDispatcher() tools.response_headers.on: True tools.response_headers.headers: [('Content-Type', 'application/json')]
Service Configuration File¶
The service configuration is an INI file and an example is as follows:
[posix] ; use id2filename method when accessing a posix endpoint use_id2filename = false [hpss] ; IBM HPSS HSM settings auth = /var/hpss/etc/hpss.unix.keytab sitename = example.com user = hpss.unix [hsm_sideband] ; Oracle HSM Sideband database settings (MySQL) sam_qfs_prefix = /tmp/path schema = schema_name host = host user = user password = pass port = 3306
ID Mapping to File Names¶
The Pacifica software depends on a flat ID space for indexing files. This needs
to map to filenames on the backend storage in a nice way. To limit the number of
files in a single directory (or number of directories in a directory) we use the
archiveinterface.id2filename. This takes a number and breaks it
into bytes. Each byte is then represented in hex and used to build the directory
/39/3039 on the backend file system.
See more documentation in the id2filename module.
There are two ways of running the archive interface, using CherryPy builtin server and uWSGI to run the server.
CherryPy does provide a server and this works for some workloads.
pacifica-archiveinterface -t posix -p 8080 -a 127.0.0.1 --prefix /path
uWSGI is recommended for this service as it performs a lot better for higher throughput workloads.
export PAI_BACKEND_TYPE=posix export PAI_PREFIX=/path uwsgi --http-socket :8080 --master -p 8 --module pacifica.archiveinterface.wsgi
Post Deployment Testing¶
post_deployment_test directory there is a file called
This file will run a series of tests against a deployed archive interface. The test
are ordered so that they post, stage, status, and get files properly.
There are a few global variables at the top of the file that need to be adjusted to each deployment.
export ARCHIVE_URL='http://127.0.0.1:8080/' export LARGE_FILE_SIZE=$(( 1024 * 1024 * 1024)) export MANY_FILES_TEST_COUNT=1000
- ARCHIVE_URL is the URL to the newly deployed archive_interface
- LARGE_FILE_SIZE is the size of the large file to test with (default 1Gib)
- MANY_FILES_TEST_COUNT is the number of small files to spam (default 1000)
pytest -v post_deployment_tests/deployment_test.py
Output will be the status of the tests against the archive interface