The configuration of the Archive Interface service is done in two files. The REST API utilizes CherryPy and review of their configuration documentation is recommended. The service configuration file is a INI formatted file containing configuration for accessing the HSM backend.
CherryPy Configuration File¶
An example of Archive Interface server CherryPy configuration:
[global] log.screen: True log.access_file: 'access.log' log.error_file: 'error.log' server.socket_host: '0.0.0.0' server.socket_port: 8080 [/] request.dispatch: cherrypy.dispatch.MethodDispatcher() tools.response_headers.on: True tools.response_headers.headers: [('Content-Type', 'application/json')]
Service Configuration File¶
The service configuration is an INI file and an example is as follows:
[posix] ; use id2filename method when accessing a posix endpoint use_id2filename = false [hpss] ; IBM HPSS HSM settings auth = /var/hpss/etc/hpss.unix.keytab sitename = example.com user = hpss.unix [hsm_sideband] ; Oracle HSM Sideband database settings (MySQL) sam_qfs_prefix = /tmp/path schema = schema_name host = host user = user password = pass port = 3306
ID Mapping to File Names¶
The Pacifica software depends on a flat ID space for indexing files. This needs
to map to filenames on the backend storage in a nice way. To limit the number of
files in a single directory (or number of directories in a directory) we use the
archiveinterface.id2filename. This takes a number and breaks it
into bytes. Each byte is then represented in hex and used to build the directory
/39/3039 on the backend file system.
There are two ways of running the archive interface, using CherryPy builtin server and uWSGI to run the server.
CherryPy does provide a server and this works for some workloads.
pacifica-archiveinterface -t posix -p 8080 -a 127.0.0.1 --prefix /path
uWSGI is recommended for this service as it performs a lot better for higher throughput workloads.
export PAI_BACKEND_TYPE=posix export PAI_PREFIX=/path uwsgi --http-socket :8080 --master -p 8 --module pacifica.archiveinterface.wsgi
Post Deployment Testing¶
post_deployment_test directory there is a file called
This file will run a series of tests against a deployed archive interface. The test
are ordered so that they post, stage, status, and get files properly.
There are a few global variables at the top of the file that need to be adjusted to each deployment.
export ARCHIVE_URL='http://127.0.0.1:8080/' export LARGE_FILE_SIZE=$(( 1024 * 1024 * 1024)) export MANY_FILES_TEST_COUNT=1000
- ARCHIVE_URL is the URL to the newly deployed archive_interface
- LARGE_FILE_SIZE is the size of the large file to test with (default 1Gib)
- MANY_FILES_TEST_COUNT is the number of small files to spam (default 1000)
pytest -v post_deployment_tests/deployment_test.py
Output will be the status of the tests against the archive interface