Changelog:
* Add content_length to phobos metadata
* Send the content_length from phobos metadata as header Content-Length
* Add all Phobos Metadata (formerly retrieved only by a HEAD request) as headers to the response to a GET request

Authorisation unit tests were being run when the project was built with authorisation disabled. Aurhorisation unit tests are now moved to a dedicated file that is built conditionally. Some changes are also made to the CI YAML file.

Change log:
* Move authorisation tests to dedicated file
* disable authorisation tests when building without authorisation
* default CI job runs unit tests and integration tests
* add CI job for build w/o authorisation
* rename Doxygen job to Documentation

This adds the CMake variable FIPHOBOSERVER_DISABLE_AUTHORISATION to disable the authentication

Changelog:
* Add option and variable to CMake
* Add ifndef pragmas to request handlers
* Update Readme

All different kinds of errors our server sends are in compliance with Amazon S3 errors. They are now stored in one place and should be accessed from anywhere.

Changelog:
* add the file s3_errors.h that contains const objects for all errors that might occur (most of them are not checked yet)
* change struct error_info into a class and add a function to create the XML from the object's variables
* Use these errors in the request handlers
* Add an error object to the s3 authorisation that will always contain "Access Denied" or another error if applicable
* Update tests to test the code generation and error return values

Sophie Wenzel-Teuber authored May 11, 2020 and Aaron Dees committed May 11, 2020

Changelog:
* Build documentation
* Add new flag to CMake to build Documentation with Doxygen, but without the Markdown converter
* Update CI for new CMake variable
* Update README to reflect changes

Outline usage profile of the fiphoboserver executable in the README

Change log:
* start server daemon subcommand
* move appropriate flags to start subcommand
* main function forks process given start command
* stop command to kill running server process
* integration tests: reflect subcommand invoking of server
* detail usage in README

Change log:
* src/utils directory for configuration, logging, etc
* Config class for handling configuration (command line parsing, etc)
* include Catch2 and CLI11 as system files so clang-tidy will ignore
  them
* support basic command line options:
      - hostname (positional, required)
      - number of threads
      - http_port
      - https_port
* add configuration to fiphoboserver executable main function
* return on exception for incompatible hostname
* return after parsing command line arguments if arguments are bad
  or --help is supplied
* don't pass command line arguments to folly
* Config returns struct of command line options
* accept configuration options from config file
* option to write current configuration to file
* integration tests: reflect changes to fiphoboserver executable

Add queries to authentication

See merge request oilgas/ltfs/fiphoboserver!31

S3 authorisation

See merge request oilgas/ltfs/fiphoboserver!29

The signature of amazon users is checked against a text file of known users and requests are accepted or rejected depending on the signature

Resolve "Python Linting"

See merge request oilgas/ltfs/fiphoboserver!28

Commit log:
* add yapf execution to formatting script
* run formatting script
* rename formatting and linting jobs
* install yapf in Format job

Resolve "Add Stress Test"

See merge request oilgas/ltfs/fiphoboserver!27

Change log:
* remove empty get file
* add function for PUT, GETMD GET with random key and get_file_name
* add stress test involving multiple parallel requests

General improvements

See merge request oilgas/ltfs/fiphoboserver!26

Change log:
* Dockerfile that inherits from an image with dependencies installed
* describe build process in README
* refer to superfiphoboserver in README
* rename m_file_closed to m_stream_completed
* formalise README

S3 error handling

See merge request oilgas/ltfs/fiphoboserver!23

Sophie Wenzel-Teuber authored Apr 08, 2020 and Aaron Dees committed Apr 08, 2020

All errors returned from phobos now have at least a little distinct error messages (as far as the return values permit it) and send an S3 readable XML along with the error number.

Resolve "Add Integration Tests"

See merge request oilgas/ltfs/fiphoboserver!25

* separate directories for unit tests and integration tests
* text files directory for testing server functions
* module for s3 utility functions
* module for file comparison utility functions
* module for generating random keys and meta data
* handle server executable within tests
* basic test for PUT and GET
* PUT and GET test for zero-byte file
* add integration testing to CI
* correct Clang Tidy job
* PUT command supports optional meta data
* test for GETMD
* test that GET with bad key fails
* test that PUT with used key fails

Resolve "CI"

See merge request oilgas/ltfs/fiphoboserver!22

* src/storage/phobos_file.hexplicit copy constructors to avoid double free of xd_objid
* tests: reflect meta data change to std::map from std::vector
* run Clang Format
* resolve Clang Tidy warnings
* add Gitlab-CI
* use fiphoboserver docker image
* update DockerHub image with Doxygen requirements
* CI: Clang Format job
* CI: Clang Tidy job
* CI: Doxygen build job
* CI: Unit Tests job

Cleanup codebase

See merge request oilgas/ltfs/fiphoboserver!20

Sophie Wenzel-Teuber authored Mar 31, 2020 and Aaron Dees committed Mar 31, 2020

Add a creation timestamp to the metadata and return only original Metadata in getmd (not the server related stuff)

Resolve "Add Unit Tests"

See merge request oilgas/ltfs/fiphoboserver!17

* utility functions for tests (utils.h, utils.cc)
* copy test text files to test binary directory during build
* tests for Phobos_file constructors
* tests for Phobos_file PUT and GET
* Phobos_file PUT and GET tests for empty file
* Phobos_file GETMD test
* in-memory (disk) storage solution for testing stream objects
* Fifo file descriptors must be copied explicitly as they are now an array
* Fifo tests for PUT and GET
* repeat Fifo tests for empty file
* Fifo tests for GETMD
* tests for Fifo object construction
* need to determine file size before db_get
* Phobos_file PUT  test for bad file descriptor exception
* Phobos_file PUT test for used object_id exception
* Phobos_file GETMD test for bad object_id
* Phobos_file GET test for bad object_id
* meta data is only written upon PUT operation in disk storage

Fix a documentation build issue: Now builds correctly without documentation enabled (Cmake variable CUSTOM_DOCU_PATH)

This reverts commit 1c7a83c1

Minor Code Improvements

See merge request oilgas/ltfs/fiphoboserver!19

Documentation

See merge request oilgas/ltfs/fiphoboserver!13

Resolve "Add Clang-Tidy"

See merge request oilgas/ltfs/fiphoboserver!16

Change log:
* use std::move to return from get_meta_data
* Fifo::put: replace Folly object argument with void pointer and size_t as arguments and return ssize_t
* Fifo::put considers rc < count to be an exception
* run Clang Tidy
* rename i_ps to IPs in main.cc