- 21 Oct, 2021 1 commit
-
-
Sophie Wenzel-Teuber authored
Building Deimos takes up too many resources to run two build jobs at the same time. Therefore I added new stages for the CentOS 7 and 8 build respectively, disabling the Authorisation build because that is rather unimportant anyway. Changelog: * Separate test CI stages for CentOS 7 and 8 * Disable 'Authorisation Disabled' test
-
- 20 Oct, 2021 1 commit
-
-
Sophie Wenzel-Teuber authored
Add dependencies as rpms to Deimos, update CI to use them and let CMake create an rpm of Deimos itself Changelog * Add folder dependencies and fill with rpms for all dependencies for CentOS 7 and 8 * Update the CI to install the dependency rpms to a clean CentOS docker image instead of using the superbuild image * Update server to use Proxygen Version 2021.10.11.00 * Add Phobos start scripts to the test folder to setup a database for testing and development * Add subtree for Doxygen XML Parser to recreate the Markdown documentation * Add CPack to Deimos CMakeLists.txt to create an rpm
-
- 08 Oct, 2021 1 commit
-
-
Sophie Wenzel-Teuber authored
Add support of the DELETE HTTP request and call the Phobos function phobos_delete Changelog: * Add new class DeleteRequestHandler and add it to the Factory * Add phobos_delete_cpp to the Phobos wrapper, which calls phobos_delete * Add a delete_object function to Stream/Fifo and Storage/Phobos * Add tests to delete objects
-
- 12 Aug, 2021 1 commit
-
-
Sophie Wenzel-Teuber authored
Changelog: * Update wrapper to new function signatures and structs * Run format and lint * Temporarily remove Centos7 build from CI
-
- 23 Nov, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Create object in backend for buckets to affirm their existence and to store meta data. Support S3 bucket listing. Generalise bucket logic to 'container' where the protocol should not be known. Change log * Disk - append to bucket list instead of overwriting - parse bucket name from object path - store list of objects in each bucket in a unique file - store list of buckets in a file - get_object_list function * use bucket + object path in unit tests * test Fifo::get_bucket_object_list * bucket listing in stream, storage and disk implementation * list buckets using Phobos * test for Phobos bucket listing * function to format ListBuckets response * bucket list and object list request identification * handle bucket list request * list buckets integration test * https confict response if bucket already exists for create_container * bucket list unit tests ignore buckets created by previous test runs * function to reset Phobos descriptor * function to reset Fifo descriptors and storage backend * make sure to deallocate objid if it is allocated before reallocating * function to check if object exists in the Phobos store * function to check if object exists in Disk store * test object_exists functions * create container if object is PUT to a not yet existing one * create container function in PutRequestHandler * remove bucket logic from Disk * remove bucket tests from Phobos_file and Fifo tests * remove bucket logic from Stream and Storage as this is handled elsewhere * handle bucket logic in Protocol Formatter * rename bucket to container where appropriate * support create bucket request * test create bucket request * function to check for container list request * use sane bucket identifier * function to extract container name * use try-catch in create_container function * test that creating an existing container fails * update documentation * flush log messages to logfile immediately
-
- 27 Oct, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
Add a protocol interface to abstract the s3 implementation away from the server. The server handlers now call generic protocol functions on objects created in a protocol factory. Only this factory knows, that the underlying objects are S3. ChangeLog: * add virtual classes for protocols * move s3 specific implementations to own folder/namespace and inherit generic protocol * add protocol_factory to generate s3 objects for the server * move fifo and phobos implementations to own namespace to mirror the same layout as for the protocol
-
- 28 Sep, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Support the list objects functionality of s3. Bucket only GET requests get a response with a list of the objects in the bucket Change log: * add list object functions to the phobos_cpp_wrapper * add listing objects to storage functionality * add bucket object listing wrapper to stream * empty list objects function is disk storage backend to satisfy storage class * string_to_map function to aid returning object_list in storage * remove Catch2 header file * oid_to_key function to parse away bucket from object name * s3_utilities: function to create xml response for list objects * bucket only GET request calls list objects * get_request_handler: respond with object_list for bucket only requests * add object listing to unit tests * integration test for object listing * unit test for empty bucket * integration test for empty bucket * Use bucket and key in unit tests to reflect actual usage * generate random bucket name for testing so number of objects in returned list can be measured * replace assertTrue with assertEqual in integration tests where appropriate * number integration tests to ensure order (alphanumerical) of execution * instruct integration tests to stop on first error * free object meta data struct * fix development flags not being used during debug build * use debug build for CI * use release build for CentOS7 * appease development flags * disable LeakSanitization for CI (does not work on Gitlab-CI) * source relevant env list in CI before_script * export LD_LIBRARY_PATH in CI configure step * pass bucket name to xml parser to avoid attempting to access an object of an empty list * update documentation
-
- 07 Sep, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Integration tests executable using 'make integration_tests'. Clean up CI a bit. Change log: * add integration_tests as a CMake target * use integration_tests make target in CI * update README * CMake command compatible with CentOS7 * CI adjustments
-
- 02 Sep, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
* Update Findspdlog.cmake script to use spdlog's own cmake scripts * Add linking to spdlog::spdlog to all deimos targets * Clean up and update Findphobos.cmake
-
- 24 Aug, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Create a CI job for running the unit tests and integration tests on CentOS7. Change log: * Fix Documentation path * Ci job for CentOS7
-
- 21 Aug, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Change log: * get unit tests and integration tests working for default job * get format job working * satisfy clang-format * get Documentation job working * use custom target_link_system_libraries command to force system includes * statements inside braces to satisfy clang-tidy * remove clang-tidy macro checks
-
- 18 Aug, 2020 2 commits
-
-
Ciarán Ó Rourke authored
Various adjustments to build and pass tests on CentOS8 Change log: * move setting of installation directories to src/CMakeLists.txt (needs to be where target is created) * explicitly find Boost::iosteams * buffer for storing empty file causes exception so add 1 to size
-
Ciarán Ó Rourke authored
Changes to Phobos wrapper to reflect updated Phobos API. Change log: * make unit test a make target again * remove deprecated and redundant target_link_directories call * redesign phobos_cpp_wrapper for new Phobos version * hardcode request of directory storage in Phobos for now
-
- 21 Jul, 2020 1 commit
-
-
Change log: * Close file descriptors later in the process
-
- 07 Jul, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
Changelog: * Add Find scripts to CMake to search for phobos and spdlog * Add integrations tests to cmake by just copying the python code to the build directory * Update Readme to reflect new SuperDeimos layout and add brief description
-
- 03 Jul, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
-
- 02 Jul, 2020 1 commit
-
-
Ciarán Ó Rourke authored
* set install directories through CMake * print built subdirectories during build * set up installation directories * use development flags during debug build * remove authorisation double negative cmake variable * update README
-
- 22 Jun, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
get body length was throwing an exception when the Content-Length header was missing Changelog: * Add more error checks to s3 header class * Add unit tests for s3 header class covering all typical use cases * Add check for Markdown documentation to gitlab CI
-
- 18 Jun, 2020 2 commits
-
-
Ciarán Ó Rourke authored
Rename project from FiPhoboServer to Deimos Change log: * replace all occurances of variations of fiphoboserver with its deimos equivalent
-
Ciarán Ó Rourke authored
Use the spdlog library for project wide logging to a file. Path of logfile is configurable as command line option. Change log: * use spdlog * set global logging to file * logfile as command line option * update README command line option section with logfile option * update README command line option section with default values for options * control log level with CMake build type variable * log received request * get request logging * put request logging * getmd request logging * fifo logging * rename authorisation macro to comply with linter * use rotating files with size limit
-
- 15 Jun, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
Changelog: * in authorisation check if body is existent and signed * add string `UNSIGNED_PAYLOAD` to the canonical request if requested * check in put_request_handler if authorisation is waitig for the body or not
-
- 11 Jun, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
Changelog: * Add `fiphobo:` prefix to all metadata saved by our server, that should not be sent back directly * When sending the metadata, all entries starting with `fiphobo:` are ignored * remove the bucket_name entry from all set metadata functions throughout the server and add the bucketname where all other metdata is set (put handler)
-
- 10 Jun, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
Changelog: * Add content_length to phobos metadata * Send the content_length from phobos metadata as header Content-Length * Add all Phobos Metadata (formerly retrieved only by a HEAD request) as headers to the response to a GET request
-
- 05 Jun, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Authorisation unit tests were being run when the project was built with authorisation disabled. Aurhorisation unit tests are now moved to a dedicated file that is built conditionally. Some changes are also made to the CI YAML file. Change log: * Move authorisation tests to dedicated file * disable authorisation tests when building without authorisation * default CI job runs unit tests and integration tests * add CI job for build w/o authorisation * rename Doxygen job to Documentation
-
- 26 May, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
This adds the CMake variable FIPHOBOSERVER_DISABLE_AUTHORISATION to disable the authentication Changelog: * Add option and variable to CMake * Add ifndef pragmas to request handlers * Update Readme
-
- 13 May, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
All different kinds of errors our server sends are in compliance with Amazon S3 errors. They are now stored in one place and should be accessed from anywhere. Changelog: * add the file s3_errors.h that contains const objects for all errors that might occur (most of them are not checked yet) * change struct error_info into a class and add a function to create the XML from the object's variables * Use these errors in the request handlers * Add an error object to the s3 authorisation that will always contain "Access Denied" or another error if applicable * Update tests to test the code generation and error return values
-
- 11 May, 2020 1 commit
-
-
Changelog: * Build documentation * Add new flag to CMake to build Documentation with Doxygen, but without the Markdown converter * Update CI for new CMake variable * Update README to reflect changes
-
- 07 May, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Outline usage profile of the fiphoboserver executable in the README Change log: * start server daemon subcommand * move appropriate flags to start subcommand * main function forks process given start command * stop command to kill running server process * integration tests: reflect subcommand invoking of server * detail usage in README
-
- 05 May, 2020 1 commit
-
-
Ciarán Ó Rourke authored
Change log: * src/utils directory for configuration, logging, etc * Config class for handling configuration (command line parsing, etc) * include Catch2 and CLI11 as system files so clang-tidy will ignore them * support basic command line options: - hostname (positional, required) - number of threads - http_port - https_port * add configuration to fiphoboserver executable main function * return on exception for incompatible hostname * return after parsing command line arguments if arguments are bad or --help is supplied * don't pass command line arguments to folly * Config returns struct of command line options * accept configuration options from config file * option to write current configuration to file * integration tests: reflect changes to fiphoboserver executable
-
- 24 Apr, 2020 2 commits
-
-
Aaron Dees authored
Add queries to authentication See merge request oilgas/ltfs/fiphoboserver!31
-
-
- 21 Apr, 2020 1 commit
-
-
Aaron Dees authored
S3 authorisation See merge request oilgas/ltfs/fiphoboserver!29
-
- 15 Apr, 2020 1 commit
-
-
Sophie Wenzel-Teuber authored
The signature of amazon users is checked against a text file of known users and requests are accepted or rejected depending on the signature
-
- 14 Apr, 2020 3 commits
-
-
Aaron Dees authored
Resolve "Python Linting" See merge request oilgas/ltfs/fiphoboserver!28
-
Ciarán Ó Rourke authored
Commit log: * add yapf execution to formatting script * run formatting script * rename formatting and linting jobs * install yapf in Format job
-
Aaron Dees authored
Resolve "Add Stress Test" See merge request oilgas/ltfs/fiphoboserver!27
-
- 09 Apr, 2020 3 commits
-
-
Ciarán Ó Rourke authored
Change log: * remove empty get file * add function for PUT, GETMD GET with random key and get_file_name * add stress test involving multiple parallel requests
-
Aaron Dees authored
General improvements See merge request oilgas/ltfs/fiphoboserver!26
-
Ciarán Ó Rourke authored
Change log: * Dockerfile that inherits from an image with dependencies installed * describe build process in README * refer to superfiphoboserver in README * rename m_file_closed to m_stream_completed * formalise README
-
- 08 Apr, 2020 1 commit
-
-
Aaron Dees authored
S3 error handling See merge request oilgas/ltfs/fiphoboserver!23
-