Commit b11992ae authored by Sophie Wenzel-Teuber's avatar Sophie Wenzel-Teuber
Browse files

Modularise S3 protocol

Add a protocol interface to abstract the s3 implementation away from the server. The server handlers now call generic protocol functions on objects created in a protocol factory. Only this factory knows, that the underlying objects are S3.

ChangeLog:
* add virtual classes for protocols
* move s3 specific implementations to own folder/namespace and inherit generic protocol
* add protocol_factory to generate s3 objects for the server
* move fifo and phobos implementations to own namespace to mirror the same layout as for the protocol
parent 2c31093f
Pipeline #2598 passed with stages
in 17 minutes and 25 seconds
......@@ -15,6 +15,7 @@ if(NOT PHOBOS_FOUND)
${CMAKE_INSTALL_PREFIX}/../include/
${CMAKE_INSTALL_PREFIX}/../include/phobos/
${CMAKE_INSTALL_PREFIX}/../phobos/include/
${CMAKE_PREFIX_PATH}/phobos/include/
NAMES phobos_store.h pho_attrs.h
DOC "Phobos headers"
)
......@@ -63,6 +64,7 @@ if(NOT PHOBOS_FOUND)
${CMAKE_INSTALL_PREFIX}/phobos/lib64/
${CMAKE_INSTALL_PREFIX}/../phobos/lib/
${CMAKE_INSTALL_PREFIX}/../phobos/lib64/
${CMAKE_PREFIX_PATH}/phobos/lib/
NAMES libphobos_store.so libphobos_store.la
DOC "Phobos library"
)
......@@ -74,4 +76,6 @@ if(NOT PHOBOS_FOUND)
set(GLIB_INCLUDE_DIR ${GLIB_INCLUDE_DIR} ${GLIB_CONFIG_INCLUDE_DIR})
link_directories(${PHOBOS_LIBRARY_DIR})
endif()
\ No newline at end of file
......@@ -40,7 +40,8 @@ HTML_OUTPUT = html
HTML_FILE_EXTENSION = .html
GENERATE_TREEVIEW = YES
# Latex font size
# Latex
GENERATE_LATEX = NO
FORMULA_FONTSIZE = 10
# Macro options
......
......@@ -6,7 +6,7 @@ exception class for all user defined exceptions in Deimos
## Inheritance:
Is inherited by [deimos::IOException][deimos-IOException], [deimos::storage::PhobosException][deimos-storage-PhobosException].
Is inherited by [deimos::IOException][deimos-IOException], [deimos::storage::phobos::PhobosException][deimos-storage-phobos-PhobosException].
## Protected Attributes
| Name | Description |
......@@ -141,4 +141,4 @@ return the inner error value that caused this exception
[Go to Top](#deimos-DeimosException)
[deimos-IOException]:./IOException.md
[deimos-storage-PhobosException]:./storage/PhobosException.md
[deimos-storage-phobos-PhobosException]:./storage/phobos/PhobosException.md
......@@ -20,7 +20,6 @@ Inherits from RequestHandler.
| [onEgressPaused](#deimos-GetRequestHandler-onEgressPaused) | called when the queue is full. |
| [onEgressResumed](#deimos-GetRequestHandler-onEgressResumed) | called when the queue is not longer full. |
| [GetRequestHandler](#deimos-GetRequestHandler-GetRequestHandler) | Constructor for stream class initialization. |
| [list_objects](#deimos-GetRequestHandler-list_objects) | function called in the case of a list_objects request |
......@@ -200,7 +199,7 @@ Contents are copies from a proxygen example
[Go to Top](#deimos-GetRequestHandler)
### <a name='deimos-GetRequestHandler-GetRequestHandler' /> public deimos::GetRequestHandler::GetRequestHandler (std::unique_ptr< stream::Stream > input_stream)
### <a name='deimos-GetRequestHandler-GetRequestHandler' /> public deimos::GetRequestHandler::GetRequestHandler (std::unique_ptr< protocol::Protocol_factory > protocol_factory, std::unique_ptr< stream::Stream > input_stream)
Constructor for stream class initialization.
......@@ -210,6 +209,7 @@ Constructor for stream class initialization.
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| std::unique_ptr< [protocol::Protocol_factory][deimos-protocol-Protocol_factory] > | protocol_factory | Factory to create the https interface specific objects |
| std::unique_ptr< [stream::Stream][deimos-stream-Stream] > | input_stream | [stream::Stream][deimos-stream-Stream] class instance to initialize the server |
......@@ -229,25 +229,6 @@ Constructor for stream class initialization.
[Go to Top](#deimos-GetRequestHandler)
### <a name='deimos-GetRequestHandler-list_objects' /> public void deimos::GetRequestHandler::list_objects () noexcept
function called in the case of a list_objects request
Objects for a given bucket are added to the response
[Go to Top](#deimos-GetRequestHandler)
[deimos-protocol-Protocol_factory]:./protocol/Protocol_factory.md
[deimos-stream-Stream]:./stream/Stream.md
[todo]:./../todo.md#todo
......@@ -157,7 +157,7 @@ Not supported in our case!
[Go to Top](#deimos-GetmdRequestHandler)
### <a name='deimos-GetmdRequestHandler-GetmdRequestHandler' /> public deimos::GetmdRequestHandler::GetmdRequestHandler (std::unique_ptr< stream::Stream > input_stream)
### <a name='deimos-GetmdRequestHandler-GetmdRequestHandler' /> public deimos::GetmdRequestHandler::GetmdRequestHandler (std::unique_ptr< protocol::Protocol_factory > protocol_factory, std::unique_ptr< stream::Stream > input_stream)
Constructor for stream class initialization.
......@@ -167,6 +167,7 @@ Constructor for stream class initialization.
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| std::unique_ptr< [protocol::Protocol_factory][deimos-protocol-Protocol_factory] > | protocol_factory | Factory to create the https interface specific objects |
| std::unique_ptr< [stream::Stream][deimos-stream-Stream] > | input_stream | [stream::Stream][deimos-stream-Stream] class instance to initialize the server |
......@@ -186,5 +187,6 @@ Constructor for stream class initialization.
[Go to Top](#deimos-GetmdRequestHandler)
[deimos-protocol-Protocol_factory]:./protocol/Protocol_factory.md
[deimos-stream-Stream]:./stream/Stream.md
[todo]:./../todo.md#todo
......@@ -157,7 +157,7 @@ Not supported in our case!
[Go to Top](#deimos-PutRequestHandler)
### <a name='deimos-PutRequestHandler-PutRequestHandler' /> public deimos::PutRequestHandler::PutRequestHandler (std::unique_ptr< stream::Stream > input_stream)
### <a name='deimos-PutRequestHandler-PutRequestHandler' /> public deimos::PutRequestHandler::PutRequestHandler (std::unique_ptr< protocol::Protocol_factory > protocol_factory, std::unique_ptr< stream::Stream > input_stream)
Constructor for stream class initialization.
......@@ -167,6 +167,7 @@ Constructor for stream class initialization.
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| std::unique_ptr< [protocol::Protocol_factory][deimos-protocol-Protocol_factory] > | protocol_factory | Factory to create the https interface specific objects |
| std::unique_ptr< [stream::Stream][deimos-stream-Stream] > | input_stream | [stream::Stream][deimos-stream-Stream] class instance to initialize the server |
......@@ -186,5 +187,6 @@ Constructor for stream class initialization.
[Go to Top](#deimos-PutRequestHandler)
[deimos-protocol-Protocol_factory]:./protocol/Protocol_factory.md
[deimos-stream-Stream]:./stream/Stream.md
[todo]:./../todo.md#todo
......@@ -18,7 +18,7 @@
## Namespaces
| Name | Description |
| ---- | ---- |
| [s3_utilities](./s3_utilities/index.md) | namespace for S3 specific utilities |
| [protocol](./protocol/index.md) | namespace for http interface specific functionality |
| [storage](./storage/index.md) | namespace for storage classes that belong to / inherit from from [deimos::storage::Storage][deimos-storage-Storage] |
| [stream](./stream/index.md) | namespace for stream classes that belong to / inherit from from [deimos::stream::Stream][deimos-stream-Stream] |
| [util](./util/index.md) | namespace for utility classes |
......@@ -28,6 +28,6 @@
[deimos-DeimosException]:./DeimosException.md
[deimos-IOException]:./IOException.md
[deimos-UnsupportedRequestHandler]:./UnsupportedRequestHandler.md
[deimos-storage-PhobosException]:./storage/PhobosException.md
[deimos-storage-Storage]:./storage/Storage.md
[deimos-storage-phobos-PhobosException]:./storage/phobos/PhobosException.md
[deimos-stream-Stream]:./stream/Stream.md
# <a name='deimos-protocol-Authorisation' /> public deimos::protocol::Authorisation
class to check the authorisation of a request
## Inheritance:
Is inherited by [deimos::protocol::s3::S3_authorisation][deimos-protocol-s3-S3_authorisation].
## Public Functions
| Name | Description |
| ---- | ---- |
| [Authorisation](#deimos-protocol-Authorisation-Authorisation) | default constructor |
| [~Authorisation](#deimos-protocol-Authorisation-~Authorisation) | default destructor |
| [authorise](#deimos-protocol-Authorisation-authorise) | main method to run the authorisation algorithm |
| [add_chunk](#deimos-protocol-Authorisation-add_chunk) | add a chunk of data to the payload |
| [is_valid](#deimos-protocol-Authorisation-is_valid) | checks if the authorisation was successful |
| [get_status](#deimos-protocol-Authorisation-get_status) | returns the current status of the authorisation |
| [get_error](#deimos-protocol-Authorisation-get_error) | returns the internal error |
## Public Functions
### <a name='deimos-protocol-Authorisation-Authorisation' /> public deimos::protocol::Authorisation::Authorisation ()
default constructor
#### Qualifiers:
* inline
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-~Authorisation' /> public deimos::protocol::Authorisation::~Authorisation ()
default destructor
#### Qualifiers:
* inline
* virtual
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-authorise' /> public [Authorisation_status][deimos-protocol-Authorisation_status] deimos::protocol::Authorisation::authorise (const Headers &headers)=0
main method to run the authorisation algorithm
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| const [Headers][deimos-protocol-Headers] & | headers | of the message to authenticate |
#### Returns:
| Type | Description |
| ---- | ---- |
| [Authorisation_status][deimos-protocol-Authorisation_status] | new status of the authorisation |
#### Qualifiers:
* virtual
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-add_chunk' /> public void deimos::protocol::Authorisation::add_chunk (std::string chunk)=0
add a chunk of data to the payload
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| std::string | chunk | the string to add |
Since the whole body has to be hashed in order to verify the signature of the message each chunk of data has to be added here in order.
It lies in the responsibily of the implementation of this function to decide if this functionality is used or not.
#### Qualifiers:
* virtual
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-is_valid' /> public bool deimos::protocol::Authorisation::is_valid () const =0
checks if the authorisation was successful
#### Returns:
| Type | Description |
| ---- | ---- |
| bool | true if it has been valid, false if it hasn't started, it is in progress, or if an error occurred. |
#### Qualifiers:
* const
* virtual
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-get_status' /> public [Authorisation_status][deimos-protocol-Authorisation_status] deimos::protocol::Authorisation::get_status () const =0
returns the current status of the authorisation
#### Returns:
| Type | Description |
| ---- | ---- |
| [Authorisation_status][deimos-protocol-Authorisation_status] | the current status of the authorisaton |
#### Qualifiers:
* const
* virtual
[Go to Top](#deimos-protocol-Authorisation)
### <a name='deimos-protocol-Authorisation-get_error' /> public const [Error_info][deimos-protocol-Error_info] * deimos::protocol::Authorisation::get_error () const =0
returns the internal error
#### Returns:
| Type | Description |
| ---- | ---- |
| const [Error_info][deimos-protocol-Error_info] * | the error that might have occurred in the process |
#### Qualifiers:
* const
* virtual
[Go to Top](#deimos-protocol-Authorisation)
[deimos-protocol-Authorisation_status]:./index.md#deimos-protocol-Authorisation_status
[deimos-protocol-Error_info]:./Error_info.md
[deimos-protocol-Headers]:./Headers.md
[deimos-protocol-s3-S3_authorisation]:./s3/S3_authorisation.md
# <a name='deimos-protocol-Error_info' /> public deimos::protocol::Error_info
(virtual) class to contain all the information needed to send a proper error message
## Inheritance:
Is inherited by [deimos::protocol::s3::S3_error_info][deimos-protocol-s3-S3_error_info].
## Public Attributes
| Name | Description |
| ---- | ---- |
| [https_error_code](#deimos-protocol-Error_info-https_error_code) | https error code (400s or 500s) |
| [https_error_identifier](#deimos-protocol-Error_info-https_error_identifier) | readable string for the code [https_error_code][deimos-protocol-Error_info-https_error_code] |
| [https_error_message](#deimos-protocol-Error_info-https_error_message) | human readable message with information on the error |
## Public Functions
| Name | Description |
| ---- | ---- |
| [Error_info](#deimos-protocol-Error_info-Error_info) | default constructor - intitialises with `internal error` |
| [Error_info](#deimos-protocol-Error_info-Error_info-1) | constructor for filling the basic information directly |
| [get_error_response_body](#deimos-protocol-Error_info-get_error_response_body) | created a message containing all the error information |
| [operator==](#deimos-protocol-Error_info-operator==) | equality operator |
| [operator!=](#deimos-protocol-Error_info-operator!=) | inequality operator |
## Public Attributes
### <a name='deimos-protocol-Error_info-https_error_code' /> public deimos::protocol::Error_info::https_error_code
https error code (400s or 500s)
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-https_error_identifier' /> public deimos::protocol::Error_info::https_error_identifier
readable string for the code [https_error_code][deimos-protocol-Error_info-https_error_code]
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-https_error_message' /> public deimos::protocol::Error_info::https_error_message
human readable message with information on the error
[Go to Top](#deimos-protocol-Error_info)
## Public Functions
### <a name='deimos-protocol-Error_info-Error_info' /> public deimos::protocol::Error_info::Error_info ()
default constructor - intitialises with `internal error`
#### Qualifiers:
* inline
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-Error_info-1' /> public deimos::protocol::Error_info::Error_info (const int error_code, std::string error_identifier, std::string error_message)
constructor for filling the basic information directly
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| const int | error_code | https error code (400s or 500s) |
| std::string | error_identifier | readable string for the code [https_error_code][deimos-protocol-Error_info-https_error_code] |
| std::string | error_message | human readable message with information on the error |
#### Qualifiers:
* inline
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-get_error_response_body' /> public std::string deimos::protocol::Error_info::get_error_response_body (const std::string request) const =0
created a message containing all the error information
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| const std::string | request | the request that was processed when this error occurred |
#### Returns:
| Type | Description |
| ---- | ---- |
| std::string | a string containg an error message compatible with the underlying protocol |
This must be implemented for every protocol as the layout/structure will change even if the error message itself is transferable
#### Qualifiers:
* const
* virtual
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-operator==' /> public bool deimos::protocol::Error_info::operator== (const Error_info &other) const
equality operator
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| const [Error_info][deimos-protocol-Error_info] & | other | the other error info object to compare |
#### Returns:
| Type | Description |
| ---- | ---- |
| bool | |
return true if all inner members are equal, false otherwise
#### Qualifiers:
* const
* inline
[Go to Top](#deimos-protocol-Error_info)
### <a name='deimos-protocol-Error_info-operator!=' /> public bool deimos::protocol::Error_info::operator!= (const Error_info &other) const
inequality operator
#### Parameters:
| Type | Name | Description |
| ---- | ---- | ---- |
| const [Error_info][deimos-protocol-Error_info] & | other | the other error info object to compare |
#### Returns:
| Type | Description |
| ---- | ---- |
| bool | |
return true if any inner member is different, false otherwise (all equal)
#### Qualifiers:
* const
* inline
[Go to Top](#deimos-protocol-Error_info)
[deimos-protocol-Error_info]:./Error_info.md
[deimos-protocol-Error_info-https_error_code]:./Error_info.md#deimos-protocol-Error_info-https_error_code
[deimos-protocol-s3-S3_error_info]:./s3/S3_error_info.md
# <a name='deimos-protocol-Formatter' /> public deimos::protocol::Formatter
helper class for formatting strings according to the implementing protocol
## Inheritance:
Is inherited by [deimos::protocol::s3::S3_formatter][deimos-protocol-s3-S3_formatter].