ExSeisDat issueshttps://git.ichec.ie/oilgas/ExSeisDat/-/issues2020-05-28T23:20:24Zhttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/119Add unit test coverage2020-05-28T23:20:24ZPádraig Ó ConbhuíAdd unit test coverageAdd coverage for unit test files and code coverage for individual unit test execution.Add coverage for unit test files and code coverage for individual unit test execution.ExSeisDat Cleanuphttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/116Identify repeated patterns in the project that should be abstracted into util...2019-07-16T15:47:29ZPádraig Ó ConbhuíIdentify repeated patterns in the project that should be abstracted into utilities.e.g. block decomposing data for use with collective routines.e.g. block decomposing data for use with collective routines.https://git.ichec.ie/oilgas/ExSeisDat/-/issues/112Generated .gitlab-ci.yml files for matrix builds.2019-05-01T14:34:55ZPádraig Ó ConbhuíGenerated .gitlab-ci.yml files for matrix builds.To cleanly test combinations of compilers, flags, and libraries, a "matrix build" would typically be used.
Gitlab CI doesn't support matrix builds, instead requiring each case be written explicitly.
A script should be written to define...To cleanly test combinations of compilers, flags, and libraries, a "matrix build" would typically be used.
Gitlab CI doesn't support matrix builds, instead requiring each case be written explicitly.
A script should be written to define all the combinations needed and generate the appropriate .gitlab-ci.yml file. A pipeline test should also be added to test the current .gitlab-ci.yml file is equivalent to the file generated by the generator, to ensure it's up-to-date.https://git.ichec.ie/oilgas/ExSeisDat/-/issues/111Seis Format2019-03-04T21:21:44ZPádraig Ó ConbhuíSeis FormatAdd support for reading / writing in Shearwater's `Seis` format.Add support for reading / writing in Shearwater's `Seis` format.https://git.ichec.ie/oilgas/ExSeisDat/-/issues/107Rewrite tests using Catch22019-03-07T13:13:35ZPádraig Ó ConbhuíRewrite tests using Catch2As a user of the library, I would like some proof that the library does what it says it does. I'd also like some advanced examples of how to use the library.
The current test suite suggests it does, but its spaghetti-layout makes that i...As a user of the library, I would like some proof that the library does what it says it does. I'd also like some advanced examples of how to use the library.
The current test suite suggests it does, but its spaghetti-layout makes that impossible to verify by eye.
Rewriting the tests for the core features needed to implement `ReadSEGY` and `WriteSEGY` with a framework like Catch2, with clarity of test cases in mind, should fulfill these needs.Aaron DeesAaron Deeshttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/106Clean up Python tests and bindings2019-01-07T11:42:16ZChristopher WernerClean up Python tests and bindingshttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/105Integrate Python bindings with cmake2019-01-07T11:40:55ZChristopher WernerIntegrate Python bindings with cmakeFollowing the initial bindings of python to C++, ensure it can be integrated with cmakeFollowing the initial bindings of python to C++, ensure it can be integrated with cmakehttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/103SEIS to SEGY (serial)2018-11-06T14:07:10ZChristopher WernerSEIS to SEGY (serial)S-01719
As a ExSeisDat Developer I want to write a SEIS to SEGY utility with a serial implantation for reading seis based implementation to read SEIS and ExSeisDat to write the SEGY file because I want to better understand the I/O patte...S-01719
As a ExSeisDat Developer I want to write a SEIS to SEGY utility with a serial implantation for reading seis based implementation to read SEIS and ExSeisDat to write the SEGY file because I want to better understand the I/O patterns of SEIS while also providing an initial version of the SEIS to SEGY converter.
**PreDev Notes:**
Description of the compression algorithm:
Seis traces can be stored as 1 byte (int16), 2 byte (int32), or 4 byte (double) elements. When the traces are compressed from doubles, they are sectioned into groups (packets) that each get a scalar, equal to the largest number able to be stored in the compressed integer divided by the largest (magnitude) number in the packet. Every other element in the packet is multiple by that scalar.
**Implementation Notes:**
The JSON file is parsed when the Seis reader is initialised and the header descriptors within the JSON file are also parsed at this time. Since the Seis header entries are self describing, they do not require the complex rule structure that SEGY did. Rather, a PIOL_META value can be mapped to the default seis header name and then simply read in. It reads each header parameter individually (i.e. it does not read in the entire file and then parse it). The traces are read in, using the parsed JSON header to determine the compression type and packet size.
Further improvements to reader:
* Currently the seis reader does not read in files with multiple .hdr/.tr subfiles.
* Currently the seis header reads in all entries for a particular metadata entry; it cannot get a header value from a single trace.
* Currently the seis header does not read in the entire .hdr and then parse the data, it only reads in the values used.
* Needs to write a checksum "test" to show that it works
* Needs to write utility SEIS to SEGY
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
* Reader should read in the type of SEIS files used by Tullow
* Checksums for original SEGY and SEIS to SEGY should be the same
**Tasks:**
* [ ] Review
* [x] Read Json file
* [x] Read tr file
* [ ] Read HDR file (in prog)
* [ ] SEIS to SEGY util (in prog)
CREATED ON - 01/05/2018March 2019 Roadmap2019-03-01https://git.ichec.ie/oilgas/ExSeisDat/-/issues/102SEGY to SEIS (serial)2018-11-06T14:07:02ZChristopher WernerSEGY to SEIS (serial)S-01713
As a ExSeisDat Developer I want to write a SEIS to SEGY utility using MPI IO to read because I want to better understand the I/O patterns of SEIS while also providing an initial version of the SEGY to SEIS converter.
**PreDev ...S-01713
As a ExSeisDat Developer I want to write a SEIS to SEGY utility using MPI IO to read because I want to better understand the I/O patterns of SEIS while also providing an initial version of the SEGY to SEIS converter.
**PreDev Notes:**
{- ADDME -}
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
* SEIS file written should be format that is usable by Tullow
* SEIS files that are tested should be able to be read by Reveal and converted into SEG-Y; the SEG-Y file should be the equivalent to the original.
**Done:**
Currently, we can write traces into the tr files serially and headers into hdr files serially. Writing data into the seis file is partially complete, and data needs to be stored into classes.
**Tasks:**
* [ ] Review
* [x] Write Traces into .tr files
* [x] Write Headers into .hd file
* [x] Write Metadata into .seis metadata file
* [x] Write particular headers into .db files
* [ ] Write complete SEGY to SEIS utility
CREATED ON - 03/04/2018March 2019 Roadmap2019-03-01https://git.ichec.ie/oilgas/ExSeisDat/-/issues/100SEIS Rule Structure (P2)2018-11-06T13:56:30ZChristopher WernerSEIS Rule Structure (P2)S-01739
As a ExSeisDat Developer I want to add the SEIS format to the rule structure because we want ExSeisPIOL to be able to parse SEIS header and trace files.
**PreDev Notes:**
The inherent SEG-Y behaviour in the rule structure was ...S-01739
As a ExSeisDat Developer I want to add the SEIS format to the rule structure because we want ExSeisPIOL to be able to parse SEIS header and trace files.
**PreDev Notes:**
The inherent SEG-Y behaviour in the rule structure was removed so that the rule structure calls to a separate file to get the SEG-Y offsets (similar to SEIS). Therefore, the implementation for SEIS should be straight forward.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
{- ADDME -}
CREATED ON 03/08/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/99Code Coverage Tool2018-11-06T13:55:50ZChristopher WernerCode Coverage ToolS-01734
As a ExSeisDat Developer I want a code coverage tool to check how much of the code is covered by unit tests and integration tests.
A given percentage of the code should be covered by tests, with that value ratcheted to reduce t...S-01734
As a ExSeisDat Developer I want a code coverage tool to check how much of the code is covered by unit tests and integration tests.
A given percentage of the code should be covered by tests, with that value ratcheted to reduce the uncovered code over time.
**PreDev Notes:**
It looks like clang has a coverage tool.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
When tests are run, the coverage of the tests run should also be checked. If below a certain value, the test should fail.
CREATED ON - 27/06/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/98WriteSEGY broken for rules without full extent2018-11-06T13:55:59ZChristopher WernerWriteSEGY broken for rules without full extentD-01135
**PreDev Notes:**
The SEGY header offsets held in Tr are 1-indexed. This 1-indexing is handled throughout the code, but not sufficiently consistently.
All indices in the project should be 0-indexed.
One compromise between 1 and...D-01135
**PreDev Notes:**
The SEGY header offsets held in Tr are 1-indexed. This 1-indexing is handled throughout the code, but not sufficiently consistently.
All indices in the project should be 0-indexed.
One compromise between 1 and 0 indexed is to write the indices in a 1-indexed -1 form.
``
enum class Indices {
index_one = 1 - 1,
index_two = 5 - 1,};
``
etc. where the index is written in 1-indexed and immediately has 1 subtracted.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
{- ADDME -}
CREATED ON 28/05/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/97SEIS Rule Structure (P1)2018-11-06T14:06:54ZChristopher WernerSEIS Rule Structure (P1)As a ExSeisDat Developer I want a SEIS specific Rule structure for reading / writing SEIS file headers.
**Pre-Dev Notes:**
The current Rule structure can be renamed to SEGY_Rule. An interface Rule class can replace SEGY_Rule, with SEGY...As a ExSeisDat Developer I want a SEIS specific Rule structure for reading / writing SEIS file headers.
**Pre-Dev Notes:**
The current Rule structure can be renamed to SEGY_Rule. An interface Rule class can replace SEGY_Rule, with SEGY_Rule inheriting from it. And a SEGY_Rule instance should be attached to the SEGY reader / writer, and a SEIS_Rule instance attached to the SEIS reader / writer.
We will only worry about standard headers for the moment.
**Implementation Notes:**
The hard-coded SEGY header byte locations have been removed from the existing rule structure. Now the SEG-Y rules get the byte offsets from a separate file in a similar fashion to the SEIS rules.
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
Coordinate Headers for SEGY (source X, Source Y, Receiver X, Receiver Y) was being read in as a float, while it was being stored in the SEGY file as a double. They are now read in as doubles to prevent precision loss.
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
{- ADDME -}
CREATED ON 06/06/2018 March 2019 Roadmaphttps://git.ichec.ie/oilgas/ExSeisDat/-/issues/95Auto build and deploy documentation2018-11-06T13:57:24ZChristopher WernerAuto build and deploy documentationS-01724
As a ExSeisDat Developer I want the Doxygen documentation automatically built and deployed on GitHub so the documentation is available and up to date.
**PreDev Notes:**
Using Travis CI integration with GitHub, it should be pos...S-01724
As a ExSeisDat Developer I want the Doxygen documentation automatically built and deployed on GitHub so the documentation is available and up to date.
**PreDev Notes:**
Using Travis CI integration with GitHub, it should be possible to add a script to build the docs, then add them to a GitHub Pages repo for ExSeisDat, likely at ichec.github.io/exseisdat, whenever a push is made to the devel branch.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
Pushing code to the devel branch automatically builds and updates the online documentation for the head of the devel branch.
CREATED ON 08/05/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/94Tooling for documenting tests and cross-referencing them with the code they test2018-11-06T13:57:00ZChristopher WernerTooling for documenting tests and cross-referencing them with the code they testS-01743
As a ExSeisDat Developer I want the tests to be documentable, with a cross-reference between functions and their unit tests.
**PreDev Notes:**
Doxygen has capabilities for cross-referencing. It looks like there's a \test comma...S-01743
As a ExSeisDat Developer I want the tests to be documentable, with a cross-reference between functions and their unit tests.
**PreDev Notes:**
Doxygen has capabilities for cross-referencing. It looks like there's a \test command, but it's not clear if it does the cross-referencing.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
The test documentation, and the test to function cross reference should reside with the test code.
When looking at a function in the documentation, a link to the relevant test or tests should be available.
CREATED ON 03/08/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/93CMake project configuration for external projects2018-11-06T13:55:11ZChristopher WernerCMake project configuration for external projectsS-01730
As a ExSeisDat Developer I want to add a CMake project configuration script to make adding ExSeisDat to external (CMake-based) projects easier.
**PreDev Notes:**
A project configuration script can save information like the com...S-01730
As a ExSeisDat Developer I want to add a CMake project configuration script to make adding ExSeisDat to external (CMake-based) projects easier.
**PreDev Notes:**
A project configuration script can save information like the compiler and flags used during compilation, along with definitions used, libraries linked, and installation directories.
This information can then be accessed from a CMake script using the `find_package` command.
See tutorials on making a projectconfig.cmake file:
https://gitlab.kitware.com/cmake/community/wikis/doc/tutorials/How-to-create-a-ProjectConfig.cmake-file
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
A CMake configure script for an external project using ExSeisDat can be written (assuming exseisdat is built and installed somewhere findable):
~~~
cmake_minimum_required(VERSION 3.5.2)
find_package(exseisdat REQUIRED)
add_executable(my_executable my_executable.cc)
target_link_libraries(my_executable exseisdat)
~~~
CREATED ON: 28/05/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/92Add code duplication checker2018-11-06T13:54:42ZChristopher WernerAdd code duplication checkerS-01725
As a ExSeisDat Developer I want to include a test for code duplication to maximize the maintainability of the code.
**PreDev Notes:**
This is needed for the review process, which says code duplication should be below some as-y...S-01725
As a ExSeisDat Developer I want to include a test for code duplication to maximize the maintainability of the code.
**PreDev Notes:**
This is needed for the review process, which says code duplication should be below some as-yet unnamed value.
Need to find a good tool for this. Ideally it could be run from a developer's own computer.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
It should be possible to run a test which returns how much of the code is duplicated. The test should fail if it's over some number, and that number should be tunable.
CREATED ON - 08/05/2018https://git.ichec.ie/oilgas/ExSeisDat/-/issues/91Create C and C++ examples2018-11-06T14:05:37ZChristopher WernerCreate C and C++ examplesS-01704
As a Tullow Geophysicist I want examples of how to use ExSeisDat in C and C++ environments, as I want to use the library and may not have experience in the language
**PreDev Notes:**
Make sure examples have both C and C++ vers...S-01704
As a Tullow Geophysicist I want examples of how to use ExSeisDat in C and C++ environments, as I want to use the library and may not have experience in the language
**PreDev Notes:**
Make sure examples have both C and C++ versions.
The files produced should be the same (use checksum to check them).
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
{- ADDME -}
CREATED ON - 09/01/2018March 2019 Roadmap2019-03-01https://git.ichec.ie/oilgas/ExSeisDat/-/issues/90Python API2019-01-07T15:40:54ZChristopher WernerPython APIS-01443
As a Tullow Geophysicist, I want to be able to call PIOL features from Python because I don't know or necessarily have experience with the language the PIOL is written in.
**Pre-Dev Notes:**
A number of operations are done in ...S-01443
As a Tullow Geophysicist, I want to be able to call PIOL features from Python because I don't know or necessarily have experience with the language the PIOL is written in.
**Pre-Dev Notes:**
A number of operations are done in ExSeisDat during class destruction. This doesn't play well with Python lifetime management. Implicit operations in the destructor should be made available explicitly, either as a method, or done at the start of other write functions when the file metadata has been changes on the object.
SWIG can be used to automatically generate bindings from C++ to a number of languages, i.e. Python.
The SWIG based wrapping needs to be tested. The `wraptests` should cover this.
The C and C++ API names have become out-of-sync. These need to be updated.
Ideally, some mechanism could be put in place to confirm the coverage and naming schemes of the C wrapper, but this could prove difficult.
Ensure all destructors of exposed classes can also be called explicitly. This is needed to match the memory model of Python. Currently, some classes perform actions such as writing to file, or executing a pipeline, during destruction. Since Python is garbage collected, when this action is completed is a lot less clear than it is in C++.
**Implementation Notes:**
{- ADDME -}
**System Test Changes:**
{- ADDME -}
**Bug Fixes:**
{- ADDME -}
**C++ API Changes:**
{- ADDME -}
**C API Changes:**
{- ADDME -}
**Success Criteria:**
* The full API is available through python calls
* The Python API is documented
* The user guide is updated to reflect python usage
* At least one python API example is created
**Tasks**
* [ ] Ensure all destructors of exposed classes can also be called explicitly
* [ ] Refactor: Re-sync the C and C++ APIs
* [ ] Add Python bindings
* [ ] Test Python bindings
CREATED ON - 20/04/2016March 2019 Roadmap2019-01-01https://git.ichec.ie/oilgas/ExSeisDat/-/issues/89Literature review on 5d Interpolation2018-11-06T13:53:24ZChristopher WernerLiterature review on 5d InterpolationS-01649
As an ICHEC computational scientist I want to see a short literature review of 5d interpolation to see what options are available and how practical to implement each method is.
There is a paper in the shared documents to perfor...S-01649
As an ICHEC computational scientist I want to see a short literature review of 5d interpolation to see what options are available and how practical to implement each method is.
There is a paper in the shared documents to perform the literature review.
Time box for 2 weeks.
CREATED ON - 06/06/2017