Commit 8301c8c7 authored by Sophie Wenzel-Teuber's avatar Sophie Wenzel-Teuber
Browse files

Build rpm packages

Additionally to building the packages in the docker container also create rpms

Changelog
* Add Postgres 9.4 rpms for Phobos
* Add spec files for all other dependencies
* Add new stage called 'package' to all build scripts to build rpm packages
* If packages are built, install them to the container instead of 'make install'
* Remove GFlags patch due to newer Proxygen version
parent a98e704a
# SuperDeimos
SuperDeimos is the superbuild project to [Deimos](https://git.ichec.ie/oilgas/ltfs/deimos).
This project contains all dependencies and scripts to build and install them on CentOS 7.
This project contains all dependencies and scripts to build and install them on CentOS 7 and 8 and/or create rpm files for them.
## Building
The main targets to be build are [Phobos](https://github.com/cea-hpc/phobos), [Proxygen](https://github.com/facebook/proxygen) and [Deimos](https://git.ichec.ie/oilgas/ltfs/deimos).
The main targets to be build are [Phobos](https://github.com/cea-hpc/phobos), [Proxygen](https://github.com/facebook/proxygen) and [Deimos](https://git.ichec.ie/performance/storage/deimos).
In the folder `src` there exist build scripts for each of these targets, while `src/deimos/build.sh` will build and install everything.
In the folder `src` there exist build scripts called `src/[target]/build.sh` for each of these targets.
The following options exist for all build scripts:
......@@ -16,8 +16,8 @@ The following options exist for all build scripts:
| `--dockerize` / `--no-dockerize` | no-dockerize | rerun the script in a docker container |
| `--image=` | centos:7 | image for the docker container if `dockerize` is set |
| `-j` | 1 | number of threads for `make`|
| `--skip=` / `--only=` | build all | Comma separated list of targets to skip / build (options are catch2, deimos, fizz, folly, fmt, gflags, googletest, openssl, phobos, proxygen, spdlog, wangle) |
| `--stages=` / `--skip-stages=` | execute all stages | Comma seperated list of stages to execute / skip (Stages are `build` and `test`) |
| `--skip=` / `--only=` | build all | Comma separated list of targets to skip / build (options are catch2, deimos, fizz, folly, fmt, gflags, googletest, phobos, proxygen, spdlog, wangle) |
| `--stages=` / `--skip-stages=` | execute all stages | Comma seperated list of stages to execute / skip (Stages are `build`, `test` and `package`) |
| `--verbose` / `--no-verbose` | verbose | Set verbosity of build script |
| `--prefix=` | `$PWD/build/install` | installation directory |
| `--namespaced-install` / `--no-namespaced-install` | namespaced install | create named directories in the install directory or install everything directly |
......@@ -25,7 +25,7 @@ The following options exist for all build scripts:
## Development
For development it is advised to build a docker image with this project and then mount a current Deimos directory in the container when executing the image.
For development it is advised to use the rpm files contained in Deimos itself to build it in a docker container. It is also possible to build a docker image with this project and then mount a current Deimos directory in the container when executing the image.
An example [Dockerfile](/Dockerfile) for this purpose is provided in this project.
An image can be built with the command
......@@ -53,7 +53,7 @@ export PYTHONPATH=/home/superdeimos/build/install/phobos/lib/python3.6/site-pack
```
when working in the container interactively. These can also be found in the file [envs.list](/envs.list) and added to the docker container using the [`--env-file`](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file) option.
when working in the container interactively. These can also be found in the file [centos7_envs.list](/centos7_envs.list) or [centos8_envs.list](/centos8_envs.list) and added to the docker container using the [`--env-file`](https://docs.docker.com/engine/reference/commandline/run/#set-environment-variables--e---env---env-file) option.
To setup a Phobos-Postgres database, the functions in the script [`src/phobos/testdb-setup.sh`](/src/phobos/testdb-setup.sh) should be used. An example of how to use that is given in the file [`start-db.sh`](/start-db.sh).
......
......@@ -33,8 +33,8 @@ BuildRequires: systemd
Requires: %{postgres_prefix}-server
Requires: %{postgres_prefix}-contrib
Requires: glib2 >= 2.28
Requires: jansson
Requires: glib2-devel >= 2.28
Requires: jansson-devel
Requires: libini_config
Requires: openssl
Requires: attr
......@@ -107,6 +107,7 @@ rm -f $RPM_BUILD_ROOT/%{_libdir}/phobos/libpho_layout_*.la
%files devel
%{_includedir}/pho_attrs.h
%{_includedir}/pho_types.h
%{_includedir}/phobos_store.h
%{_includedir}/phobos_admin.h
%{_libdir}/libphobos_store.la
......
......@@ -2242,7 +2242,7 @@ static int lazy_find_deprecated_object(struct dss_handle *hdl,
if (rc < 0)
LOG_RETURN(rc = -ENOMEM, "Cannot allocate filter string");
rc = dss_filter_build(&filter, json_filter);
rc = dss_filter_build(&filter, "%s", json_filter);
free(json_filter);
if (rc)
LOG_RETURN(rc, "Cannot build filter");
......@@ -2311,7 +2311,7 @@ int dss_lazy_find_object(struct dss_handle *hdl, const char *oid,
if (rc < 0)
LOG_RETURN(rc = -ENOMEM, "Cannot allocate filter string");
rc = dss_filter_build(&filter, json_filter);
rc = dss_filter_build(&filter, "%s", json_filter);
free(json_filter);
if (rc)
LOG_RETURN(rc, "Cannot build filter");
......
Name: Catch2
Version: 2.13.7
Release: 1%{?dist}
Summary: Catch2 is mainly a unit testing framework for C++, but it also provides basic micro-benchmarking features, and simple BDD macros.
License: Boost Software License
URL: https://github.com/catchorg/Catch2
Source0: https://github.com/catchorg/Catch2/archive/refs/tags/v%{version}.tar.gz
BuildRequires: make
BuildRequires: cmake3
BuildRequires: devtoolset-8-gcc-c++
%description
Catch2 is mainly a unit testing framework for C++, but it also provides basic
micro-benchmarking features, and simple BDD macros.
Catch2's main advantage is that using it is both simple and natural.
Tests autoregister themselves and do not have to be named with valid identifiers,
assertions look like normal C++ code, and sections provide a nice way to share
set-up and tear-down code in tests.
%prep
%setup -q
%build
mkdir build
cd build
cmake3 \
-DCATCH_BUILD_TESTING=OFF \
-DCMAKE_INSTALL_PREFIX:PATH=/usr \
..
%install
cd build
%make_install
%files
%{_includedir}/catch2/*
/usr/lib64/cmake/*
/usr/share/*
%changelog
* Mon Oct 18 2021 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- First Catch2 package
Name: deimos
Version: 1.0
Release: 1%{?dist}
Summary: Webserver to integrate S3 with Phobos
License: MIT
URL: https://git.ichec.ie/performance/storage/deimos
Source0: %{name}.tar.gz
%if 0%{?rhel} >= 8
%global cmake cmake
%global gcc gcc-c++
%else
%global cmake cmake3
%global gcc devtoolset-8-gcc,devtoolset-8-gcc-c++
%endif
BuildRequires: %{cmake}
BuildRequires: %{gcc}
BuildRequires: make
BuildRequires: phobos-devel
BuildRequires: folly
BuildRequires: fizz
BuildRequires: wangle
BuildRequires: proxygen
BuildRequires: spdlog-devel
BuildRequires: glib2-devel
BuildRequires: openssl-devel
BuildRequires: openssl-libs
%if 0%{?rhel} < 8
BuildRequires: libevent-devel
BuildRequires: zlib-devel
BuildRequires: boost169-devel
BuildRequires: jansson-devel
BuildRequires: double-conversion-devel
BuildRequires: glog-devel
BuildRequires: gflags-devel
BuildRequires: libsodium-devel
BuildRequires: libzstd-devel
%endif
Requires: phobos-devel
Requires: folly
Requires: fizz
Requires: wangle
Requires: proxygen
Requires: spdlog-devel
Requires: glib2
Requires: openssl-libs
%if 0%{?rhel} < 8
Requires: glog
Requires: gflags
%endif
%description
Deimos is a webserver to accept S3 REST API requests for object storage and passes the contents to storage using the Phobos library.
%global debug_package %{nil}
%prep
echo "Start Prep"
%setup -q -n %{name}
%build
echo "Start Build"
%if 0%{?rhel} >= 8
%global build_flags -DCMAKE_INSTALL_PREFIX:PATH=/usr
%else
declare -a OPENSSL_CMAKE_FLAGS
OPENSSL_CMAKE_FLAGS+=(
-DOPENSSL_CRYPTO_LIBRARY=/usr/lib64/libcrypto.so
-DOPENSSL_SSL_LIBRARY=/usr/lib64/libssl.so
-DOPENSSL_INCLUDE_DIR=/usr/include/
)
declare -a BOOST_CMAKE_FLAGS
BOOST_CMAKE_FLAGS+=(
-DBOOST_ROOT=/usr/include/boost169
-DBOOST_INCLUDEDIR=/usr/include/boost169
-DBOOST_LIBRARYDIR=/usr/lib64/boost169
)
%global build_flags "${BOOST_CMAKE_FLAGS[@]}" "${OPENSSL_CMAKE_FLAGS[@]}" -DCMAKE_INSTALL_PREFIX:PATH=/usr
%endif
%{cmake} %{build_flags} .
make %{?_smp_mflags}
%install
%make_install
%files
%license LICENSE
/usr/bin/deimos
%changelog
* Mon Oct 11 2021 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v1.0
- Combine for CentOS7 and CentOS8
* Mon Aug 31 2020 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v1.0
- First deimos package
\ No newline at end of file
Name: fizz
Version: 2021.10.11.00
Release: 1%{?dist}
Summary: Fizz is a TLS 1.3 implementation.
License: BSD License
URL: https://github.com/facebookincubator/fizz
Source0: https://github.com/facebookincubator/%{name}/archive/v%{version}.tar.gz
%if 0%{?rhel} >= 8
%global cmake cmake
%global gcc gcc-c++
%else
%global cmake cmake3
%global gcc devtoolset-8-gcc,devtoolset-8-gcc-c++
%endif
BuildRequires: make
BuildRequires: %{cmake}
BuildRequires: %{gcc}
BuildRequires: folly
Requires: folly
%description
Fizz is a TLS 1.3 implementation.
Fizz currently supports TLS 1.3 drafts 28, 26 (both wire-compatible with the final specification), and 23. All major handshake modes are supported, including PSK resumption, early data, client authentication, and HelloRetryRequest.
More background and details are available on the Facebook Code Blog.
%global debug_package %{nil}
%prep
%setup -q
%build
%if 0%{?rhel} >= 8
%global build_flags -DBUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX:PATH=/usr
%else
declare -a OPENSSL_CMAKE_FLAGS
OPENSSL_CMAKE_FLAGS+=(
-DOPENSSL_CRYPTO_LIBRARY=/usr/lib64/libcrypto.so
-DOPENSSL_SSL_LIBRARY=/usr/lib64/libssl.so
-DOPENSSL_INCLUDE_DIR=/usr/include/
)
declare -a BOOST_CMAKE_FLAGS
BOOST_CMAKE_FLAGS+=(
-DBOOST_ROOT=/usr/include/boost169
-DBOOST_INCLUDEDIR=/usr/include/boost169
-DBOOST_LIBRARYDIR=/usr/lib64/boost169
)
%global build_flags -DBUILD_EXAMPLES=OFF -DBUILD_TESTS=OFF "${BOOST_CMAKE_FLAGS[@]}" "${OPENSSL_CMAKE_FLAGS[@]}" -DCMAKE_INSTALL_PREFIX:PATH=/usr
%endif
%{cmake} %{build_flags} ./%{name}/
make %{?_smp_mflags}
%install
%make_install
%files
%license LICENSE
/usr/lib/cmake/*
/usr/lib/lib%{name}*
%{_includedir}/%{name}/*
%if 0%{?rhel} >= 8
%{_bindir}/%{name}*
%endif
%changelog
* Mon Oct 11 2021 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- Combine for CentOS7 and CentOS8
* Mon Aug 31 2020 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- First fizz package
Name: fmt-devel
Version: 7.0.3
Release: 1%{?dist}
Summary: {fmt} is an open-source formatting library for C++. It can be used as a safe and fast alternative to (s)printf and iostreams.
License: custom
URL: https://github.com/fmtlib/fmt
Source0: https://github.com/fmtlib/fmt/archive/%{version}.tar.gz
BuildRequires: cmake3,devtoolset-8-gcc,devtoolset-8-gcc-c++
%description
{fmt} is an open-source formatting library for C++. It can be used as a safe and fast alternative to (s)printf and iostreams.
%global debug_package %{nil}
%prep
%setup -q -n fmt-%{version}
%build
echo "Start Build"
cmake3 -DFMT_DOC=OFF \
-DFMT_TEST=OFF \
-DCMAKE_INSTALL_PREFIX:PATH=/usr \
.
make %{?_smp_mflags}
%install
%make_install
%files
%license LICENSE.rst
/usr/lib64/*
/usr/include/*
%changelog
* Thu Sep 03 2020 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - 7.0.3
- First fmt package
Name: folly
Version: 2021.10.11.00
Release: 1%{?dist}
Summary: Folly is a library of C++14 components designed with practicality and efficiency in mind.
License: Apache License 2.0
URL: https://github.com/facebook/folly
Source0: https://github.com/facebook/%{name}/archive/v%{version}.tar.gz
%if 0%{?rhel} >= 8
%global cmake cmake
%global gcc gcc-c++
%global boost boost-devel
%else
%global cmake cmake3
%global gcc devtoolset-8-gcc,devtoolset-8-gcc-c++
%global boost boost169-devel
%endif
BuildRequires: make
BuildRequires: %{gcc}
BuildRequires: %{cmake}
BuildRequires: gperf
BuildRequires: perl
BuildRequires: perl-core
BuildRequires: libevent-devel
BuildRequires: zlib-devel
BuildRequires: fmt-devel
BuildRequires: openssl-devel
BuildRequires: openssl-libs
BuildRequires: %{boost}
BuildRequires: double-conversion-devel
BuildRequires: gflags
BuildRequires: gflags-devel
BuildRequires: glog-devel
BuildRequires: libsodium-devel
BuildRequires: libzstd-devel
Requires: gperf
Requires: perl
Requires: perl-core
Requires: libevent-devel
Requires: zlib-devel
Requires: fmt-devel
Requires: openssl-devel
Requires: openssl-libs
Requires: %{boost}
Requires: double-conversion-devel
Requires: gflags
Requires: gflags-devel
Requires: glog-devel
Requires: libsodium-devel
Requires: libzstd-devel
%description
Folly (acronymed loosely after Facebook Open Source Library) is a library of C++14 components designed with practicality and efficiency in mind. Folly contains a variety of core library components used extensively at Facebook. In particular, it's often a dependency of Facebook's other open source C++ efforts and place where those projects can share code.
It complements (as opposed to competing against) offerings such as Boost and of course std. In fact, we embark on defining our own component only when something we need is either not available, or does not meet the needed performance profile. We endeavor to remove things from folly if or when std or Boost obsoletes them.
Performance concerns permeate much of Folly, sometimes leading to designs that are more idiosyncratic than they would otherwise be (see e.g. PackedSyncPtr.h, SmallLocks.h). Good performance at large scale is a unifying theme in all of Folly.
%global debug_package %{nil}
%prep
%setup -q
%build
%if 0%{?rhel} >= 8
%global build_flags -DBUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX:PATH=/usr
%else
declare -a OPENSSL_CMAKE_FLAGS
OPENSSL_CMAKE_FLAGS+=(
-DOPENSSL_CRYPTO_LIBRARY=/usr/lib64/libcrypto.so
-DOPENSSL_SSL_LIBRARY=/usr/lib64/libssl.so
-DOPENSSL_INCLUDE_DIR=/usr/include/
)
declare -a BOOST_CMAKE_FLAGS
BOOST_CMAKE_FLAGS+=(
-DBOOST_ROOT=/usr/include/boost169
-DBOOST_INCLUDEDIR=/usr/include/boost169
-DBOOST_LIBRARYDIR=/usr/lib64/boost169
)
%global build_flags -DBUILD_TESTS=OFF "${BOOST_CMAKE_FLAGS[@]}" "${OPENSSL_CMAKE_FLAGS[@]}" -DCMAKE_INSTALL_PREFIX:PATH=/usr
%endif
%{cmake} %{build_flags} .
make %{?_smp_mflags}
%install
%make_install
%files
%license LICENSE
/usr/lib/cmake/*
/usr/lib/pkgconfig/*
/usr/lib/lib%{name}*
%{_includedir}/%{name}/*
%changelog
* Mon Oct 11 2021 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- Combine for CentOS7 and CentOS8
* Mon Aug 31 2020 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- First folly package
Name: proxygen
Version: 2021.10.11.00
Release: 1%{?dist}
Summary: Proxygen comprises the core C++ HTTP abstractions used at Facebook.
License: BSD License
URL: https://github.com/facebook/proxygen
Source0: https://github.com/facebook/%{name}/archive/v%{version}.tar.gz
%if 0%{?rhel} >= 8
%global cmake cmake
%global gcc gcc-c++
%else
%global cmake cmake3
%global gcc devtoolset-8-gcc,devtoolset-8-gcc-c++
%endif
BuildRequires: make
BuildRequires: %{cmake}
BuildRequires: %{gcc}
BuildRequires: folly
BuildRequires: fizz
BuildRequires: wangle
BuildRequires: python36
Requires: folly
Requires: fizz
Requires: wangle
%description
This project comprises the core C++ HTTP abstractions used at Facebook. Internally, it is used as the basis for building many HTTP servers, proxies, and clients. This release focuses on the common HTTP abstractions and our simple HTTPServer framework. Future releases will provide simple client APIs as well. The framework supports HTTP/1.1, SPDY/3, SPDY/3.1, HTTP/2, and HTTP/3. The goal is to provide a simple, performant, and modern C++ HTTP library.
%global debug_package %{nil}
%prep
%setup -q
%build
%if 0%{?rhel} >= 8
%global build_flags -DBUILD_TESTS=OFF -DCMAKE_INSTALL_PREFIX:PATH=/usr
%else
declare -a OPENSSL_CMAKE_FLAGS
OPENSSL_CMAKE_FLAGS+=(
-DOPENSSL_CRYPTO_LIBRARY=/usr/lib64/libcrypto.so
-DOPENSSL_SSL_LIBRARY=/usr/lib64/libssl.so
-DOPENSSL_INCLUDE_DIR=/usr/include/
)
declare -a BOOST_CMAKE_FLAGS
BOOST_CMAKE_FLAGS+=(
-DBOOST_ROOT=/usr/include/boost169
-DBOOST_INCLUDEDIR=/usr/include/boost169
-DBOOST_LIBRARYDIR=/usr/lib64/boost169
)
%global build_flags -DBUILD_TESTS=OFF "${BOOST_CMAKE_FLAGS[@]}" "${OPENSSL_CMAKE_FLAGS[@]}" -DCMAKE_INSTALL_PREFIX:PATH=/usr
%endif
# Remove uneccessary dependency from cmake config file
sed -i '/find_dependency(mvfst)/d' ./cmake/proxygen-config.cmake.in
%{cmake} %{build_flags} .
make %{?_smp_mflags}
%install
%make_install
%files
%license LICENSE
/usr/lib/cmake/*
/usr/lib/lib%{name}*
%{_includedir}/%{name}/*
%{_bindir}/%{name}_*
%changelog
* Mon Oct 11 2021 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- Combine for CentOS7 and CentOS8
* Mon Aug 31 2020 Sophie Wenzel-Teuber <sophie.wenzel-teuber@ichec.ie> - v2020.08.17.00
- First proxygen package
Name: spdlog
Version: 1.8.0
Release: 1%{?dist}
Summary: Very fast, header-only/compiled, C++ logging library.
License: MIT
URL: https://github.com/gabime/spdlog
Source0: https://github.com/gabime/%{name}/archive/v%{version}.tar.gz
%if 0%{?rhel} >= 8
%global cmake cmake
%global gcc gcc-c++
%else
%global cmake cmake3
%global gcc devtoolset-8-gcc,devtoolset-8-gcc-c++
%endif
BuildRequires: make
BuildRequires: %{cmake}
BuildRequires: %{gcc}
%description
Very fast, header-only/compiled, C++ logging library.