Skip to content
GitLab
Commit 6acd5684 authored by Ciarán Ó Rourke's avatar Ciarán Ó Rourke
Browse files

Multiple OS support 

Build scripts for Phobos, Proxygen, and Deimos work on CentOS7 and
CentOS8.

Change log:
* generalise build scripts for different OSs
* modularise CentOS7 specific building
* update Phobos to current master
* functions for building on CentOS8
* reflect new Phobos version
* link to more boost libraries for CentOS7 Proxygen build
parent dd5d0ffc
Pipeline #2121 skipped with stage
Expand all Hide whitespace changes
Inline Side-by-side
external/phobos/ChangeLog deleted 100644 → 0
View file @ dd5d0ffc
Detailed list of changes since phobos 1.0:
----------
scripts: phobos 1.0 to 1.1 DB conversion
Adds a simple script 'pho_convert_db_10to11.sh' to convert DB schema
from phobos 1.0 to phobos 1.1.
----------
doc/config: add store's 'layout' parameter to config template
Add store's 'layout' parameter to config template.
This parameter determines which layout must be used for put operations.
----------
layout: implement replication module
Add raid1 module that replicates an object N times across N media.
Rework test_store slightly so that the test suite can invoke it repeatedly using
different layouts.
Phobos-Issue: PHO-58
----------
Fix dead code in test_store.sh
The GET tests were not executed because the extents were deleted before the
test had any chance to browse and check them.
----------
common: implement pho_ht_foreach
new iterator over entries of a GHashTable that can be interrupted by
a callback returning a non-zero value. Such a value is propagated back
to the caller.
----------
common: add comments to the SAJ module
Phobos-Issue: PHO-135
----------
ldm: fix memory leak
unconditionally release result of realpath() and manually
assign errno in case a function fails w/o setting it.
----------
Retrieve fs label at format and check it at mount
Make sure that a filesystem (especially LTFS) that was mounted is the expected
one. Achieve this by (optionally) comparing the FS label at mount time against
the one stored in DSS, retrieved after formatting the medium.
Phobos-Issue: PHO-57
----------
Introduce FS label field into media_info
Introduce a new fs_label field into DSS media description.
Group filesystem related informations into a struct media_fs.
This value (null-terminated, possibly empty) string will be used to
compare the expected value to what is actually mounted to ensure system
safety.
Phobos-Issue: PHO-57
----------
Introduce .fs_get_label in LDM fs operations
Allow LDM callers to retrieve the label of a mounted filesystem.
Use the ltfs.volumeName xattr facility for LTFS and a hidden file for
POSIX.
Phobos-Issue: PHO-57
----------
doc: fix file mode
Remove executable bit that was set by mistake
----------
ldm: fix typo in comment
----------
scripts: indentation fixes in pho_*_helper
Replaced all tabs by spaces for consistency.
----------
dss: prevent printf from receiving null params
----------
layout: introduce new layout management layer
Layout modules are autonomous implementations of a given layout type.
They abstract actual object layouts by translating data xfer requests
into corresponding storage resources and manipulation.
Although the layout code is interleaved with the store state machine, layout
can be seen as an interface between store and {LRS, IO}.
Phobos-Issue: PHO-121
----------
lrs: tag unwritable media as full
LTFS has been observed to mount almost-full tapes as read-only. When
this happens, tag the media as full and retry. Update LRS device
selection logic accordingly.
One problem we have here is that there is no easy way to identify
R/O-FS before linux 2.6.36 (ie. before RHEL7). Therefore, only rhel7
will benefit from early detection. Older platforms will have to go
through mput and watch it fail.
If at least one slice of an mput fails with impossibility to further
write on the media we mark the media as full. Note that no retry is
attempted at this step.
Phobos-Issue: PHO-129
----------
lib scsi: search a free slot to unload drive if none is specified
In some cases (e.g. after library power reset) the library doesn't provide
a source slot number for tapes loaded in drives.
In this case, we must search for an empty slot where the tape can be moved.
Completed test_scsi to check this case.
Also fixed it to properly handle drive serials.
----------
tests: add status check in test_scsi
Check that the drive or slot status after a move_medium operation
is the expected one.
Also handle the case when no tape is seen in the library (neither
in slots nor in drives).
----------
lrs: added extra information to log message.
Add the available size on a media to a verbose level log message
so we can better keep track of space-related issues.
----------
lrs: release device on write-prepare failure
A device was allocated by the get_write_res() function that must
be released on error.
----------
ldm: used signed types to represent media space
Both are big enough to not overflow but unsigned types can easily
lead to miscalculation of available space if a difference returns
a "negative" result.
This has been observed in the 5% margin calculation, in
ldm/ldm_fs_ltfs.c
Phobos-Issue: PHO-129
----------
doc: update layout design
Reduce exposed APIs and get rid of GLI.
- libpho_layout: init/declare/acquire/fini
- libpho_layout_mod_xxx: {de}register
- layout implementation: compose/io/commit
----------
dss: replace layout_type by a module description
Introduce module description (name, major, minor, attrs) and store it into DSS
as layout information instead of an empty array. Add the appropriate
encoding/decoding functions.
Remove layout_type (and copy_num) which are no longer needed.
Note that this commit changes the DSS DB schema w/o providing a conversion tool.
This will come before the next release in order to provide compatibility and
ease upgrade of existing instances.
----------
dss: rework json parsing helpers
Use consistent return values and signed types. Add a couple macro to simplify
the code.
----------
dss: use signed integer to represent media stats and extent.
Unsigned integers are cumbersome and useless here.
----------
cli: Fix inheritance in DSS object managers
phobos::dss:MediaManager and phobos::dss:DeviceManager classes operate
on wrapped objects, although their parent class does not and manipulates
raw SWIG structures.
Add the missing ::delete methods appropriately so as to invoke the
underlying function with the expected argument.
----------
doc: add a template configuration.
Add a template configuration with comments.
Included to the RPM, it is initially installed
as "/etc/phobos.conf" (if it does not already exists).
Else it will be installed as /etc/phobos.conf.rpmnew
----------
doc: Update developer README.
PKG_CONFIG_PATH must be set with the right postgres version.
----------
cfg: each module defines its own configuration.
Configuration variables and default values were previously
defined in single centralized header. These values are now
defined independantly by each module. This prepares the
support of dynamically loaded modules so they can define their
own parameters.
----------
cli: Do not emit empty filters
Prevent the python dss filter converter from emitting empty filters and optimize
these that only contain a single element.
----------
scripts: fix typo in comment
----------
minor release 1.0.1
----------
ltfs: reduce available media space by 5%.
LTFS keeps some reserved space to flush its index but do not
reflect this in statfs().
The amount of reserved space is not clearly documented, but
it appears to be around 5% in some real cases.
So, reduce the free space returned by statfs() on LFTS by 5%.
This should address one of the aspects of RM#129.
----------
lrs: mark media full if there is no space free.
----------
cli: fix minor typo
----------
doc: layout composition layer
High-level description of the layout composition layer.
This will be responsible for managing evoluted layouts (replication,
erasure coding...) and translating store requests into proper
sub-requests to the lower layers.
----------
lrs: mark R/O-mounted media as full
Mark media as full if operations happen to have failed with -EROFS.
This can typically occur when LTFS decides that a tape does not contain
enough space to be mounted writable.
----------
dss: make dss_unlock return ENOLCK when a lock is not owned.
When releasing a lock that is not owned by the caller, the resulting
error code was EEXIST, which was somehow confusing.
Replace this error by ENOLCK (No lock available).
----------
dss: make dss_get resilient to missing fields in media stats.
Commit 4c2b7801 added new fields to media_stats (JSON stored in the DB).
When running dss_get on a previous phobos DB, these fields are missing.
Don't break the whole dss_get in that case. Actually make all
media_stats non-critical. Simply return 0 for the missing ones.
external/phobos/Makefile.am
View file @ 6acd5684
......@@ -9,7 +9,7 @@ rpm: dist phobos.spec
rpmbuild --define="_topdir $(rpm_dir)" -ta $(distdir).tar.gz
EXTRA_DIST= phobos.spec phobos.spec.in scripts/pho_ldm_helper \
scripts/phobos_db doc/README.md doc/cfg/template.conf ChangeLog
scripts/phobos_db README.md doc/cfg/template.conf
sbin_SCRIPTS=scripts/pho_ldm_helper \
scripts/phobos_db
external/phobos/README.developer
View file @ 6acd5684
......@@ -4,5 +4,5 @@
* To setup autotools stuff the first time you check out the repo, run:
./autogen.sh
* Add PGSQL 9.4 to the build environment
* (Before RHEL8/CentOS8) Add PGSQL 9.4 to the build environment
export PKG_CONFIG_PATH=/usr/pgsql-9.4/lib/pkgconfig
external/phobos/README.md deleted 120000 → 0
View file @ dd5d0ffc
doc/README.md
\ No newline at end of file
external/phobos/README.md 0 → 100644
View file @ 6acd5684
# Phobos documentation
## Installation
### Requirements for tape access
You need to install LTFS >= 2.4 to enable phobos access to tapes.
LTFS RPM can be found on IBM Fix Central: [ltfs 2.4](https://www-945.ibm.com/support/fixcentral/swg/selectFixes?parent=Tape%20drivers%20and%20software&product=ibm/Storage_Tape/Long+Term+File+System+LTFS&release=2.4&platform=Linux&function=all)
You can also retrieve its sources on gihub:
https://github.com/LinearTapeFileSystem/ltfs
If you want to build RPMs from these sources, you can find packaging resources
(i.e. spec file) for LTFS here: https://github.com/piste2750/rpm-ltfs
Note: since LTFS 2.4, `lin_tape` driver is no longer needed to access tapes.
LTFS now uses the standard linux tape driver (st).
### Phobos installation
Install phobos and its requirements:
```
yum install phobos
```
### Database setup
#### On RHEL8/CentOS8
Install postgresql:
```
dnf install postgresql-server postgresql-contrib
```
Initialize postgresql directories:
```
postgresql-setup --initdb --unit postgresql
```
Edit `/var/lib/pgsql/data/pg_hba.conf` to authorize access from phobos host
(localhost in this example):
```
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
```
Start the database server:
```
systemctl start postgresql
```
Finally, create phobos database and tables as postgres user (the password for
SQL phobos user will be prompted for unless provided with -p):
```
sudo -u postgres phobos_db setup_db -s
```
#### On RHEL7/CentOS7
Install postgresql version >= 9.4 (from EPEL or any version compatible with
Postgres 9.4):
```
yum install postgresql94-server postgresql94-contrib
```
Initialize postgresql directories:
```
/usr/pgsql-9.4/bin/postgresql94-setup initdb
```
Edit `/var/lib/pgsql/9.4/data/pg_hba.conf` to authorize access from phobos host
(localhost in this example):
```
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all 127.0.0.1/32 md5
# IPv6 local connections:
host all all ::1/128 md5
```
Start the database server:
```
systemctl start postgresql-9.4.service
```
Finally, create phobos database and tables as postgres user (the password for
SQL phobos user will be prompted for unless provided with -p):
```
sudo -u postgres phobos_db setup_db -s
```
## Upgrade of an existing instance
After upgrading phobos, run the DB conversion script (credentials to connect to
the database will be retrieved from /etc/phobos.conf):
```
/usr/sbin/phobos_db migrate
# 'y' to confirm
```
## First steps
### Launching phobosd
To use phobos, a daemon called phobosd needs to be launched. It mainly manages
the drive/media allocation and sets up resources to read or write your data.
To launch the daemon:
```
# systemctl start phobosd
```
To stop the daemon:
```
# systemctl stop phobosd
```
phobosd comes with its option set:
- -i: launch phobosd in an interactive way (attached to a terminal)
- -c: path to phobosd configuration file
- -s: print log messages into syslog
- -q/v: decrease/increase log verbosity
### Adding drives
Use the `phobos` command line to add new tape drives to be managed by phobos.
It is recommanded to specify a device path which is persistent after reboot
(eg. managed by dev mapper).
By default, drives are added in a locked state, so they are not immediatly used
in production.
To enable using a drive in production as soon as it is added, specify the
'--unlock' option:
```
phobos drive add --unlock /dev/mapper/LTO6-012345
```
### Adding media
To add tapes to be managed by phobos, use the `phobos` command line.
It is mandatory to specify a media type (like LTO8, T10KD...) with option `-t`:
```
phobos tape add -t lto6 [073200-073222]L6
```
**Note1:** this operation requires no access to the medium.
**Note2:** the set of supported media types, drive types, and compatibility rules
between drives and media are configurable. See section '*Configuring device
and media types*' below for more details.
Once a media has been added, it needs to be formatted. You can optionally unlock
the tape when the operation is done, so it becomes usable in production.
```
phobos tape format --unlock 073200L6
```
### Writing data
#### Using the API
You can access data in phobos using the phobos API:
* Header file is `phobos_store.h`.
* Library is `libphobos_store` (link using `-lphobos_store`).
Calls for writing and retrieving objects are `phobos_put()` and
`phobos_get()`.
#### Using the CLI
The CLI wraps `phobos_put()` and `phobos_get()` calls.
##### Writting objects
When pushing an object you need to specify an **identifier** that can be later
used to retrieve this data.
Phobos also allows specifying an arbitrary set of attributes (key-values) to be
attached to the object. They are specified when putting the object using the
'-m' option.
```
# phobos put <file> <id> -m <k=v,k=v,...>
phobos put myinput_file foo-12345 -m user=$LOGNAME,\
put_time=$(date +%s),version=1
```
For better performances when writing to tape, it is recommanded to write batches
of objects in a single put command. This can be done using `phobos mput`
command.
`phobos mput` takes as agument a file that contains a list of input files (one
per line) and the corresponding identifiers and metadata, with the following format:
`<src_file> <object_id> <metadata|->`.
Example of input file for mput:
```
/srcdir/file.1 obj0123 -
/srcdir/file.2 obj0124 user=foo,md5=xxxxxx
/srcdir/file.3 obj0125 project=29DKPOKD,date=123xyz
```
The mput operation is simply:
```
phobos mput list_file
```
##### Reading objects
To retrieve the data of an object, use `phobos get`. Its arguments are the
identifier of the object to be retrieved, as well as a path of target file.
For example:
```
phobos get obj0123 /tmp/obj0123.back
```
##### Reading object attributes
To retrieve custom object metadata, use `phobos getmd`:
```
$ phobos getmd obj0123
cksum=md5:7c28aec5441644094064fcf651ab5e3e,user=foo
```
##### Listing objects
To list objects, use `phobos object list`:
```
$ phobos object list -o oid,user_md
| oid | user_md |
|-------|-----------------|
| obj01 | {} |
| obj02 | {"user": "foo"} |
```
You can add a pattern to match object oids:
```
phobos object list "obj.*"
```
The accepted patterns are Basic Regular Expressions (BRE) and Extended
Regular Expressions (ERE). As defined in [PostgreSQL manual](https://www.postgresql.org/docs/9.3/functions-matching.html#POSIX-SYNTAX-DETAILS),
PSQL also accepts Advanced Regular Expressions (ARE), but we will not maintain
this feature as ARE is not a POSIX standard.
The option '-m' applies a filter on user metadata
```
phobos object list -m user=foo,test=abd
```
## Device and media management
### Listing resources
Any device or media can be listed using the 'list' operation. For instance,
the following will list all the existing tape identifiers:
```
phobos tape list
```
The 'list' output can be formatted using the -o option, following a list of
comma-separated attributes:
```
phobos tape list -o stats.phys_spc_used,tags,adm_status
```
If the attribute 'all' is used, then the list is printed with full details:
```
phobos tape list -o all
```
Other options are available using the -h option.
### Locking resources
A device or media can be locked. In this case it cannot be used for