Commit d073d06c authored by Ciarán Ó Rourke's avatar Ciarán Ó Rourke
Browse files

Squashed 'external/phobos/' changes from a327ed60..087c71a3

087c71a3 doc: Update for RHEL8/CentOS8
70a94883 Fix warnings reported by gcc8: 'duplicate const declaration specifier'
5c927409 cli: 'list -o' prints following user order
b6354c77 Fix gcc8 warning 'suggest parentheses around assignment used as truth value'
742f9d72 Explicit use of python2
4522ed5b Remove hardcoded 'python' from Makefile
dbe0e5f9 Replace ENOATTR with its actual value ENODATA
3c4430ab Fix warnings reported by gcc8: 'output may be truncated'
04ebe7de checkpatch: allow using long URLs in commits after 'See:' keyword
dbef6a86 update m4 macros to manage python with autoconf
557d479c remove ChangeLog file
3b3703a0 clarify phobos_db command usage
55d06a12 compliance with the mobile version of github
9a5ea483 refactor: aggregate XFer PUT parameters in a new structure
c73783a7 refactor: remove DeviceOptHandler inheritance of DirOptHandler
bed22631 cli: remove format command from drive and device
40adb7e0 cli: add a '-l/--layout' option for put/mput commands
37ccdb7a cli: remove printing if getmd request returns no result
8271ed67 tests: requests processing depending on resource availability
6abf3779 ldm_dev_scsi: Replace GSList with simple linked list implementation
b5723a4e cli: add a '-f/--family' option for put/mput commands
12cd08e1 checkpatch: recognize Python types
50f7f581 ci: chmod o+rx on all src to allow use by postgres user
74cb2535 ldm_dev_scsi_tape: use 'sg' device path for LTFS
3523f520 scheduler: fix medium selection if locked but in a device
e8f402f0 phobosd: allow the utilization of multiple schedulers
d8cc4ede tests: reorder database setup and daemon invocation
79408e0e make: fix 'warning: jobserver [...] Add + to parent make rule
52fea1e8 serializer: correct req to resp in doxygen blocks
bec4b839 scheduler: fix sched_init() behavior in case of error
c0b94f66 refactor: unify resource structures of protobuf headers
342ad2e5 refactor: unify medium and device admin statuses
3611fa38 refactor: change cli cenum field to a family enum class
8befe889 refactor: unify medium and device identifiers
2910adda configure: update contact e-mail
141f4995 doc: specify default list output
23c8f816 tests: add tests for phobosd bad requests reception
63c03506 tests: wait phobosd termination before continuing tests
870af64a phobosd: do not quit if error during request processing
01e651ee fix: correct a check from EAGAIN to -EAGAIN
5a4df77e tests: add a cleanup trap to
dcf81acb phobosd: add a -h/--help option to the parser
9b23e5ee fix: segfault when using phobosd with a wrong option
989076cc cli: merge list and show commands

git-subtree-dir: external/phobos
git-subtree-split: 087c71a3
parent 8a9d5dca
Detailed list of changes since phobos 1.0:
scripts: phobos 1.0 to 1.1 DB conversion
Adds a simple script '' to convert DB schema
from phobos 1.0 to phobos 1.1.
doc/config: add store's 'layout' parameter to config template
Add store's 'layout' parameter to config template.
This parameter determines which layout must be used for put operations.
layout: implement replication module
Add raid1 module that replicates an object N times across N media.
Rework test_store slightly so that the test suite can invoke it repeatedly using
different layouts.
Phobos-Issue: PHO-58
Fix dead code in
The GET tests were not executed because the extents were deleted before the
test had any chance to browse and check them.
common: implement pho_ht_foreach
new iterator over entries of a GHashTable that can be interrupted by
a callback returning a non-zero value. Such a value is propagated back
to the caller.
common: add comments to the SAJ module
Phobos-Issue: PHO-135
ldm: fix memory leak
unconditionally release result of realpath() and manually
assign errno in case a function fails w/o setting it.
Retrieve fs label at format and check it at mount
Make sure that a filesystem (especially LTFS) that was mounted is the expected
one. Achieve this by (optionally) comparing the FS label at mount time against
the one stored in DSS, retrieved after formatting the medium.
Phobos-Issue: PHO-57
Introduce FS label field into media_info
Introduce a new fs_label field into DSS media description.
Group filesystem related informations into a struct media_fs.
This value (null-terminated, possibly empty) string will be used to
compare the expected value to what is actually mounted to ensure system
Phobos-Issue: PHO-57
Introduce .fs_get_label in LDM fs operations
Allow LDM callers to retrieve the label of a mounted filesystem.
Use the ltfs.volumeName xattr facility for LTFS and a hidden file for
Phobos-Issue: PHO-57
doc: fix file mode
Remove executable bit that was set by mistake
ldm: fix typo in comment
scripts: indentation fixes in pho_*_helper
Replaced all tabs by spaces for consistency.
dss: prevent printf from receiving null params
layout: introduce new layout management layer
Layout modules are autonomous implementations of a given layout type.
They abstract actual object layouts by translating data xfer requests
into corresponding storage resources and manipulation.
Although the layout code is interleaved with the store state machine, layout
can be seen as an interface between store and {LRS, IO}.
Phobos-Issue: PHO-121
lrs: tag unwritable media as full
LTFS has been observed to mount almost-full tapes as read-only. When
this happens, tag the media as full and retry. Update LRS device
selection logic accordingly.
One problem we have here is that there is no easy way to identify
R/O-FS before linux 2.6.36 (ie. before RHEL7). Therefore, only rhel7
will benefit from early detection. Older platforms will have to go
through mput and watch it fail.
If at least one slice of an mput fails with impossibility to further
write on the media we mark the media as full. Note that no retry is
attempted at this step.
Phobos-Issue: PHO-129
lib scsi: search a free slot to unload drive if none is specified
In some cases (e.g. after library power reset) the library doesn't provide
a source slot number for tapes loaded in drives.
In this case, we must search for an empty slot where the tape can be moved.
Completed test_scsi to check this case.
Also fixed it to properly handle drive serials.
tests: add status check in test_scsi
Check that the drive or slot status after a move_medium operation
is the expected one.
Also handle the case when no tape is seen in the library (neither
in slots nor in drives).
lrs: added extra information to log message.
Add the available size on a media to a verbose level log message
so we can better keep track of space-related issues.
lrs: release device on write-prepare failure
A device was allocated by the get_write_res() function that must
be released on error.
ldm: used signed types to represent media space
Both are big enough to not overflow but unsigned types can easily
lead to miscalculation of available space if a difference returns
a "negative" result.
This has been observed in the 5% margin calculation, in
Phobos-Issue: PHO-129
doc: update layout design
Reduce exposed APIs and get rid of GLI.
- libpho_layout: init/declare/acquire/fini
- libpho_layout_mod_xxx: {de}register
- layout implementation: compose/io/commit
dss: replace layout_type by a module description
Introduce module description (name, major, minor, attrs) and store it into DSS
as layout information instead of an empty array. Add the appropriate
encoding/decoding functions.
Remove layout_type (and copy_num) which are no longer needed.
Note that this commit changes the DSS DB schema w/o providing a conversion tool.
This will come before the next release in order to provide compatibility and
ease upgrade of existing instances.
dss: rework json parsing helpers
Use consistent return values and signed types. Add a couple macro to simplify
the code.
dss: use signed integer to represent media stats and extent.
Unsigned integers are cumbersome and useless here.
cli: Fix inheritance in DSS object managers
phobos::dss:MediaManager and phobos::dss:DeviceManager classes operate
on wrapped objects, although their parent class does not and manipulates
raw SWIG structures.
Add the missing ::delete methods appropriately so as to invoke the
underlying function with the expected argument.
doc: add a template configuration.
Add a template configuration with comments.
Included to the RPM, it is initially installed
as "/etc/phobos.conf" (if it does not already exists).
Else it will be installed as /etc/phobos.conf.rpmnew
doc: Update developer README.
PKG_CONFIG_PATH must be set with the right postgres version.
cfg: each module defines its own configuration.
Configuration variables and default values were previously
defined in single centralized header. These values are now
defined independantly by each module. This prepares the
support of dynamically loaded modules so they can define their
own parameters.
cli: Do not emit empty filters
Prevent the python dss filter converter from emitting empty filters and optimize
these that only contain a single element.
scripts: fix typo in comment
minor release 1.0.1
ltfs: reduce available media space by 5%.
LTFS keeps some reserved space to flush its index but do not
reflect this in statfs().
The amount of reserved space is not clearly documented, but
it appears to be around 5% in some real cases.
So, reduce the free space returned by statfs() on LFTS by 5%.
This should address one of the aspects of RM#129.
lrs: mark media full if there is no space free.
cli: fix minor typo
doc: layout composition layer
High-level description of the layout composition layer.
This will be responsible for managing evoluted layouts (replication,
erasure coding...) and translating store requests into proper
sub-requests to the lower layers.
lrs: mark R/O-mounted media as full
Mark media as full if operations happen to have failed with -EROFS.
This can typically occur when LTFS decides that a tape does not contain
enough space to be mounted writable.
dss: make dss_unlock return ENOLCK when a lock is not owned.
When releasing a lock that is not owned by the caller, the resulting
error code was EEXIST, which was somehow confusing.
Replace this error by ENOLCK (No lock available).
dss: make dss_get resilient to missing fields in media stats.
Commit 4c2b7801 added new fields to media_stats (JSON stored in the DB).
When running dss_get on a previous phobos DB, these fields are missing.
Don't break the whole dss_get in that case. Actually make all
media_stats non-critical. Simply return 0 for the missing ones.
......@@ -9,7 +9,7 @@ rpm: dist phobos.spec
rpmbuild --define="_topdir $(rpm_dir)" -ta $(distdir).tar.gz
EXTRA_DIST= phobos.spec scripts/pho_ldm_helper \
scripts/phobos_db doc/ doc/cfg/template.conf ChangeLog
scripts/phobos_db doc/cfg/template.conf
sbin_SCRIPTS=scripts/pho_ldm_helper \
......@@ -4,5 +4,5 @@
* To setup autotools stuff the first time you check out the repo, run:
* Add PGSQL 9.4 to the build environment
* (Before RHEL8/CentOS8) Add PGSQL 9.4 to the build environment
export PKG_CONFIG_PATH=/usr/pgsql-9.4/lib/pkgconfig
\ No newline at end of file
# Phobos documentation
## Installation
### Requirements for tape access
You need to install LTFS >= 2.4 to enable phobos access to tapes.
LTFS RPM can be found on IBM Fix Central: [ltfs 2.4](
You can also retrieve its sources on gihub:
If you want to build RPMs from these sources, you can find packaging resources
(i.e. spec file) for LTFS here:
Note: since LTFS 2.4, `lin_tape` driver is no longer needed to access tapes.
LTFS now uses the standard linux tape driver (st).
### Phobos installation
Install phobos and its requirements:
yum install phobos
### Database setup
#### On RHEL8/CentOS8
Install postgresql:
dnf install postgresql-server postgresql-contrib
Initialize postgresql directories:
postgresql-setup --initdb --unit postgresql
Edit `/var/lib/pgsql/data/pg_hba.conf` to authorize access from phobos host
(localhost in this example):
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all md5
# IPv6 local connections:
host all all ::1/128 md5
Start the database server:
systemctl start postgresql
Finally, create phobos database and tables as postgres user (the password for
SQL phobos user will be prompted for unless provided with -p):
sudo -u postgres phobos_db setup_db -s
#### On RHEL7/CentOS7
Install postgresql version >= 9.4 (from EPEL or any version compatible with
Postgres 9.4):
yum install postgresql94-server postgresql94-contrib
Initialize postgresql directories:
/usr/pgsql-9.4/bin/postgresql94-setup initdb
Edit `/var/lib/pgsql/9.4/data/pg_hba.conf` to authorize access from phobos host
(localhost in this example):
# "local" is for Unix domain socket connections only
local all all trust
# IPv4 local connections:
host all all md5
# IPv6 local connections:
host all all ::1/128 md5
Start the database server:
systemctl start postgresql-9.4.service
Finally, create phobos database and tables as postgres user (the password for
SQL phobos user will be prompted for unless provided with -p):
sudo -u postgres phobos_db setup_db -s
## Upgrade of an existing instance
After upgrading phobos, run the DB conversion script (credentials to connect to
the database will be retrieved from /etc/phobos.conf):
/usr/sbin/phobos_db migrate
# 'y' to confirm
## First steps
### Launching phobosd
To use phobos, a daemon called phobosd needs to be launched. It mainly manages
the drive/media allocation and sets up resources to read or write your data.
To launch the daemon:
# systemctl start phobosd
To stop the daemon:
# systemctl stop phobosd
phobosd comes with its option set:
- -i: launch phobosd in an interactive way (attached to a terminal)
- -c: path to phobosd configuration file
- -s: print log messages into syslog
- -q/v: decrease/increase log verbosity
### Adding drives
Use the `phobos` command line to add new tape drives to be managed by phobos.
It is recommanded to specify a device path which is persistent after reboot
(eg. managed by dev mapper).
By default, drives are added in a locked state, so they are not immediatly used
in production.
To enable using a drive in production as soon as it is added, specify the
'--unlock' option:
phobos drive add --unlock /dev/mapper/LTO6-012345
### Adding media
To add tapes to be managed by phobos, use the `phobos` command line.
It is mandatory to specify a media type (like LTO8, T10KD...) with option `-t`:
phobos tape add -t lto6 [073200-073222]L6
**Note1:** this operation requires no access to the medium.
**Note2:** the set of supported media types, drive types, and compatibility rules
between drives and media are configurable. See section '*Configuring device
and media types*' below for more details.
Once a media has been added, it needs to be formatted. You can optionally unlock
the tape when the operation is done, so it becomes usable in production.
phobos tape format --unlock 073200L6
### Writing data
#### Using the API
You can access data in phobos using the phobos API:
* Header file is `phobos_store.h`.
* Library is `libphobos_store` (link using `-lphobos_store`).
Calls for writing and retrieving objects are `phobos_put()` and
#### Using the CLI
The CLI wraps `phobos_put()` and `phobos_get()` calls.
##### Writting objects
When pushing an object you need to specify an **identifier** that can be later
used to retrieve this data.
Phobos also allows specifying an arbitrary set of attributes (key-values) to be
attached to the object. They are specified when putting the object using the
'-m' option.
# phobos put <file> <id> -m <k=v,k=v,...>
phobos put myinput_file foo-12345 -m user=$LOGNAME,\
put_time=$(date +%s),version=1
For better performances when writing to tape, it is recommanded to write batches
of objects in a single put command. This can be done using `phobos mput`
`phobos mput` takes as agument a file that contains a list of input files (one
per line) and the corresponding identifiers and metadata, with the following format:
`<src_file> <object_id> <metadata|->`.
Example of input file for mput:
/srcdir/file.1 obj0123 -
/srcdir/file.2 obj0124 user=foo,md5=xxxxxx
/srcdir/file.3 obj0125 project=29DKPOKD,date=123xyz
The mput operation is simply:
phobos mput list_file
##### Reading objects
To retrieve the data of an object, use `phobos get`. Its arguments are the
identifier of the object to be retrieved, as well as a path of target file.
For example:
phobos get obj0123 /tmp/obj0123.back
##### Reading object attributes
To retrieve custom object metadata, use `phobos getmd`:
$ phobos getmd obj0123
## Device and media management
### Listing resources
Any device or media can be listed using the 'list' operation. For instance,
the following will list all the existing tape identifiers:
phobos tape list
The 'list' output can be formatted using the -o option, following a list of
comma-separated attributes:
phobos tape list -o stats.phys_spc_used,tags,adm_status
If the attribute 'all' is used, then the list is printed with full details:
phobos tape list -o all
Other options are available using the -h option.
### Locking resources
A device or media can be locked. In this case it cannot be used for
subsequent 'put' or 'get' operations:
phobos drive {lock|unlock} <drive_serial|drive_path>
phobos media {lock|unlock} <media_id>
### Tagging resources
Phobos implements a flexible mechanism to partition storage resources into
(possibly overlapping) subsets.
This mechanism is based on **tags**.
Multiple tags can be associated to the storage media at creation time or later
# Add media with tags "class1" and "project2"
phobos tape add -T class1,project2 -t LTO8 [073000-073099]L8
# Update tags for these tapes
phobos tape update -T class2 [073000-073099]L8
# Clears tags
phobos dir update -T '' [073000-073099]L8