The scope of this project — undertaken in collaboration with CERN openlab associate member Comtrade Software — is the evolution of CERN’s EOS large-scale storage system. The goal is to simplify the usage, installation, and maintenance of the system, as well as adding support for new platforms.
EOS is a disk-based, low-latency storage service developed at CERN. With its highly scalable hierarchical namespace, and data access possible via the XROOT protocol (to provide scalable, fault-tolerant access), it was initially designed for the storage of physics data. Access to physics data among the LHC community is typically characterised by many concurrent users, a significant fraction of random data access, and a large file-open rate. Today, however, EOS provides storage for both physics and other use cases.
In 2016, we created a robust installation kit to enable rapid installation of EOS both for evaluation purposes and for fast deployment in production. In 2017, testing of this beta version of EOS continued at a number of external institutes beyond CERN, such as the European Commission’s Joint Research Centre in Ispra, Italy, the University of Vienna, Austria, and the University of Trieste, Italy. At CERN, we also worked on a number of further developments to EOS throughout the year.
We will continue our evolving work on EOS testing, installation, and documentation. We aim to provide users with the full set of EOS functions: sync-and-share (through a tool we call CERNBox), erasure-coding, and the use of geographically distributed multi-site instances. An additional goal for us is to provide a testing framework that will run simultaneously after every release has been built, so as to certify each EOS version as accurately and as quickly as possible.
On top of this, we will also work to integrate new Comtrade Software engineers into the development and operations team at CERN, helping them to gain experience and autonomy in operations, maintenance, and development for EOS, so as to be able to provide direct support.