Designing and operating distributed data infrastructures and computing centres poses challenges in areas such as networking, architecture, storage, databases, and cloud. These challenges are amplified and added to when operating at the extremely large scales required by major scientific endeavours. CERN is evaluating different models for increasing computing and data-storage capacity, in order to accommodate the growing needs of the LHC experiments over the next decade. All models present different technological challenges. In addition to increasing the on-premise capacity of the systems used for traditional types of data processing and storage, explorations are being carried out into a number of complementary distributed architectures and specialised capabilities offered by cloud and HPC infrastructures. These will add heterogeneity and flexibility to the data centres, and should enable advances in resource optimisation.

 

Project goal

The aim of this project is to demonstrate the scalability and performance of Kubernetes and Google Cloud, validating this set-up for future computing models. The focus is on taking both existing and new high-energy physics (HEP) use cases and exploring the best suited and most cost-effective set-up for each of them.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Ricardo Manuel Brito da Rocha
Collaborator liaison(s)
Grazia Frontoso, Karan Bhatia, Kevin Kissell

Collaborators

Project background

Looking towards the next-generation tools and infrastructure that will serve HEP use cases, exploring external cloud resources opens up a wide range of new possibilities for improved workload performance. It can also help us to improve efficiency in a cost-effective way.

The project relies on well-established APIs and tools supported by most public cloud providers – particularly Kubernetes and other Cloud Native Computing Foundation (CNCF) projects in its ecosystem – to expand available on-premises resources to Google Cloud. Special focus is put on use cases with spiky usage patterns, as well as those that can benefit from scaling out to large numbers of GPUs and other dedicated accelerators like TPUs.

Both traditional physics analysis and newer computing models based on machine learning are considered by this project.

Recent progress

The year started with consolidation of the work from the previous year. New runs of more traditional physics analysis helped with the validation of Google Cloud as a viable and performant solution for scaling our HEP workloads. A major milestone for this work was the publication of a CERN story on the Google Cloud Platform website (see publications).

The main focus in 2020 though was on evaluating next-generation workloads at scale. We targeted machine learning in particular, as this places significant requirements on GPUs and other types of accelerators, like TPUs.

In addition to demonstrating that HEP machine-learning workloads can scale out linearly to hundreds of GPUs in parallel, we also demonstrated that public cloud resources have the potential to offer HEP users a way to speed up their workloads by at least an order of magnitude in a cost-effective way.

Next steps

Work is now continuing in the following areas:

  • Further expanding the existing setup to burst on-premises services to public cloud resources, building on cloud-native technologies.
  • Onboarding more use cases where on-demand resource expansion is beneficial, with a special focus on workloads requiring accelerators (such as those for machine learning).
  • Expanding the cost analysis for each of these workloads and improving the feedback to end users on the efficiency of their workloads.

Publications

    R. Rocha, L. Heinrich, Higgs-demo. Published on GitHub. 2019. cern.ch/go/T8QQ
    R. Rocha, Helping researchers at CERN to analyse powerful data and uncover the secrets of our universe. Published at the Google Cloud Platform CERN, 2020. cern.ch/go/Q7Tn

Presentations

    R. Rocha, L. Heinrich, Reperforming a Nobel Prize Discovery on Kubernetes (21 May). Presented at Kubecon Europe 2019, Barcelona, 2019. cern.ch/go/PlC8
    R. Rocha, L. Heinrich, Higgs Analysis on Kubernetes using GCP (19 September). Presented at Google Cloud Summit, Munich, 2019. cern.ch/go/Dj8f
    R. Rocha, L. Heinrich, Reperforming a Nobel Prize Discovery on Kubernetes (7 November). Presented at the 4th International Conference on Computing in High-Energy and Nuclear Physics (CHEP), Adelaide, 2019. cern.ch/go/6Htg
    R. Rocha, L. Heinrich, Deep Dive into the Kubecon Higgs Analysis Demo (5 July). Presented at CERN IT Technical Forum, Geneva, 2019. cern.ch/go/6zls

Oracle WebLogic on Kubernetes

Project goal

The aim of this project is to improve the deployment of Oracle WebLogic infrastructure in large-scale deployments, profiting from new technologies such as Kubernetes and Docker Containers. These technologies will help to make the infrastructure deployment process portable, repeatable, and faster, enabling CERN service managers to be more efficient in their daily work.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Antonio Nappi
Team members
Lukas Gedvilas
Collaborator liaison(s)
Monica Riccelli, Will Lyons, Maciej Gruzka, Cris Pedregal, David Ebert, Dmitrij Dolgušin

Collaborators

Project background

The Oracle WebLogic service has been active at CERN for many years, offering a very stable way to run applications core to the laboratory. However, we would like to reduce the amount of time we spend on maintenance tasks or creating new environments for our users. Therefore, we started to explore solutions that could help us to improve how we deploy Oracle WebLogic. Kubernetes has now made our deployment much faster, reducing the time spent on operational tasks and enabling us to focus more on developers’ needs.

Recent progress

In 2020, we focused on the structure of monitoring and logging systems; these dramatically improved the way we work.

Using Prometheus for the monitoring part, we were able to get much more information on both the infrastructure itself and the application layer. We can now easily see resource use level in Kubernetes, as well as how containers are behaving. Thanks to this, it will be much easier to build an efficient alert system.

We decided to introduce Fluentd as our logging component. This helped us to reduce the number of hosts writing to elastic search and to standardise the logs produced by our system. Meanwhile, we also managed to migrate more applications to Kubernetes: 70-80% of our current production is now running on K8s.

 

Next steps

We aim to complete the migration of production applications by mid 2021. We will also focus on the integration of Prometheus into our new alert system. In addition, we would also like to start investigating ArgoCD and Flux2 as solutions for making application and infrastructure deployments more oriented towards GitOps.

Publications

    A. Nappi. HAProxy High Availability Setup. Databases at CERN blog. 2017. cern.ch/go/9vPf
    A. Nappi. HAProxy Canary Deployment. Databases at CERN blog. 2017. cern.ch/go/89ff

Presentations

    A. Nappi, WebLogic on Kubernetes at CERN (16 May). Presented at WebLogic Server Summit, Rome, 2019.
    A. Nappi, One Tool to Rule Them All: How CERN Runs Application Servers on Kubernetes (16 September). Presented at Oracle Code One 2019, San Francisco, 2019. cern.ch/go/DbG9
    D. Ebert (Oracle), M. Martin, A. Nappi, Advancing research with Oracle Cloud (18 September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/LH6Z
    E. Screven, A. Nappi, Cloud Platform and Middleware Strategy and Roadmap (17 September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/d8PC
    M. Riccelli, A. Nappi, Kubernetes: The Glue Between Oracle Cloud and CERN Private Cloud (17 September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/Bp8w
    A. Nappi, L. Rodriguez Fernández, WebLogic on Kubernetes (17 January). Presented at CERN openlab meeting with Oracle in Geneva, Geneva, 2017. cern.ch/go/6Z8R
    S. A. Monsalve, Development of WebLogic 12c Management Tools (15 August). Presented at CERN openlab summer students’ lightning talks, Geneva 2017. cern.ch/go/V8pM
    A. Nappi, L. Rodriguez Fernández, WebLogic on Kubernetes (16 August). Presented at Oracle Workshop Bristol, Bristol, 2017. cern.ch/go/6Z8R
    A. Nappi, WebLogic on Kubernetes (21 September). Presented at CERN openlab Open Day, Geneva, 2017. cern.ch/go/6Z8R
    A. Nappi, L. Rodriguez Fernández, Oracle Weblogic on Containers: Beyond the frontiers of your Data Centre Openday (21 September). Presented at CERN openlab Open Day, Geneva, 2017. cern.ch/go/nrh8
    A. Nappi, L. Gedvilas L. Rodríguez Fernández, A. Wiecek, B. Aparicio Cotarelo (9-13 July). Presented at 23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP), Sofia, Bulgaria, 2018. cern.ch/go/dW8J
    A. Nappi, L. Gedvilas L. Rodríguez Fernández, A. Wiecek, B. Aparicio Cotarelo (9-13 July). Presented at 23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP), Sofia, Bulgaria, 2018. cern.ch/go/dW8J
    L. Rodriguez Fernandez, A. Nappi, Weblogic on Kubernetes (11 January). Presented at CERN Openlab Technical Workshop, Geneva, 2018. cern.ch/go/6Z8R
    B. Cotarelo, Oracle Weblogic on Kubernetes (July). Presented at 23rd International Conference on Computing in High Energy and Nuclear Physics (CHEP), Sofia, 2018. cern.ch/go/6MVQ
    M. Riccelli, D. Cabelus, A. Nappi, Running a Modern Java EE Server in Containers Inside Kubernetes (23 October). Presented at Oracle OpenWorld 2018, San Francisco, 2018. cern.ch/go/b6nl
    W. Coekaerts, A. Nappi, Cloud Platform and Middleware Strategy and Roadmap (15 January). Presented at Oracle OpenWorld Middle West, Dubai, 2020. cern.ch/go/9kqK
    M. Gruszka, W. Lyons, A. Nappi, Deploying Oracle WebLogic Server on Kubernetes and Oracle Cloud (15 January). Presented at Oracle OpenWorld Middle West, Dubai, 2020. cern.ch/go/6FVF
    W. Coekaerts, M. McMaster, R. Hussain, A. Nappi, Cloud Platform and Middleware Strategy and Roadmap (13 February). Presented at Oracle Openworld, London, 2020. cern.ch/go/f8FZ
    M. Gruszka, W. Lyons, A. Nappi, Deploying Oracle WebLogic Server on Kubernetes and Oracle Cloud (13 February). Presented at Oracle Openworld, London, 2020. cern.ch/go/6SLl

Hybrid disaster-recovery solution using public cloud

Project goal

In 2020, the Database Services group in the CERN IT department launched this project, in collaboration with Oracle, to explore how integration of commercial cloud platforms with CERN on-premises systems might improve the resilience of services. The aim of this project is to understand the benefits, limitations and cost of the cloud environment for application servers and databases.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Viktor Kozlovzky
Team members
Aimilios Tsouvelekakis, Alina Andreea Grigore, Andrei Dumitru, Antonio Nappi, Arash Khodabandeh, Artur Wiecek, Borja Aparicio Cotarelo, Edoardo Martelli, Ignacio Coterillo Coz, Sebastian Lopienski
Collaborator liaison(s)
Cris Pedregal, Alexandre Reigada, David Ebert, Vincent Leocorbo, Dmitrij Dolgušin

Collaborators

Project background

Today, high availability is a key requirement for most platforms or services. The Database Services group maintains critical services for CERN that are used daily by the majority of users. This project enables us to review the current system, assess its scalability and explore further capabilities by integrating them with new technologies.

Recent progress

For the integration exercise, the group is using the Oracle Cloud Infrastructure (OCI).

The project team successfully replicated its process for creating virtual machines on OCI. This involved registering public-cloud virtual machines on the CERN main network, as well as integrating them with CERN’s central configuration management system.

The ability to deploy machines on OCI within the CERN network enabled us to use OCI running Oracle databases like on-premises Oracle databases. Moreover, we automated the procedure for creating standby databases for our on-premises primary databases, and we configured a Data Guard broker for data synchronisation between them. We performed tests with different data sets to evaluate the performance on the network and storage sides.

The team maintaining Kubernetes applications investigated the container engine provided by Oracle for the Kubernetes (OKE) platform. We extended the application deployment process, which is now capable of deploying applications to the OCI running OKE platform.

Furthermore, using an Oracle REST Data Services (ORDS) application, we performed a complex integration test; for forwarding network traffic, we used a proxy server. The tests showed vulnerabilities and integration limitations for the applications. 

Next steps

We will continue to evaluate the performance of storage and network for databases and see what the best fits would be for our use cases. We will also investigate the cost of running the databases on OCI compared to running them in CERN’s data centre.

The members of the project would like to thank the support teams for Oracle Cloud Infrastructure and Oracle Terraform for their valuable assistance.

Dynamical Exascale Entry Platform – Extreme Scale Technologies (DEEP-EST)

Project goal

The main focus of the project is to build a new kind of a system that makes it possible to run efficiently a wide range of applications — with differing requirements — on new types of high-performance computing (HPC) resources. From machine-learning applications to traditional HPC applications, the goal is to build an environment which is capable of accommodating workloads that pose completely different challenges for the system.

Collaborators:

DEEP-EST collaboration

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Maria Girone (for DEEP-EST task 1.7)
Team members
Viktor Khristenko (for DEEP-EST task 1.7)

Collaborators

Project background

DEEP-EST is an EC-funded project that launched in 2017, following on from the successful DEEP and DEEP-ER projects. The project involves 27 partners in more than 10 countries and is coordinated from Jülich Supercomputing Centre at Forschungszentrum Jülich in Germany.

Overall, the goal is to create a modular supercomputer that best fits the requirements of diverse, increasingly complex, and newly emerging applications. The innovative modular supercomputer architecture creates a unique HPC system by coupling various compute modules according to the building-block principle: each module is tailored to the needs of a specific group of applications, with all modules together behaving as a single machine.

Specifically, the prototype consists of three compute modules: the cluster module (CM), the extreme scale booster (ESB), and the data-analytics module (DAM).

  • Applications requiring high single-thread performance are targeted to run on the CM nodes, where Intel® Xeon® Scalable Processor (Skylake) provide general-purpose performance and energy efficiency.
  • The architecture of ESB nodes was tailored for highly scalable HPC software stacks capable of extracting the enormous parallelism provided by Nvidia V100 GPUs.
  • Flexibility, large memory capacities using Intel® Optane™ DC persistent memory and different acceleration capabilities (provided by Intel Stratix 10 and Nvidia V100 GPU on each node) are key features of the DAM; they make it an ideal platform for data-intensive and machine-learning applications.

CERN, in particular the CMS Experiment, participates by providing one of the applications that are used to evaluate this new supercomputing architecture.

Recent progress

During the past year, the focus was on testing the functionality developed during the previous year using the full DEEP-EST prototype system and optimising for the needs of the CMS Experiment. In particular, the tests we ran included more than 1000 cores and just under 100 Nvidia GPUs simultaneously, close to 100% of the available computing resources for our prototype. Furthermore, tests included not only traditional CMS Experiment reconstruction, but also a complex deep-learning pipeline. The workflow was used to benchmark the usage of Nvidia GPUs with MPI (over InfiniBand) for the purpose of distributed training of a model based on Graph Neural Networks.

Next steps

The work is ongoing to integrate the usage of Message Passing Interface (MPI) to add the ability to use not only GPUs available on the node that is performing data processing, but also on the remotely connected nodes. This will allow more efficient utilisation of heterogeneous resources, providing an overall speed up (i.e. higher throughput) of the CMS reconstruction application. The DEEP-EST project came to a successful conclusion during the first quarter of 2021.

Publications

    The following deliverables were submitted to the European Commission:
    Deliverable 1.2: Application Use Cases and Traces
    Deliverable 1.3: Application Distribution Strategy
    Deliverable 1.4: Initial Application Ports
    M. Girone, Common challenges for HPC integration into LHC computing. Published on ZENODO, 2020. cern.ch/go/7bt9
    The following deliverable was submitted to the European Commission:
    Deliverable 1.5: Final report on application experience

Presentations

    V. Khristenko, CMS Ecal Reconstruction with GPUs (23 October). Presented at CMS ECAL DPG Meeting, Geneva, 2019. cern.ch/go/FC6j
    V. Khristenko, CMS Hcal Reconstruction with GPUs (8 November). Presented at CMS HCAL DPG Meeting, Geneva 2019. cern.ch/go/P6Js
    V. Khristenko, Exploiting Modular HPC in the context of DEEP-EST and ATTRACT projects (22 January). Presented at CERN openlab Technical Workshop, Geneva, 2020. cern.ch/go/rjC7

Heterogeneous I/O for Scale

Project goal

We are working to develop a proof of concept for an FPGA-based I/O intermediary. The potential impact would be to change the way data ingestion happens when using remote storage locations. In view of the enormous amounts of data to be employed in the future for data analytics, it is crucial to efficiently manage the flow in order to harness the computational power provided by high-performance computing (HPC) facilities.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Maria Girone, Viktor Khristenko
Collaborator liaison(s)
Ulrich Bruening (University of Heidelberg), Mondrian Nuessle (Extoll GmbH)

Collaborators

Project background

One of the common aspects of all data-intensive applications is the streaming of recorded data from remote storage locations. This often imposes constraints on the network and forces a compute node to introduce complex logic to perform aggressive caching in order to remove latency. Moreover, this substantially increases the memory footprint of the running application on a compute node. This project, abbreviated to ‘HIOS’, aims to provide a scalable solution for such data-intensive workloads by introducing heterogeneous I/O units directly on the compute clusters. This makes it possible to offload the aggressive caching functionality onto these heterogeneous units. By removing this complicated logic from compute nodes, the memory footprint decreases for data-intensive applications. Furthermore, the project will investigate the possibility of including additional logic, coding/decoding, serialisation, I/O specifics, directly onto such units.

An integral part of the project will be the ability to integrate the units developed directly with current HPC facilities. One of the main outcomes of the project will be the reduced time required to extract insights from large quantities of acquired information, which, in turn, directly impacts society and scientific discoveries.

HIOS is one of the 170 breakthrough projects receiving funding through the ATTRACT initiative. ATTRACT, which is part of the European Union’s Horizon 2020 programme, is financing breakthrough ideas in the fields of detection and imaging.

Recent progress

During the past year, the first phase of the ATTRACT project, we performed testing and validation of the TCP/IP network stack. This targeted an FPGA and could be used to integrate with Extoll’s existing solution. At the same time, a range of compression/decompression algorithms (e.g. LZMA, LZ4, zstd, Deflate) were explored in order to incorporate not only network-related functionality, but also data-processing primitives.

Next steps

The first phase of the project was concluded at the end of 2020. Discussions are currently underway regarding how to take this work forwards.

Publications

    V. Khristenko, M. Girone, M. Nuessle, D Frey, U. Bruening, J. Schonbohm, HIOS: Heterogenous I/O for Scale. Published on ATTRACT, 2020. cern.ch/go/qH7N

EOS productisation

Project goal

This project is focused on the evolution of CERN’s EOS large-scale storage system. The goal is to simplify the usage, installation, and maintenance of the system. In addition, the project aims to add native support for new client platforms, expand documentation, and implement new features/integration with other software packages.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Luca Mascetti
Team members
Fabio Luchetti, Elvin Sindrilaru
Collaborator liaison(s)
Gregor Molan, Branko Blagojević, Ivan Arizanović, Svetlana Milenković

Collaborators

Project background

Within the CERN IT department, a dedicated group is responsible for the operation and development of the storage infrastructure. This infrastructure is used to store the physics data generated by the experiments at CERN, as well as the files of all members of personnel.

EOS is a disk-based, low-latency storage service developed at CERN. It is tailored to handle large data rates from the experiments, while also running concurrent complex production workloads. This high- performance system now provides more than 350 petabytes of raw disks.

EOS is also the key storage component behind CERNBox, CERN’s cloud-storage service. This makes it possible to sync and share files on all major mobile and desktop platforms (Linux, Windows, macOS, Android, iOS), with the aim of providing offline availability to any data stored in the EOS infrastructure.

Recent progress

In 2020, Comtrade's team continued to improve and update the EOS technical documentation. Due to the COVID pandemic and the travel restrictions during the year, the hosting of dedicated hardware resources at CERN, to support the prototyping of an EOS-based appliance, was postponed.

The EOS-Comtrade team therefore focused their efforts on the development of a native way to interact from the Windows environment to the distributed storage system. Currently, Windows users access EOS only via dedicated gateways, where the system is exposed with SAMBA shares.

In order to provide more performant and seamless access to EOS for Windows users, Comtrade started developing a dedicated Windows client, codenamed EOS-wnc.

This new, dedicated, command-line Windows client was developed against the EOS server APIs. It now covers all EOS commands from the standard user listing and copy commands to the administrative ones.

On top of this, some EOS commands on Windows have additional functionalities, like improved tab-completion and history features.

Next steps

Together, we will focus on improving the current EOS Windows native client, adding new functionalities, and integrating it in the Windows user interface, developing a dedicated driver.

It is still planned to host dedicated hardware resources at CERN to support prototyping of an EOS-based appliance. This will enable Comtrade to create a first version of a full storage solution and to offer it to potential customers in the future.

Publications

    X. Espinal, M. Lamanna, From Physics to industry: EOS outside HEP. Published in Journal of Physics: Conference Series, Vol. 898, 2017. cern.ch/go/7XWH

Presentations

    L. Mascetti, Comtrade EOS productization (23 January). Presented at CERN openlab technical workshop, Geneva, 2019. cern.ch/go/W6SQ
    G. Molan, EOS Documentation and Tesla Data Box (4 February). Presented at CERN EOS workshop, Geneva, 2019. cern.ch/go/9QbM
    L. Mascetti, EOS Comtrade project (23 January). Presented at CERN openlab Technical workshop, Geneva, 2020. cern.ch/go/l9gc
    L. Mascetti, CERN Disk Storage Services (3 February 2020). Presented at CERN EOS workshop, Geneva, 2020. cern.ch/go/pF97
    G. Molan, Preparing EOS for Enterprise Users (27 January 2020). Presented at Cloud Storage Services for Synchronization and Sharing (CS3), Copenhagen, 2020. cern.ch/go/tQ7d
    G. Molan, EOS Documentation for Enterprise Users (3 February 2020). Presented at CERN EOS workshop, Geneva, 2020. cern.ch/go/swX8
    G. Molan, EOS Windows Native Client (3 February 2020). Presented at CERN EOS workshop, Geneva, 2020. cern.ch/go/P7DX
    G. Molan, EOS Storage Appliance Prototype (5 February 2020). Presented at CERN EOS workshop, Geneva, 2020. cern.ch/go/q8qh
    G. Molan, EOS-wnc demo (30 October 2020). Presented at IT-ST-PDSComtrade workshop, Geneva, 2020. cern.ch/go/q8qh

Oracle Management Cloud

Project goal

We are testing Oracle Management Cloud (OMC) and providing feedback to Oracle, including proposals for the evolution of the platform. We are assessing the merits and suitability of this technology for applications related to databases at CERN, comparing it with our current on-premises infrastructure.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Eva Dafonte Perez, Eric Grancher
Team members
Aimilios Tsouvelekakis
Collaborator liaison(s)
Simone Indelicato, Vincent Leocorbo, Cristobal Pedregal-Martin, David Ebert, Dmitrij Dolgušin

Collaborators

Project background

The group responsible for database services within CERN’s IT department uses and provides specialised monitoring solutions to teams across the laboratory that use database infrastructure. Since the beginning of 2018, we have an agreement in place with Oracle to test OMC, which offers a wide variety of monitoring solutions.

At CERN, as at other large organisations, it is very important to be able to monitor at all times what is happening with systems and applications running both locally and in the cloud. Thus, we conducted tests of OMC involving hosts, application servers, databases, and Java applications

Recent progress

Improvements proposed to Oracle during the previous year were implemented within new releases of the platform in 2019. Initial investigation shows that the platform has been enhanced with features covering most of our needs.

Furthermore, we deployed the registration application for the CERN Open Days in Oracle Cloud Infrastructure (see project ‘Developing a ticket reservation system for the CERN Open Days 2019’ for further details). The application made use of the Oracle Autonomous Transaction Processing (ATP) database to store visitor information. The behaviour of the ATP database was monitored using OMC, providing meaningful insights into the stresses put on the database and the database-hosting system during the period in which registration for the event was open.

Next steps

The next step for OMC is to use it as a monitoring platform for all the projects that are to be deployed in the Oracle Cloud Infrastructure.


Presentations

    A. Tsouvelekakis, Oracle Management Cloud: A unified monitoring platform (23 January). Presented at CERN openlab Technical Workshop, Geneva, 2019. cern.ch/go/Z7j9
    A. Tsouvelekakis, Enterprise Manager and Management Cloud CAB (April). Presented at Oracle Customer Advisory Board, Redwood Shores, 2019. cern.ch/go/tZD8
    A. Tsouvelekakis, CERN: Monitoring Infrastructure with Oracle Management Cloud (September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/mMd8

Developing a ticket reservation system for the CERN Open Days 2019

Project goal

The goal of this project was to deliver a web-based ticketing system, with zero downtime, for the CERN Open Days 2019. The system had to be capable of handling multiple ticket-release dates and high loads, as well as providing a user-friendly and responsive interface.

 

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Artur Wiecek
Team members
Antonio Nappi, Franck Pachot, Luis Rodriguez Fernandez, Nuno Guilherme, Matos De Barros, Thomas Løkkeborg, Viktor Kozlovszky
Collaborator liaison(s)
Pauline Gillet-Mahrer, Christian Jacobsen, Eric Mitha, Cristobal Pedregal-Martin, David Ebert, Dmitrij Dolgušin

Collaborators

Project background

With the LHC stopped for upgrade and maintenance during 2019, CERN took the opportunity to organise one of its major Open Days events in September. At these events, members of the public are able to visit many areas that are normally restricted. Our team's responsibility was to design, deliver, and maintain the ticketing system for the Open Days. The event took place over a full weekend, and was attended by 75,000 people.

Recent progress

The project was delivered in 75 days and was maintained for three months from mid-June. The system was made available in English and French, and it had a multi-layer architecture. Thanks to the cutting-edge technologies used, adherence to coding standards, and the fine-tuning possibilities offered by the various layers, we were able to successfully deliver a highly available and scalable system.

We used an Angular framework, with Angular material design on the front-end application layer and a REST API based on the Java Spring boot application as the back-end layer. These were run in containers, hosted in a Kubernetes environment. Each of the components were individually scalable based on the load, as was the Oracle Autonomous Transaction Processing database. All the components were hosted on the Oracle cloud, using cloud-native services like a load balancer, a container engine for Kubernetes, and Oracle Analytics Cloud. The system was stress tested and was capable of handling 20,000 parallel users across a period of six minutes.

The reservation system played an important role in the organisation of this large event.

Next steps

The project came to a successful close on 16 September 2019. As part of the project, we tested and evaluated the capabilities of the Oracle Cloud, which will serve as the foundation for a planned upcoming project on disaster recovery.

 

The project members would also like to thank the following people at CERN for their assistance:

  • Sébastien Masson, Manuel Martin Marquez, Aimilios Tsouvelekakis and Aparicio Cotarelo for their consultation services.
  • Francois Briard, the project owner.
  • Tobias Betz, for his assistance with the mail templates and the countdown timer.
  • Liviu Valsan, Sebastian Lopienski, and Jose Carlos Luna Duran for their assistance with security.
  • Bas Wallet and Ewa Lopienska for their work on the user interface and experience.
  • Ana Godinho, Odile Martin, Achintya Rao, and Corinne Pralavorio for their translation work.

Publications

    V. Kozlovszky, Open Days reservation system's high level overview – 2019. Databases at CERN blog. 2019. cern.ch/go/ff6k
    V. Kozlovszky, Internationalization of the 2019 Open Days reservation system. Databases at CERN blog. 2019. cern.ch/go/6qQB

Presentations

    E. Grancher, V. Kozlovszky, 100% Oracle Cloud: Registering 90,000 People for CERN Open Days (16 September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/6xZZ
    E. Grancher, 11 Months with Oracle Autonomous Transaction Processing (18 September). Presented at Oracle OpenWorld 2019, San Francisco, 2019. cern.ch/go/8PQ6