CERN Accelerating science

Docker on OpenStack

Date published: 
Monday, 1 September, 2014
Document type: 
Summer student report
N. Agarwal
Project Specification: CERN is establishing a large scale private cloud based on OpenStack as part of the expansion of the computing infrastructure for storing the data coming out of the Large Hadron Collider (LHC) experiments. As the data coming out of the detectors is increasing continuously that needs to be stored in the data center, we need more physical resources (more money) and since Virtual machines takes lot of CPU and memory overhead and minutes for creating the images, booting up and for snapshotting as well. So here comes the solution to use Docker containers. Docker is an open platform to build, ship and run distributed applications. Docker being a container based virtualisation framework makes use of LXC. Docker containers are lightweight and fast and docker makes use of Union File System which makes it unique. Docker comes with the Docker Index/ Hub where you can store and share the docker images. This project involves the understanding of Docker and docker containers in detail, deployment of private Docker Registry as well as the integration of docker with Openstack to enable the Nova compute service to use the docker API as compute driver instead of the libvirt API. Abstract: At CERN, with the ever increasing amount of data coming out of the detectors that needs to be stored in the data center, new ways are sought to help analyze and store this data as well as help researchers perform their own experiments. To help offer solutions to such problems, CERN has employed the use of cloud computing and in particular OpenStack; an open source and scalable platform for building public and private clouds. OpenStack is used to view, create, and manage resources in a cloud and automate the tasks. Compute nodes form the resource core of the OpenStack Compute cloud, providing the processing, memory, network and storage resources to run instances. As the data is increasing continuously around 50 PB/sec and about 5 PB/day of data that needs to be stored, CERN is looking for new ways to utilise the hardware resources of the data center more efficiently. In this project we outline and document the integration of Docker with the Nova compute service of OpenStack (Devstack, Packstack), deployment of private Docker Registry at CERN for pushing and pulling the docker images. To allow the Nova compute service to use to the Docker API as compute driver instead of the Libvirt driver and to allow nova to boot the docker images, we need to store the docker images in glance that acts as an independent docker registry after configuration. In this report, we describe about docker, its basics and importance of docker containers in comparison to virtual machines, steps for deploying and configuring the private Docker Registry at CERN and steps for configuring the Nova to use docker driver in Devstack on Ubuntu cloud image and Packstack on RHEL 7.