Heterogeneous I/O for Scale


Project goal

We are working to develop a proof of concept for an FPGA-based I/O intermediary. The potential impact would be to change the way data ingestion happens when using remote storage locations. In view of the enormous amounts of data to be employed in the future for data analytics, it is crucial to efficiently manage the flow in order to harness the computational power provided by high-performance computing (HPC) facilities.

R&D topic
Data-centre technologies and infrastructures
Project coordinator(s)
Maria Girone, Viktor Khristenko
Collaborator liaison(s)
Ulrich Bruening (University of Heidelberg), Mondrian Nuessle (Extoll GmbH)


Project background

One of the common aspects of all data-intensive applications is the streaming of recorded data from remote storage locations. This often imposes constraints on the network and forces a compute node to introduce complex logic to perform aggressive caching in order to remove latency. Moreover, this substantially increases the memory footprint of the running application on a compute node. This project, abbreviated to ‘HIOS’, aims to provide a scalable solution for such data-intensive workloads by introducing heterogeneous I/O units directly on the compute clusters. This makes it possible to offload the aggressive caching functionality onto these heterogeneous units. By removing this complicated logic from compute nodes, the memory footprint decreases for data-intensive applications. Furthermore, the project will investigate the possibility to include additional logic, coding/decoding, serialisation, I/O specifics, directly onto such units.

An integral part of the project will be the ability to integrate developed units directly with current HPC facilities. One of the main outcomes of the project will be the reduced time required to extract insights from large quantities of acquired information, which, in turn, directly impacts society and scientific discoveries.

HIOS is one of the 170 breakthrough projects receiving funding through the ATTRACT initiative. ATTRACT, which is part of the European Union’s Horizon 2020 programme, is financing breakthrough ideas in the fields of detection and imaging.


Recent progress

Development work began in September 2019. The first task was to provide the full specification and initial implementation that will become the proof of concept.

Next steps

In 2020, work will be carried out to complete the proof of concept and to submit the deliverables defined within the ATTRACT project.