Facility for Antiproton and Ion Research

The mission of the international FAIR particle accelerator facility in Darmstadt is to unravel unsolved secrets regarding the structure of matter and the evolution of the Universe. FAIR will generate particle beams of a previously unparalleled intensity and quality. Ions of all the natural elements in the periodic table, as well as antiprotons, can be accelerated.

FAIR Scientific Challenges

FAIR will be the leading European Facility for accelerator-based atomic, nuclear and hadronic physics, as well on its applications. It is currently being constructed close to the existing GSI Helmholtz Centre, whose accelerators are integrated into the larger facility. The various experiments will provide unique in-sights into the fundamental building-blocks of our universe, so casually speaking, one could say we investigate “The Universe in the Lab”.

FAIR provides new insights into the structure of matter and the evolution of the universe from the Big Bang until today. Here, it is possible to create states of matter in the laboratory that otherwise only exist in the universe e.g. in stellar explosions or in the core of planets.

ESCAPE Impact to FAIR

The ESCAPE services were used by some of FAIR nuclear physics experiments, investigating matter at extreme densities or studying nuclei far off stability. They were also used in the simulations and test-beams preparing for the upcoming FAIR experiments.

The ESCAPE services are tightly integrated into the overall FAIR services, thus many researchers use them already without even knowing. An example is the OSSR software repository. By its tight integration with many OpenScience services and the EOSC, it supports the accessibility of our software chains, also making them available to other research communities. One can thus enjoy the benefits of Open Science without the overhead which normally comes with it.

ESCAPE DIOS

Many of the experiments at FAIR will produce data at high rates. An example is the Compressed Baryonic Matter (CBM) experiment which is at the centre of one of the FAIR research pillars. It will produce around 20PByte of scientific data per year when operational, which need to be made available to the world-wide research community for full scientific exploitation in an open way.

A ESCAPE DIOS service was partially developed and operated at FAIR in the form of an ESCAPE datalake implementation on an existing Lustre filesystem used for experiment and simulation data storage, where data could be sent, retrieved, and registered using, for example, a RUCIO python client. In the role of a test consumer for the DIOS service operated at FAIR, three experiments at FAIR/GSI were independently involved, namely the CBM experiment, the PANDA experiment, and the R3B experiment, to verify and validate data ingestion using either simulated or processed experimental particle physics data from their respective experiments. As part of a data acquisition challenge, data from mCBM, a prototype experiment of the CBM experiment, was registered in the FAIR datalake in real-time using an Infiniband backbone connection from the data acquisition and high level trigger system, miniFLES, of the experiment to the GSI Green Cube batch farm. This showed that we had, in principle, obtained the necessary tools, technology and expertise to implement this as live part of functioning particle physics experiment.

Without DIOS, FAIR would have been forced to develop such a sharing solution on its own; where obviously not such a broad functionality spectrum would have been reached. For the CBM example, this would have been the continuation of the traditional way of storing data in a facility specific data-storage system (Lustre) which would have restricted the data access to FAIR international partners until a proper solution would be available.

Data storage and handling constitute an important cost factor in research. For a researcher, before trying to set up his/her own solution, why not use something established among other projects of similar size and complexity? It will save researchers’ time and development costs, and will may get a better solution than a self-developed one, albeit one which probably still requires tailoring to the researchers’ specific needs.

Thanks to ESCAPE ESAP, the unspecified “other” researcher could have such a diverse range of challenges that answering this is impossible without first supposing a researcher with a very specified use case. FAIR’ advice to ESCAPE ESAP service providers, is that if they want to convince a future researcher, to make the use of ESCAPE ESAP services as easy as possible. Towards this aim, FAIR would strongly advise the keeping and publishing of simple straight forward concise documentation which is easily obtained by a simple search engine search.

ESCAPE OSSR

FAIR shared scientific software/workflows and services to the science community and across different communities.

For FAIR, as a large European research infrastructure, it is important to also make the software developed in its various experiments available in an open way. Taking CBM, which is a large international research collaboration with multiple institutional members, as an example, there are a number of” technical”,” designation”, and” distribution” software challenges in scientific research for the CBM experiment which may be alleviated by use of a service such as those offered by ESCAPE OSSR. CBMROOT is our software suite used to simulate and design the future CBM experiment at FAIR and will be used to process online and offline data once the full experiment become functional later this decade. As such it is important for the success of the CBM collaboration, to distribute community-validated software releases of our software framework to our world- wide members. Ability to permanently designate official releases for the purposes of record keeping, documentation and citation is highly advantageous to research projects such as CBM.

It fulfils our requirements for an open access repository and software catalogue. It is also compatible with outside requirements, e.g. it fulfils the needs of the Helmholtz research field matter. We are not aware of any other suitable solution of the same quality available. Being an active participant on the ESCAPE project, it was important for us to test with our own use-cases and promote the development of services which could be useful to large scale scientific research facility in Europe and elsewhere. Having members of the CBM collaboration with insider knowledge of OSSR during its development of the repository tools allowed us to naturally use these tools once applications were encountered.

FAIR made much of its science analysis software available to the different communities. FAIR took part in the development of metadata description for OSSR and in adjustment and optimisation of the onboarding procedure. So far, GSI/FAIR had on boarded following software packages:

1. - FairRoot (https://fairrroot.gsi.de)

2. - FairMQ (https://github.com/FairRootGroup/FairMQ)

3. - DDS (https://github.com/FairRootGroup/DDS)

4. - R3BRoot (https://r3broot.gsi.de)

Additionally, as a main hurdle to the inclusion of CBMROOT (https://git.cbm.gsi.de/computing/cbmroot), CBM's main simulation and data processing software framework, into the ESCAPE OSSR service, copyright licence agreement to distribute through the GNU public licence v3 was obtained from the institutional members of the CBM collaboration so recently CBMROOT is now scheduled to be included in the ESCAPE OSSR service as part of FAIR next software release (October 22) after the standard validation and verification as a fit-for-purpose research tool by our community of developers and physicists is completed.

ESCAPE ESAP

Without ESCAPE ESAP, the user analysis workflow would be much more site-specific and localised at FAIR as host institute. Missing the ESCAPE ESAP benefits would entail more required knowledge and efforts on installation and maintenance of the software infrastructure on the user side, reducing the effectiveness of the analysis work.  The Docker containers developed in the context of ESCAPE ESAP were very helpful to execute and distribute simulation and analysis workflows on various computing platforms (local desktop or notebook, compute farm) without the need to care about details of implementation and installation. This eased and accelerated FAIR research.

The organisation of the analysis of experiment data on a multi-PB scale by a large community of users worldwide is no trivial task. It is, however, required to efficiently and exhaustively make use of this data in order to provide publishable results in a reasonable timeframe. What is needed are standardised and easy-to-use computing platforms, abstracting from the underlying computing infrastructure and providing a low-threshold entry point for user analysis. Such platforms could already be distributed along with the software packages required for the data analysis, such that the users themselves need not bother with installation and maintenance. Such standardised platforms are also the key to distribute the analysis efforts to compute centres and sites worldwide, circumventing the potential bottleneck of concentrating the analysis efforts at the host lab of the experiment.

Many challenges, FAIR is confronted with in setting-up world-class particle and nuclear physics experiments, are shared by other large research institutes and projects in Europe and elsewhere. With this, FAIR wants to synergise the effort so that its labour requirement is reduced, but also, so that its generic solutions may be advantageous to other research institutes. Therefore, FAIR experiments have a strong interest in engaging with services such as ESAP. In short, FAIR hopes to profit from the experience of other scientific communities with similar needs towards the development of a commonly usable solution.

FAIR has provided interactive prototype analysis workflows exemplary for the CBM experiment based upon Jupyter Notebooks and Docker Containers. Upon the standard Ubuntu minimal installation of Linux, the Docker Containers were enabled with token based authentication via voms-client, with the ability to interact with the FAIR and CERN data lakes via Rucio, using gfal2 and xrootd. Upon these standalone container, a full suite of our bespoke scientific research software including FAIRMQ, FAIRROOT and CBMROOT was installed which allows data processing and analysis algorithms to be used within the ESCAPE project.

The notebook was provided with interfaces to the data lakes so that data can be retrieved from and the analysis results can be store in the ESCAPE data lakes. This includes an interface to the Rucio and an interface to the FAIR data lake implementation. In close collaboration with RUG, the CBM notebook has been integrated with the multi-user Jupyter Hub provided by RUG at Groningen. From there, including the Jupyter notebook for R3B as developed by RUG interface are provided to the ESCAPE ESAP platform so that sets can access the FAIR analysis workflows via the ESCAPE API Gateway.

As the FAIR experiments have not commenced data taking yet, no direct scientific research results were achieved for now. However, in the preparation and development phase, intense simulation and analysis efforts are needed, e.g., in order to aid design decisions on the shape of the detectors and to assess whether sufficient physics performance is obtainable. Thus, although FAIR principle aims as a proof-of-concept application, the CBM Jupyter notebooks can be used by researchers today to take notes with integrated snippets of code whilst the docker containers may be used as a fast way to onboard new members in the use of the science analysis platform, all with easy access to token-based authentication with the ESCAPE data lakes. The docker container were tested by the CBM experiment and the Jupyter notebooks were already tested by the R3B experiment for robustness and suitability.

FAIR expectations towards ESCAPE

FAIR hopes that ESCAPE will continue to operate and further develop and improve the services. Having some commonality between different communities may result in interesting solutions and increased sharing of technologies and methods.

LEARN MORE ABOUT HOW ESCAPE IS SUPPORTING THE OTHER RESEARCH INFRASTRUCTURES