ADASS posters are displayed all week
When
Theme: Science with data archives: challenges in multi-wavelength and time domain data analysis
The SKA Observatory is a next-generation radio astronomy facility that will help to revolutionise our understanding of the Universe and the laws of fundamental physics. The observatory has three locations: in South Africa's Karoo region (SKA_MID), Western Australia's Murchison Shire (SKA_LOW) and the Global Headquarters in the United Kingdom. The SKA_MID and SKA_LOW locations will be capable of producing a stream of science data products on the order of 700 PB/year. This large data volume is unprecedented for the astronomical community and thus poses unique challenges for curating and providing access to the datasets and resources required to analyse them in order to derive the final scientific insights. The approach chosen is the development and adoption of the SKA regional centre concept in the form of a loose SRCNet association consisting of regionally funded contributions.
The SRCNet data lake will be centrally managed but distributed and federated at the storage elements level. Known challenges of data lakes should be addressed like data exploitation of the data lake through the integration of data and computing and data latency due to distributed repositories. We present the architecture design that is being developed for the SRCNet to allow scientific analysis of the SKA data from the SRCNet data lake that minimises as much as possible the throwbacks of the federated data lakes.