C102: Multi-wavelength archival research: where are the obstacles and how to tackle them?

Theme: Science with data archives: challenges in multi-wavelength and time domain data analysis

pretalx

Multi-wavelength analysis of archival data can lead to groundbreaking discoveries. Following our ADASS tutorial in 2015, we discovered a population of intermediate-mass black holes using archival data from SDSS, Chandra, and XMM-Newton, which became a major contribution to the field. Despite the final success, our team faced numerous quite serious issues with data formats, access, reduction and analysis tools which slowed down the project by about 1.5 years. In this presentation, I will outline the most important challenges we faced and discuss the workarounds and a path forward. Presently, in order to succeed in multi-wavelength archival research, one really needs to be an expert in observations and data reduction in every spectral domain that is used. The main obstacle is that some major space missions and most ground-based observatories do not provide science ready data in the archives or provide them only for a small fraction of all observations. Data reduction efforts require very high time and manpower investments. Even when reduced data are available, they often do not conform to any standards (from FITS WCS representation to IVOA standards for metadata). A potential path forward is to organize a push from the funding agencies towards data providers to deliver science ready data which would comply to the FAIR principles.