POSTER P916: An original way to manage huge astronomical table

ADASS posters are displayed all week

When

10:16 p.m., Nov. 9, 2023

Theme: Other creative topics in astronomical software

pretalxeposter

We explored a way to wrap in database large astronomical tables stored in a remote repository and in a suitable format. The architecture is based on ATSS (Astronomical Table Serialisation System) which exploits binary file allowing a powerful position indexation well suited to intensive use. This binary format is used in the VizieR catalogue service which duplicates large tables in ATSS and in a postgreSQL database to provide an efficient conesearch service and the SQL flexibility needed by the IVOA Table Access Protocol (TAP). The database architecture requires an important capacity to deal with datasets exceeding already the billion of records (Gaia, SDSS, GSC, etc). Some new surveys such as LSST will drive a significant increase in volume which may require a redesign of the architecture. We explored a cost-effective solution in term of funds and maintenance that consists in wrapping ATSS binary data as foreign table. We developed a PostgreSQL extension using the Foreign-Data-Wrapper that maps SQL with the API of the binary file including its cone search and indexation capabilities. The solution has been tested with a selection of large tables in a TAP service, we will discuss the technology, its potential but also the limitation of the current development.

Contacts

Gilles Landais, Université de Strasbourg