TUTORIAL T003: Exploring big data efficiently with the Astro Data Lab science platform

In the era of data-intensive astronomy the community needs to acquire skills to handle increasingly larger and more complex datasets, and to access high-performance computing and analysis tools. In this tutorial we will teach participants how to use data-proximate science platforms to conduct astronomy research. Using the Astro Data Lab science platform and the new SPARCL (SPectra Analysis and Retrievable Catalog Lab) service for spectroscopy, participants will first learn how to find documentation, information about all of Astro Data Lab’s data holdings of over 100 TB of wide- field survey catalogs, 2.5 PB of imagery, and millions of spectra from DESI and SDSS, and how to access help from the Astro Data Lab team. We will then teach the group in an interactive mode how to use various data services and analysis tools at Data Lab, including how to crossmatch tables, build and submit catalog queries, search for images and create cutouts, search for and download spectra, and how to use the Astro Data Lab Jupyter notebook server. The participants will execute and modify a number of science-case example notebooks from various domains of astronomy focusing on data analysis. The tutorial will also make use of some amenities on science platforms, including remote file storage and remote user-owned database tables.

Primary learning objectives

1. Construct SQL queries to query large datasets through a dedicated Jupyter Notebook server, web-interface, and command-line interface
2. Use Astro Data Lab’s X-match web service to crossmatch datasets
3. Discover and query for Sloan Digital Sky Survey (SDSS) and Dark Energy Spectroscopic Instrument (DESI) spectra with the new SPARCL tool
4. Create plots with the data obtained to realize the graphical and visualization capabilities within our notebooks
5. Get image cutouts for a set of objects using an image discovery service and a cutout tool

Register for an Astro Data Lab account (ideally done in advance): https://datalab.noirlab.edu/account/register.html 

Tutorial notebooks are available here: https://github.com/astro-datalab/Tutorial-ADASS-2023 

Click for instructions for tutorial T003 participants