Data Access and Analysis Tools

The Rubin Science Platform (RSP) is the set of integrated web-based applications, services, and tools to query, visualize, subset, and analyze LSST data - including the DP0.1 simulated data products. Throughout DP0 the RSP will be in active development by Rubin Observatory staff, with access provided to delegates on a shared-risk basis.

RSP usage: risks and responsibilities

All RSP users are responsible for being aware of the risks inherent in using software in active development, and must familiarize themselves with these risks prior to using the RSP.

Rubin Science Platform (RSP)

During DP0, the Rubin Science Platform (RSP) provides access to Rubin Observatory data products via two services: a Notebook Aspect and a Portal Aspect. The Notebook Aspect enables programmatic interaction with, and analysis of, data products in a python environment. The Portal Aspect provides interactive query, data discovery, and visualization tools. In the Operations era, the RSP users will be able to use the Portal and Notebook aspects in tandem (whereas for DP0.1 there is no connection between the two), and will have access to a third service: the Application Programming Interface (API) aspect. A full description of the Operations-era RSP can be found in the RSP Vision Document (LSE-319), but again, some of that envisioned RSP functionality is not available for DP0 (see RSP usage: risks and responsibilities).

../_images/RSP_home.png

Above is shown the landing page of data.lsst.cloud. The Notebook and Portal Aspects can be accessed by clicking on their respective icons on this dashboard. The API aspect is not available during DP0.1.

Portal Aspect

The Portal Aspect of the RSP provides an environment for data discovery, query, filtering, and visualization. During DP0.1, only catalog data will be accessible via the Portal (not images), and in particular only catalog data generated from the coadded images, and not the single-visit images, which means there is no time-domain data accessible via the Portal during DP0.1). Note that the Portal Aspect has not been under active development recently and it is expected to evolve significantly before the first LSST annual data release.

Notebook Aspect

The Notebook Aspect of the RSP provides an environment from which users can access and manipulate Rubin data products on the same machines where those data products reside (i.e., “next-to-the-data” – no downloading of data to your local machine). In particular, the Notebook Aspect offers Python-based access to DP0.1 data products via a custom implementation of web-based JupyterLab Notebooks (JupyterLab documentation). A file browser and a terminal command-line interface are also available within the JupyterLab environment.

Within the RSP Notebook Aspect, users can query DP0.1 data sets, subset and retrieve catalogs, manipulate and display images, calculate derived properties, plot results, reprocess the data, and most other analyses you can imagine performing with Python on astronomical images and catalogs. A stable software environment is provided and maintained for users, which includes many commonly-used packages and the Data processing tools including the LSST Pipelines. For DP0, this environment will only support Python 3. For DP0.1, images can only be accessed via the Notebook Aspect, and not the Portal.

API Aspect

The API (Application Programming Interface) Aspect of the RSP enables programmatic access to the Rubin data products via Virtual Observatory (VO) interfaces. Users will be able to remotely access the LSST data and DAC services using tools they’re already familiar with, e.g. TOPCAT or libraries such as Astropy. The Portal and Notebook Aspects of the RSP make use of the same APIs to internally access the LSST datasets.

The API Aspect of the RSP is very powerful and will eventually allow for federation with other astronomical archives, bringing added value to the LSST dataset.

Data processing tools

Documentation for the LSST Science Pipelines, a software package which is available to all RSP users via the Notebook Aspect, can be found at pipelines.lsst.io.

A brief summary of how the LSST Science Pipelines were used to create the DP0.1 data products can be found in this Data Processing Overview.

To view a list of packages available to you in the Notebook Aspect of the RSP, type pip install in a terminal.