SkyDriver v2

A SaaS Solution for Neutrino Event Reconstruction using the Skymap Scanner

_Authors

_{WIPAC Developers / developers@icecube.wisc.edu}

_Keywords

_{WIPAC  ·  IceCube  ·  Skymap Scanner  ·  Reconstruction  ·  IceTray  ·  EWMS}

_URLs

_{Homepage  ·  Tracker  ·  Source  ·  Documentation}

Overview

SkyDriver automates the entire scanning of an event: starting all servers and workers, transferring all needed data, and finally, all tear-down. SkyDriver also includes a database for storing scan requests, progress reports, and results. The computational engine for a scan is the Skymap Scanner. The main interface is a REST server with several routes and methods.

One of many workflows may be:

1. Request a scan (POST @ /scan)

2. Monitor the scanning status (GET @ /scan/SCAN_ID/status)

3. Check for progress updates (GET @ /scan/SCAN_ID/manifest)

4. Check for partial results (GET @ /scan/SCAN_ID/result)

5. Get a final result (GET @ /scan/SCAN_ID/result)

6. Make plots

Another workflow:

1. Find a scan id for a particular run and event (POST @ /scans/find)

2. Get the scan’s manifest and result (GET @ /scan/SCAN_ID)

 

Getting Started

Users interface with SkyDriver via REST calls, so first, you will need to get a connection. This example uses wipac-rest-tools:

from rest_tools.client import RestClient, SavedDeviceGrantAuth


def get_rest_client() -> RestClient:
    """Get REST client for talking to SkyDriver.

    This will present a QR code in the terminal for initial validation.
    """

    # NOTE: If your script will not be interactive (like a cron job),
    # then you need to first run your script manually to validate using
    # the QR code in the terminal.

    return SavedDeviceGrantAuth(
        "https://skydriver.icecube.aq",
        token_url="https://keycloak.icecube.wisc.edu/auth/realms/IceCube",
        filename="device-refresh-token",
        client_id="skydriver-external",
        retries=0,
    )


rc = get_rest_client()

Now, you can make all the REST calls needed:

rc.request_seq(method, path, args_dict)

Two Quick Examples

To request a new scan (see POST @ /scan):

manifest = rc.request_seq("POST", "/scan", {"docker_tag": ...})
print(json.dumps(manifest))

To see your scan’s status (see GET @ /scan/SCAN_ID/status):

status = rc.request_seq("GET", f"/scan/{scan_id}/status")
print(json.dumps(status))

 

REST API

See SkyDriver Docs for the public-facing routes and methods:

• API Endpoints

• Object Glossary

 

Using a Scan Result Outside of SkyDriver

Making Plots with a Scan’s Result (using the scan_id)

See skyreader’s plot_skydriver_scan_result.py

Creating a SkyScanResult Instance from a Scan’s Result (using the scan_id)

Also, see skyreader’s plot_skydriver_scan_result.py