and last modified on
in mini projects2.155 minutes read
The Problem
We need to minimise time between a SN explosion and the PESSTO classification of the SN. We’ve minimised delays on the PESSTO side of the timeline (the easy bit!), but we need to help LSQ reduced their time delays if possible.
Timeline from Explosion to Classification
lsq timeline
The 2 time blocks highlighted in pink are:
Lag time between viable detections being added to the LSSNDB candidates table and human screening of those objects to find the ‘good’ LSQ targets.
Lag time between a ‘good’ target being screened, forced photometry being performed on the target and data added to the LSSNDB ms_lightcurves table.
Components of Problem
1. Lag between LSQ ‘Discovery’ and Human Screening
I’ve done some digging around in the data to try and quantify this lag time. In the plot below I define ‘discovery’ as the earilest JD in the LSSNDB candidates table for a given object. The screening date comes from the screening_date in the LSSNDB summary table (translated to UT).
Screenshot 2015–06–03 15.14.11
If this plot is correct, then you can see that 30% of the discovered objects are screened within the first 24 hrs, and the remaining 70% within around 3 days.
We are guessing the 2 bottlenecks here are:
Processing the data at NERSC, and
Eyeballing all of the potential candidates
I also tried to work out how many candidates you typically have to eyeball from a single night’s observing, but I think I’ve probably got this wrong. This is data taken from the candidates table - so these are probably the good targets you select from some larger list of potential candidates?
Screenshot 2015–06–03 15.30.02
2. Lag between Human Target Selection and Forced Photometry
From my estimates below you seem to be doing pretty well at getting the forced photometry done quickly after a taret is screened as a ‘good’ target. Our efforts are therefore probably best placed at trying to reduces the discovery to screening lag rather than here.
Screenshot 2015–06–03 15.39.52
Some questions for the LSQ guys -
Is the data streamed to NERSC for processing throughout the La Silla night, or is it a bulk dump at the end of the night?
How many candidates do you typically have to screen from a single night’s data?
Can PESSTO help with the screening of candidates to reduce the load on you guys and speed up the eyeballing process?
How CPU intensive is the forced photometry? Would it be impossible to perform the force photometry on all LSQ candidates BEFORE they pass human screening? This should help find good targets sooner.
one primary header (no data in the primary HDU, that is NAXIS=0)
one single extension with:
one header unit and
one BINTABLE with NAXIS=2.
Primary Header
Here are the mandatory primary header keywords included
SIMPLE = T / Standard FITS format (NOST-100-2.0)
BITPIX = %d / Number of bits per data pixel
NAXIS = 0 / Number of data axes
EXTEND = T / Extensions may be present
ORIGIN = 'ESO' / European Southern Observatory
DATE = %s / Date this file was written
TELESCOP= %s / ESO Telescope designation
INSTRUME= %s / Instrument name
DISPELEM= %s / Dispersive element name
SPECSYS = %s / Reference frame for spectral coordinates
OBJECT = %s / Target designation
EXT_OBJ = %c / TRUE if extended
RA = %f / [deg] Spectroscopic target position (J2000.0)
DEC = %f / [deg] Spectroscopic target position (J2000.0)
EQUINOX = %.0f / Standard FK5 (years)
RADECSYS= %s / Coordinate reference frame
EXPTIME = %f / Total integration time per pixel (s)
MJD-OBS = %.8f / [d] Start of observations (days)
MJD-END = %.8f / [d] End of observations (days)
PROG_ID = %20s / ESO programme identification
OBID1 = %d / Observation block ID
M_EPOCH = %d / TRUE if resulting from multiple epochs
PROV1 = %s / Originating science file
OBSTECH = %s / Technique of observation
PRODCATG= 'SCIENCE.SPECTRUM' / Data product category
FLUXCAL= %s / Type of flux calibration
CONTNORM= %c / TRUE if normalised to the continuum
WAVELMIN= %s / [nm] Minimum wavelength
WAVELMAX= %s / [nm] Maximum wavelength
SPEC_BIN= %f / Wavelength bin size
TOT_FLUX= %c / TRUE if photometric conditions and all src flux is captured
FLUXERR = %f / Uncertainty in flux scale (%)
REFERENC= %s / Bibliographic reference
SNR = %f / Average signal to noise ratio per pixel
SPEC_RES= %f / [nm] Reference spectral resolution (FWHM)
CHECKSUM= %s / HDU checksum
DATASUM = %s / Data unit checksum
END
Note: the format strings %c, %d, %f, and %s correspond to the data types boolean, integer number, floating point number, and character string, respectively.
Extension BINTABLE
The data arrays are stored as vectors in single cells. As a consequence, there shall be only one row in the BINTABLE, that is NAXIS2=1.
Extension Header
To ensure compliance with the VO standard, the keywords listed here are present in the extension header.
VOCLASS - The data model name and version, fixed to ‘SPECTRUM V1.0’.
VOPUB - The name of the publisher, fixed ‘ESO/SAF’.
TITLE - The value of the OBJECT keyword & short description of data included.
APERTURE - Aperture angular size, in degrees [deg]. It shall be set to the width of the slit or the diameter of the fiber.
TELAPSE - Total elapsed time in seconds [s], defined as MJD-END-MJD- OBS.
TMID - Exposure midpoint (MJD). It shall be set to (MJD-OBS+MJD- END)/2.0.
SPEC_VAL - Characteristic spectral coordinate value in nanometers [nm]. Should WCS information be available, it can be calculated as:
CRVAL1+(0.5-CRPIX1+NAXIS1*0.5)*CDELT1.
Otherwise, it is set to:
(WAVELMAX+WAVELMIN)/2.0.
SPEC_BW - Width of the spectrum in nanometers [nm]. Should WCS information be available, it can be calculated as:
NAXIS1*CDELT1.
Otherwise, it is set to:
WAVELMAX-WAVELMIN.
Each field of the BINTABLE is further described in the extension header as specified in Table 11 of the ESO Phase III SDP Document. Mandatory fields included are WAVE, FLUX, and ERR, in that particular order.
Here are the mandatory extension header keywords included:
XTENSION =BINTABLE / FITS Extension first keyword
BITPIX = 8 / Number of bits per data pixel
NAXIS = 2 / Number of data axes
NAXIS1 = %d / Length of data axis 1
NAXIS2 = %d / Length of data axis 2
VOCLASS = 'SPECTRUM V1.0' / VO Data Model
VOPUB = 'ESO/SAF' / VO Publishing Authority
TITLE = %s / Dataset title
OBJECT = %s / Target designation
RA = %f / [deg] Spectroscopic target position (J2000.0)
DEC = %f / [deg] Spectroscopic target position (J2000.0)
APERTURE= %f / [deg] Aperture diameter
TELAPSE = %f / [s] Total elapsed time
TMID = %f / [d] MJD mid exposure
SPEC_VAL= %f / [nm] Mean Wavelength
SPEC_BW = %f / [nm] Bandpass Width Wmax – Wmin
TFIELDS = %d / Number of fields in each row
NELEM = %d / Length of the data arrays
TTYPE1 = %s / Label for field 1
TUTYP1 = 'Spectrum.Data.SpectralAxis.Value'
TFORM1 = %s / Data format of field1
TUNIT1 = %s / Physical unit of field1
TUCD1 = %s / UCD of field 1
TDMIN1 = %f / Start in spectral coord.
TDMAX1 = %f / Stop in spectral coord.
TTYPE2 = 'FLUX' / Label for field 2
TUTYP2 = 'Spectrum.Data.FluxAxis.Value'
TFORM2 = %s / Data format of field 2
TUNIT2 = %s / Physical unit of field 2
TUCD2 = %s / UCD of field 2
TTYPE3 = 'ERR' / Label for field 3
TUTYP3 = 'Spectrum.Data.FluxAxis.Accuracy.StatError'
TFORM3 = %s / Data format of field 3
TUNIT3 = %s / Physical unit of field 3
TUCD3 = %s / UCD of field 3
EXTNAME = %s / FITS Extension name
INHERIT = T / Primary header keywords are inherited
CHECKSUM= %s / HDU checksum
DATASUM = %s / Data unit checksum
END
