KI V2 -- Description of Quality Assessment (QA) metrics
As part of the pipeline processing, quality assessment metrics
are generated for the Level 1 V2 data. These metrics are
available when accessing the data through V2 data retrieval
page from the
Level 1 database.
This document explains how the metrics are calculated.
Quality Assessment approach
The general approach in the KI QA metrics is to calculate
quanititative metrics for each data record and to measure the level of
confidence in the data record by comparing its metrics to a body of
previously collected data. This results in a set of statistically
based assessment metrics and does not rely on set limits. For
instance, each data record has a value for the jitter (the rms of the
phase difference betwenn samples). Empirically we know that low
jitter is good and high jitter is bad. In the QA metrics, this
judgement is quantified to say that the jitter for this data record
is 70th percentile of jitter measurements made with this instrument.
In this way, the user can place the quality of the individual data
measurement in the context of the instrument performance over a long
time.
In using these metrics, is also important for the user to keep in
mind the parameters of their particular source. The QA metrics
for a source near the sensitivity limits of the interferometer will
in general be lower than those for a much brighter source. This
does not mean that the fainter source data is not valuable, but
that the user should be more careful in evaluating the data
during calibration and in assigning final errors.
The QA grade
In addition to calculating the statistical QA metrics, each
piece of data is assigned a grade. This grade takes into
account the QA metrics, but also other factors such as
the availability of calibrator data and is specfically
intended for use in editing data while calibrating.
The grade algorithm is described here.
The format is a 3 digit number (similar to the 2MASS format)
where the value of each digit runs from 0 to 3 and
the digits represent the grade and the record, scan and
night levels. The values represent
0 = good data with no known issues or problems
1 = fair data which may have minor issues but will still calibrate normally
2 = poor data (weather or instrument) that will need very careful calibration
3 = data severely comprised by weather or instrument problems, should only
be used with extreme caution
The grades are available as part of the .qa file and
we are working to incorporate the grades as a selection criteria in
the calibration programs.
QA metrics available
The QA metrics are calculated for data records at different levels.
The three levels are the record (i.e. each line of the sum file), the scan (also called an
integration) and the entire night. A record is typically 5 seconds of
averaged data and a scan typically contains 25 records. At each
level, a set of attributes (given below) is computed and percentiles
generated through comparison to the previously collected body of data.
Record level attributes
- V2 (if calibrator)
- Wide-band photon count
- Jitter
- Ratio correction
- Lock breaks
Scan (integration) level attributes
- Mean V2 (if calibrator)
- Fractional V2 scatter about the mean
- Mean wide-band photon count
- Mean jitter
- Mean ratio
- Mean NV2 (photon count times V2)
- Mean lock breaks
Night level attributes
- Mean system visibility
- System visibility scatter
Available files
The QA data are divided into 3 files:
- .qa - The QA summary for each data line which includes the record, integration and nightly metrics.
- .qadet - The values and percentiles for each attribute.
- .qainfo - QA processing information
Follow the links for each file to see the detailed descriptions.
These files are available for download from the KI Level 1 Data Access
page.
Return to the KI Support page
last updated Nov 23, 2004