KI V2 -- Description of Quality Assessment (QA) metrics

As part of the pipeline processing, quality assessment metrics are generated for the Level 1 V2 data. These metrics are available when accessing the data through V2 data retrieval page from the Level 1 database. This document explains how the metrics are calculated.

Quality Assessment approach

The general approach in the KI QA metrics is to calculate quanititative metrics for each data record and to measure the level of confidence in the data record by comparing its metrics to a body of previously collected data. This results in a set of statistically based assessment metrics and does not rely on set limits. For instance, each data record has a value for the jitter (the rms of the phase difference betwenn samples). Empirically we know that low jitter is good and high jitter is bad. In the QA metrics, this judgement is quantified to say that the jitter for this data record is 70th percentile of jitter measurements made with this instrument. In this way, the user can place the quality of the individual data measurement in the context of the instrument performance over a long time.

In using these metrics, is also important for the user to keep in mind the parameters of their particular source. The QA metrics for a source near the sensitivity limits of the interferometer will in general be lower than those for a much brighter source. This does not mean that the fainter source data is not valuable, but that the user should be more careful in evaluating the data during calibration and in assigning final errors.

The QA grade

In addition to calculating the statistical QA metrics, each piece of data is assigned a grade. This grade takes into account the QA metrics, but also other factors such as the availability of calibrator data and is specfically intended for use in editing data while calibrating. The grade algorithm is described here. The format is a 3 digit number (similar to the 2MASS format) where the value of each digit runs from 0 to 3 and the digits represent the grade and the record, scan and night levels. The values represent
  0 = good data with no known issues or problems
  1 = fair data which may have minor issues but will still calibrate normally
  2 = poor data (weather or instrument) that will need very careful calibration
  3 = data severely comprised by weather or instrument problems, should only
    be used with extreme caution
The grades are available as part of the .qa file and we are working to incorporate the grades as a selection criteria in the calibration programs.

QA metrics available

The QA metrics are calculated for data records at different levels. The three levels are the record (i.e. each line of the sum file), the scan (also called an integration) and the entire night. A record is typically 5 seconds of averaged data and a scan typically contains 25 records. At each level, a set of attributes (given below) is computed and percentiles generated through comparison to the previously collected body of data.

Record level attributes

Scan (integration) level attributes

Night level attributes

Available files

The QA data are divided into 3 files: Follow the links for each file to see the detailed descriptions.

These files are available for download from the KI Level 1 Data Access page.

Return to the KI Support page

last updated Nov 23, 2004