nbNullCalib v1.7 – Narrow-band Interferometric Nulling Data Calibration
nbNullCalib is a product of the NASA Exoplanet Science Institute http://nexsci.caltech.edu.
--- The Detailed Node Listing ---
General Description and Features
Algorithm, Internals, and Dependencies
Ancillary Scripts and Programs
nbNullCalib is a narrow-band interferometric nulling data calibration application: it correlates observations on one or more science targets with observations on one or more calibration sources, estimates a model of the system null (see Algorithm below) at the times of the target scan, applies this system null estimate to obtain the calibrated target null amplitude at those times, and optionally computes ancillary geometric information (i.e. u-v coordinates, delays, hour angles). nbNullCalib was developed for the KI mode by adapting the narrow-band visibility amplitude program nbCalib. It reads the interferometric data from one or more "nullspec" files (L-1 data). All nbNullCalib input comes in two (or more) input ASCII files; all data output is ASCII and goes to stdout (with error messages to stderr).
The canonical use case for nbNullCalib is:
nbNullCalib [options] xxx.calScript xxx.nullspec [yyy.nullspec...] [> xxx.calData]
where xxx.calScript is a script file that contains object designation and astrometry information in a standard format, and xxx.nullspec (and any number of additional input data files yyy.nullspec) is the standard L-1 narrow-band nulling reduced data product file. Options for nbNullCalib are summarized below in Arguments.
The reader unfamiliar with the use of terms like interferometric resolution is referred to any standard interferometric textbook such as Thompson, Moran, and Swenson, the NRAO Imaging Summer School Notes, or lecture notes from the Michelson Summer School. The nulling mode is described at http://nexsci.caltech.edu/software/KISupport/nulling/index.html
nbNullCalib takes no fewer than two inputs, a calibration script that defines the targets and calibrators of interest in this run, and one or more input data file specifications. Output from nbNullCalib is written to stdout (with major warnings and errors also written to stderr):
nbNullCalib xxx.calScript xxx.nullspec [yyy.nullspec...] [> xxx.calData]
Various command-line arguments that control the behavior of nbNullCalib are also available (see Arguments).
The ASCII nbNullCalib script file contains both science target and calibrator object designations, astrometric information, and calibrator diameter information:
# Simbad Search HD 9939: Type: High proper-motion Star K0IV V=6.99
HDC9939
01 37 25.107 +25 10 03.969 -0.211 -0.208 0.0238 # K0IV V = 7.0, K = 5.0
---
2
# Simbad Search HD 7034: Type: Star F0V V=5.160
## 7 HD7034--F0V 7306 +/- 0 1.79 11 25.82 +/- 0.66 0.52 +/- 0.01 XX
HDC7034 01 11 06.768 +31 25 29.051 -0.009 -0.012 0.00582 0.50 0.05 # F0V V = 5.2, K = 4.5
# Simbad Search HD 7964: Type: Star A3V V=4.752
## 9 HD7964--A3V 8686 +/- 0 9.18 20 47.44 +/- 1.85 0.50 +/- 0.01 XX XXXX
HDC7964 01 19 27.993 +27 15 50.611 0.026 -0.012 0.01049 0.47 0.05 # A3V V = 4.7, K = 4.6
Lines begining with the pound (hash – #) sign are comments and are ignored by nbNullCalib. The astrometry format is a standard J2000 position (hh mm ss dd mm ss), proper motion (arcsec/yr), plx (arcsec), ancillary info KI-standard format. Targets to be calibrated are designated in the first section of the file, in this example a single target designated HDC9939. On a separate line the astrometry for the target is given. Then separated by a line separator (—), the calibrator section follows. The number of calibrators to use is given, followed by single-line calibrator specifications, indicating the calibrator designation, astrometry (J2000 position, proper motion, and parallax; units are RA: hh mm ss.s of time, Dec: dd mm ss.s of arc, pm: arcsec/yr, parallax arcsec), estimated angular diameter and standard error in such (units of millarcseconds, mas). Take note of the calibrator angular diameter and error fields – they are the critical parameters that determine the scale of the calibrated null.
While the script file can be composed (or edited) by hand, a template calibration script file can be composed by the getCal experiment planning tool suite.
The xxx.nullspec file is the standard L-1 data product. It's format looks like:
NULLSPEC 01/17/2008/06:29:23 6.48972222222222 HD23413 4.341976 25.1000 917.5 78.1 367 250 IN+OUT M1.1 0 KI_K1K2 10 8.250
355.2 0.0577 0.0021 20.5 0.6 355.2 0.0 8.751 500.0 0.0466 0.0021 23.3 2.2 500.0 -0.0 9.227 367.0 0.0436 0.0
015 16.0 1.1 367.0 0.0 9.768 336.8 0.0382 0.0020 12.9 1.7 336.8 -0.0 10.240 323.9 0.0353 0.0015 11.4 -0.5 3
23.9 0.0 10.697 290.4 0.0277 0.0025 8.0 -0.8 290.4 0.0 11.205 211.0 0.0248 0.0029 5.2 -2.5 211.0 0.0 11.71
1 148.1 0.0247 0.0043 3.7 -1.1 148.1 0.0 12.208 100.6 0.0261 0.0071 2.6 -1.9 100.6 0.0 12.713 62.8 0.0291 0
.0097 1.8 -1.8 62.8 0.0
NULLSPEC 01/17/2008/07:16:22 7.27277777777778 HD36780 7.923718 25.1000 937.5 52.2 375 250 IN+OUT M1.1 0 KI_K1K2 10 8.250
62.8 0.0344 0.0088 2.2 -0.3 62.8 0.0 8.751 88.7 0.0559 0.0083 5.0 0.1 88.7 0.0 9.227 67.7 0.0663 0.0086 4.5
-0.9 67.7 0.0 9.768 62.7 0.0433 0.0125 2.7 -2.3 62.7 0.0 10.240 64.8 0.0576 0.0145 3.7 -2.8 64.8 0.0 10.6
97 58.1 0.0476 0.0202 2.8 -3.5 58.1 -0.0 11.205 42.1 0.0252 0.0222 1.1 -4.0 42.1 0.0 11.711 24.6 0.0162 0.0
370 0.4 -4.4 24.6 0.0 12.208 19.2 -0.0384 0.0673 -0.7 -4.2 19.2 -0.0 12.713 15.3 0.0716 0.1146 1.1 -3.4 15.
3 0.0
and the fields are:
Tag Date/time Time Source FDL LDL #Cycles Gated_cycles Duration Mode Ports L1_prog Calflag Baseline #Channels
then for each channel
Wavelength Flux Null_leakage Leakage_error rNN jNN rPP jPP
See http://nexsci.caltech.edu/software/KISupport/nulling/nullspec.shtml for the complete description.
nbNullCalib requires at least one input nullspec file, but will in fact process any (large) number of additional nullspec file specifications on the command line (e.g.):
nbNullCalib xxx.calScript xxx.nullspec yyy.nullspec zzz.nullspec aaa.nullspec ... [> xxx.calData]
This capability allows a single nbNullCalib invocation to calibrate large amounts of null data stored separately in individual nullspec files; you can calibrate nights individually, or calibrate 3 years worth of data.
Depending on the user choice of nbNullCalib verbosity (see Arguments), default output from nbNullCalib looks like:
(gnomad:8) nbNullCalib cs.hd195987 100180.spec
# Calibrating star HDC195987 at 20 32 51.642 +41 53 54.522
# with respect to 4 calibrators:
# star HDC195194 at 20 28 25.791 +39 19 44.250 size 0.67 error 0.1 2.7 degrees away from target
# star HDC200031 at 20 59 42.912 +38 49 22.504 size 0.6 error 0.1 6 degrees away from target
# star HDC177196 at 19 01 26.368 +46 56 05.325 size 0.7 error 0.06 17 degrees away from target
# star HDC185395 at 19 36 26.538 +50 13 15.970 size 0.84 error 0.04 13 degrees away from target
# Narrow-band Calibration code settings for this run:
# > Using calibration scans within +/- 1 hr of the target scan
# > Weighting calibration scans WRT temporal proximity, variance doubles in 1 hr
# > Weighting calibration scans WRT sky proximity, variance doubles in 15 deg
# > No minimum calibrated scan uncertainty
# > Flagging calibration warnings when multiple calibrators disagree by more than 2 sigma
# > Rejecting the scan when multiple calibrators disagree by more than 3 sigma
# 271 data lines read from 100180.spec
# Calibration data format:
# name #VisPts MJD m/d/y/h:m:s UT Delay(m) #Chan WL(um) CalVis Err RawVis Err SysVis
Err... #Cal U(m) V(m) HA(hr) BLName
# Target HD123:
HD123 1 54513.58664 1/1/2008/06:54:37 6.91028 5.638653 10 8.22 0.00620568 0.0750096
0.0032 0.209523 -0.00300568 0.0492588 8.734 0.00470086 0.0609294 0.002 0.196723
-0.00270086 0.0413811 9.21 0.00435168 0.0599826 0.0022 0.196214 -0.00215168 0.037388
7 9.745 0.00384338 0.0668301 0.0034 0.198746 -0.000443379 0.0326536 10.258 0.003809
07 0.0649471 0.0032 0.18303 -0.000609072 0.0319081 10.719 0.00178038 0.0675646 0.0036
0.169706 0.00181962 0.031064 11.188 0.00517434 0.0882376 0.0071 0.164621 0.001925
66 0.0261893 11.697 0.00506119 0.0980917 0.009 0.146629 0.00393881 0.0249395
12.187 0.00832369 0.120666 0.0137 0.173781 0.00537631 0.0293292 12.687 0.0145733
0.155265 0.0237 0.2502 0.0091267 0.0201821 2 37.286624 64.699424 1.6578 KI_K1K2
The first section of the output is an extended preamble (comment lines indicated by # symbols) containing the following pieces:
The second section of the file contains a list of calibrated output data. For each specified target/pseudonym there is a designation header, and then a sequence of tab-delimited data lines. nbNullCalib output data lines have a variable number of fields depending on the number of spectrometer channels in the input data, and whether target astrometry has been provided as part of the calibration script to support u-v calculations. The contents of these data lines is described by the following table:
| Field | Item | Description
|
| 1 | Object | Object Designation (string)
|
| 2 | NSpecRecords | The number of nullspec records averaged into this target scan (integer)
|
| 3 | MJD | Modified Julian Date (day) (float)
|
| 4 | UTC Date & Time | UTC Date/time in mm/dd/yy/hh:mm:ss format
|
| 5 | UTC | UTC time (decimal hours) (float)
|
| 6 | Delay | Average delay (m) (float)
|
| 7 | NChannels | The number of spectrometer channels (integer)
|
| 8 + 7*(i-1) | WL i | Channel i wavelength (microns) (float)
|
| 9 + 7*(i-1) | CalNull i | Channel i calibrated null (dimensionless) (float)
|
| 10 + 7*(i-1) | ErrNull i | Channel i std Error in calibrated null (dimensionless) (float)
|
| 11 + 7*(i-1) | RawNull i | Channel i Raw (L-1) narrow-band null (dimensionless) (float)
|
| 12 + 7*(i-1) | ErrRawNull 1 | Channel i std Error in raw narrow-band null (dimensionless) (float)
|
| 13 + 7*(i-1) | SysNull i | Channel i system null (dimensionless) (float)
|
| 14 + 7*(i-1) | ErrSysNull i | Channel i std Error in estimated system null (dimensionless)
|
| 15 + 7 * (NChannels-1) | NCal | Number of calibrators used in calibrating this scan (integer)
|
| 16 + 7 * (NChannels-1) | uCoord | u * lambda – RA baseline projection (m) (float)
|
| 17 + 7 * (NChannels-1) | vCoord | v * lambda – dec baseline projection (m) (float)
|
| 18 + 7 * (NChannels-1) | HRangle | Hour angle on target for this scan (decimal hours) (float)
|
| 19 + 7 * (NChannels-1) | Baseline | Baseline designation for this scan (e.g. KI_K1K2) (string)
|
where the channel index i runs from 1 to NChannels.
A nbNullCalib invocation with no arguments generates a short usage reminder.
A nbNullCalib invocation with argument -help (or -h) outputs the online help.
Additional information and discussion on nbNullCalib command-line options can be found in Arguments.
The fundamental purpose of nbNullCalib is to estimate the null depth on a target as measured by an idealized interferometer, N_target, based on the nulling data measurement made by the interferometer, N_meas. The nbNullCalib algorithm for doing this uses the system null, N_sys and is simply:
(2.1) N_meas = N_target + N_sys
so of course the idealized target null is estimated by inverting Eq. 2.1:
(2.2) N_target = N_meas - N_sys
Note that N_sys may be either positive or negative.
The job of nbNullCalib is to estimate the system null at the time of the target scan by analyzing available calibration measurements, and then to apply (in the sense of Eq 2.2) this system null model to the measured target null to estimate the idealized target null. Described more fully below, the system null estimation algorithms used by nbNullCalib are designed to characterize spatial and temporal variations in system performance under a variety of calibration circumstances.
The quality of the system null depends in part on knowledge of the size of the calibrator. Unresolved calibrators are ideal, but difficult to find in practice as the point of interferometers is high angular resolution. Since all astronomical sources are of finite surface temperature, depending on the part of sensitivity/spatial frequency space that one is working in, finding an unresolved source at any spatial proximity to your target may or may not be a straightforward exercise. (A related issue is the degree to which one can be certain that a given stellar source is even single; stellar multiplicity is a pervasive phenomenon.) Finite surface temperatures and spatial variations in interferometer and atmospheric conditions unfortunately can lead one to consider calibration sources that are partially resolved by the interferometer. Therefore, nbNullCalib's calibrator observation modeling can account for resolved calibrators with finite angular size (with the associated requirement of calculating baseline projection effects). The associated planning suite getCal getCal can assist the user in selecting potential calibrators, estimating their angular sizes, retrieving available ancillary information, and preparing nbNullCalib input. In the event that calibrators in practice turn out to be less than ideal, nbNullCalib supports the use (and more to the point, the intercomparison of data from) multiple calibrators to refine the system null estimate and facilitate the identification of troublesome calibrators.
Finally, users find it useful for their calibration application to perform geometric projection (in the parlance, u-v coordinate) calculations. As these geometrical calculations are required to properly account for resolved calibrators, it is straightforward to have nbNullCalib compute the measurement u-v coordinates if the target astrometry is provided (see Astrometric_Processing).
nbNullCalib has four main algorithmic components: Data_Management and averaging, System_Null_Estimation and application, Error_Estimation, and Astrometric_Processing. Each of these is discussed in turn.
The majority of the nbNullCalib code is devoted to getting the right null measurements to the right place at the right time. This is straightforwardly and tediously implemented in nbNullCalib by creating and traversing a data base of null measurements contained in the input nullspec file set. nbNullCalib implements this as a doubly-linked list data structure. It consists of a list over the object designations used in the measurement process. Hanging from the nodes of this first list is a time-sorted list of null measurements on the particular object designation. All of the memory allocation for the database is done dynamically as the structure is constructed in the course of reading the data files specified on the command line. Consequently, there is no pre-set size limit as to the maximum amount of data nbNullCalib can process.
The L-1 data product coming into nbNullCalib is the average null depth per channel over an entire integration. This is illustrated by the following segment from a typical L-1 nullspec file (this data format was discussed above):
NULLSUM 01/01/2008/06:29:23 6.48972 HD23413 5.967427 25.100 937.5 52.2 375 250 IN+OUT M1.1 0 KI_K1K2 10 8.250 62.8 0.0344 0.0088 2.2 -0.3 62.8 0.0 8.751 88.7 0.0559 0.0083 5.0 0.1
88.7 0.0 9.227 67.7 0.0663 0.0086 4.5 -0.9 67.7 0.0 9.768 62.7 0.0433 0.0125 2.7 -2.3 62.7 0.0 10.240 64.8 0.0576 0.0145 3.7 -2.8 64.8 0.0 10.697 58.1 0.0476 0.0202 2.8 -3.5 58
.1 -0.0 11.205 42.1 0.0252 0.0222 1.1 -4.0 42.1 0.0 11.711 24.6 0.0162 0.0370 0.4 -4.4 24.6 0.0 12.208 19.2 -0.0384 0.0673 -0.7 -4.2 19.2 -0.0 12.713 15.3 0.0716 0.1146 1.1 -3.
4 15.3 0.0
NULLSUM 01/01/2008/06:54:37 6.91023 target1 4.962451 25.100 1007.5 62.5 403 250 IN+OUT M1.1 0 KI_K1K2 10 8.250 56.1 0.0917 0.0083 5.1 1.7 56.1 0.0 8.751 81.9 0.0526 0.0077 4.3 0.1
81.9 0.0 9.227 62.4 0.0451 0.0119 2.8 -0.7 62.4 -0.0 9.768 55.1 0.0245 0.0184 1.4 0.0 55.1 -0.0 10.240 58.0 0.0332 0.0094 1.9 0.9 58.0 0.0 10.697 53.2 0.0196 0.0092 1.0 0.2 53.
2 -0.0 11.205 38.3 0.0193 0.0223 0.7 -0.4 38.3 0.0 11.711 26.9 0.0099 0.0196 0.3 -1.0 26.9 -0.0 12.208 20.3 -0.0268 0.0332 -0.5 -0.5 20.3 0.0 12.713 15.4 0.0400 0.0423 0.6 -0.7
15.4 -0.0
NULLSUM 01/01/2008/07:16:22 7.27277 HD36780 2.874087 25.100 1517.5 88.1 607 250 IN+OUT M1.1 0 KI_K1K2 10 8.250 187.0 0.0725 0.0025 13.6 -1.0 187.0 0.0 8.751 244.2 0.0578 0.0021 14
.1 0.2 244.2 0.0 9.227 165.4 0.0438 0.0017 7.2 0.3 165.4 -0.0 9.768 159.3 0.0339 0.0019 5.4 0.7 159.3 0.0 10.240 153.5 0.0272 0.0027 4.2 0.2 153.5 0.0 10.697 133.0 0.0200 0.003
2 2.7 1.0 133.0 0.0 11.205 95.5 0.0095 0.0036 0.9 0.2 95.5 0.0 11.711 62.9 -0.0003 0.0072 -0.0 0.7 62.9 0.0 12.208 40.7 -0.0136 0.0121 -0.6 0.5 40.7 0.0 12.713 28.1 -0.0069 0.0
220 -0.2 1.3 28.1 0.0
NULLSUM 01/01/2008/07:36:15 7.60416 target1 6.296732 25.100 1200 74.4 480 250 IN+OUT M1.1 0 KI_K1K2 10 8.250 63.8 0.0361 0.0068 2.3 -1.0 63.8 0.0 8.751 93.6 0.0474 0.0069 4.4 -0.9
93.6 0.0 9.227 69.6 0.0439 0.0044 3.1 -0.8 69.6 0.0 9.768 61.4 0.0486 0.0112 3.0 -1.7 61.4 -0.0 10.240 64.3 0.0518 0.0122 3.3 -2.5 64.3 0.0 10.697 58.6 0.0563 0.0081 3.3 -2.0 5
8.6 -0.0 11.205 43.3 0.0667 0.0140 2.9 -2.5 43.3 0.0 11.711 32.8 0.0636 0.0171 2.1 -2.5 32.8 0.0 12.208 22.3 0.1285 0.0338 2.9 -2.3 22.3 0.0 12.713 17.8 0.0481 0.0306 0.9 -2.1
17.8 0.0
NULLSUM 01/01/2008/07:56:59 7.9497 HD36780 3.862087 25.100 2310 30.5 924 250 IN+OUT M1.1 0 KI_K1K2 10 8.250 285.3 0.0186 0.0020 5.3 -0.4 285.3 0.0 8.751 390.9 0.0162 0.0020 6.3 0
.0 390.9 0.0 9.227 279.0 0.0135 0.0026 3.8 0.4 279.0 -0.0 9.768 256.4 0.0107 0.0040 2.7 -0.3 256.4 0.0 10.240 264.6 0.0129 0.0032 3.4 0.9 264.6 -0.0 10.697 230.1 0.0123 0.0024
2.8 1.7 230.1 0.0 11.205 163.8 0.0105 0.0049 1.7 2.5 163.8 -0.0 11.711 109.1 0.0145 0.0074 1.6 2.3 109.1 0.0 12.208 72.8 -0.0025 0.0094 -0.2 1.9 72.8 0.0 12.713 44.8 0.0031 0.0
240 0.1 1.0 44.8 0.0
This example contains two integrations of the science target (target1) bracketed by the calibrators HD23413 and HD36780. Unlike the visiblity amplitude calibration (nbCalib), nbNullCalib does no averaging of the input data. There will be one output line for every input data line on the target.
As described above in Algorithm, the system null estimate, N_sys, is the point source response of an imperfect interferometer in an imperfect environment. N_sys can be a function of both time and sky location, so the problem at hand in calibrating a given target scan is estimating N_sys at the time of the target scan and sky location of the target. Once that estimate has been made, it is simple to apply Eq. 2.2 to estimate the ideal target nulling data. We shall describe the algorithm used by nbNullCalib to make this estimate in a series of steps.
First consider an individual scan at time t_i on an unresolved calibrator (one for whom the expected nulling data N_cal is zero). After computing a scan average, the estimation of N_sys at time t_i and the sky location of the calibrator follows straightforwardly from inverting Eq. 2.1:
(2.3) N_sys = N_meas - N_cal => N_meas
However, if this calibrator has the potential to be resolved by the interferometer, N_cal is no longer necessarily zero. nbNullCalib treats potentially resolved calibrators as uniformly bright disk sources of a given (in fact user-specified) angular diameter theta_cal. N_cal for the uniform disk of diameter theta_cal is computed using the Bessel function to compute the visibility:
V_cal = 2 J_1(pi * theta_cal * B_proj / lambda ) / (pi * theta_cal * B_proj / lambda)
and then calculating the null depth
(2.4) N_cal = (1 - V_cal)/(1 + V_cal)
With B_proj the projected baseline on the calibrator object (see Astrometric_Processing), lambda the operating wavelength, and J_1 the first order Bessel function of the first kind.
As our fundamental goal is to estimate (and apply) the system null at the time t_target and sky location of a given target scan, we can use our knowledge of N_sys at t_i as the basis for that estimate. In general, the system null N_sys evolves with time and sky position; we are left with the problem of formulating an estimator for N_sys at t_target and sky location of the target given some set of calibrator observations taken at t_i. Speaking momentarily of just a single calibrator, it is simple to imagine a weighted averaging scheme that would take a set of calibrator observations at times t_i and estimate the system null at time t_target:
(2.5) N_sys(t_target) = Sum(N_sys(t_i) w_i) / Sum(w_i)
but this begs the question of what are the weights w_i? A simple weighted averaging scheme (derived by a minimum variance optimality criterion) uses weights as the inverse variance of the quantity estimate:
(2.6) w_i = [sigma^2_N_sys(t_i)]^-1
The quantity sigma^2_N_sys(t_i) is calculated from the uncertainty given in the nullspec file for the calibrator scan at t_i. A second, related question is to ask what about Eq. 2.5 makes it specific to time t_target? There are two possible answers to that question – both related to the postulate that the instrumental and environmental conditions tend to be more correlated on small timescales than on long timescales. The first answer is to sample calibrator scans over a time interval symmetric around the science target scan time – that way any temporal variability in the system null (N_sys(t)) is at least sampled in an approximately unbiased way, and a weighted averaging process (Eq. 2.5) estimating the system visibility will similarly be unbiased. nbNullCalib always constructs a symmetric time window around the science scan time in order to select calibrator scans. The user can control the size of this time window; the default value is one hour.
The second variant on the weighted averaging scheme used by nbNullCalib is to manipulate the weights (Eq. 2.6) used in the averaging process so as to more heavily weight calibrator scans temporarily near the science scan time. For example, doubling the error variance in one hour works well; this is easily accomplished with a slight modification to Eq. 2.6:
w_i = [sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2)]^-1
Here T_C is the by default the one hour scan time window, but it is under the control of the the user, so nbNullCalib can supply a continuum of weighting schemes (including a nearly flat weighting scheme where T_C is large). This time weighting scheme can be turned off with the -notimeWeighting argument (see Arguments).
As well as being a function of time, the system null can be a function of sky location. Thus, one is motivated to choose calibration sources that are close to the science target; this is the primary operating model for getCal. Like the temporal argument, nbNullCalib provides a variance weighting scheme to emphasize spatially-near calibrators over those more distant:
(2.7) w_i = [sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C)]^-1
By default, Ang_C is set to 15 degrees, but like T_C, Ang_C is controlled by the user, so a nearly flat weighting scheme is again possible with Ang_C large. As before, this angle weighting scheme can be turned off with the -noangleProxWeighting switch (see Arguments).
Error estimation in nbNullCalib follows from standard error propagation calculations. From Eq. 2.2 the estimated calibrated scan variance is given by:
(2.8) sigma^2_N_target = sqrt(sigma^2_N_meas + sigma^2_N_cal)
The target null measurement variance is simply the square of the uncertainty taken from the nullspec file. The system null variance is compute by the canonical weighted average recipe adopted in Eq. 2.5 (with weights as computed in Eq. 2.7):
(2.9) sigma^2_N_sys = [Sum(1 / w_i)]^-1
= [Sum([sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C)]^-1)]^-1
The estimated calibrator null calculations for nbNullCalib (Eq. 2.4) call for the projection of the interferometer baseline on the calibrator direction unit vector s_hat:
(2.10) B_proj = B \dot s_hat
where B is the instantaneous baseline 3-vector. nbNullCalib uses the Navy NOVAS astrometric subroutine package to evaluate target astrometry (see Dependencies), and it is convenient to evaluate Eq. 2.10 in true equitorial intertial coordinates. That implies that we have to evaluate B – fixed to the rotating earth. nbNullCalib uses the NExScI baseline class library to evaluate the instantaneous baseline vector (see Dependencies).
Further, nbNullCalib users find it useful to have projected interferometer baseline coordinates (u-v coordinates times the operating wavelength) on the calibration target packaged with along the calibrated target scan – this feature is more a desirement rather than a requirement. So (in instances where users provide target astrometry in their calibration script files – in all the examples provided herein) nbNullCalib computes u-v coordinates for calibrated target scans. This calculation is straightforward:
(2.11) u = B \dot ra_hat
v = B \dot dec_hat
with ra_hat and dec_hat as unit vectors in the instantaneous directions of increasing right ascention and declination at the target location respectively:
ra_hat = (z_hat \cross s_hat) / | z_hat \cross s_hat |
dec_hat = s_hat \cross ra_hat
and z_hat a unit vector in the direction of the Earth's angular momentum vector (e.g. (0,0,1) in equitorial intertial coordinates).
nbNullCalib uses (requires) the libraries below for its build. These libraries are packaged and distributed with nbNullCalib.
The command line arguments to nbNullCalib are:
-help:
print a short help message and exit. No argument, overrides all
other options when used.
-calTimeWindow argument:
set the calibration scan acceptance time window size (hr). Default
value is 1 hr. Argument required when used.
-[no]timeWeighting:
set the [non] use of time proximity weighting of calibration scans.
Default is to use time weighting. No argument when used.
-timeErrorVal argument:
if timeWeighting, set calibration scan time weighting value (hr), the
time over which the calibrator null measurement variance is
assumed to double. Default value is 1 hr. Argument required when used.
-[no]angleProxWeighting:
set the [non] use of sky proximity weighting of calibration scans.
Default is to use sky proximity weighting. No argument when
used.
-angleErrorVal argument:
if angleProxWeighting, set target/calibrator sky separation weighting
value (deg), the angle over which the calibrator null measurement
variance is assumed to double. Default value is 15 deg. Argument
required when used.
-[no]minScanUncertainty argument:
sets the [non] use of a minimum floor to the calibrated null
uncertainty. Default is to have no artificial minimum
uncertainty. Argument required when used to set, no argument required
when not using minimum.
-chiWarning argument:
set multiple calibrator system null inconsistency threshold for
triggering a warning (sigmas – dimensionless). Default is to report
system null estimate inconsistencies at or greater than the two
(2) sigma level (assuming that reporting is on, see verbosity options
below). Argument required when used.
-chiReject argument:
set multiple calibrator system null inconsistency threshold for
rejecting a target scan (sigmas – dimensionless). Default is to reject
the science scan if multiple system null estimates are
inconsistent at or greater than the three (3) sigma level. Argument
required when used.
-delayCheck[=argument]:
check delay values for targets and calibrators, and report warnings
when data delay value is not in agreement with model calculation.
Default is to check delay values. Optional floating point argument
specifies delay check tolerance (units of meters) – default value is
0.1 m (approximately 3 arcmin on a 100 m baseline).
-nodelayCheck:
do not check delay values for targets and calibrators. Default is to
check delay values.
-doCalibrators:
causes the calibrator observations to also be calibrated.
-baseline <file>: Use the baseline supplied in <file>. Normal
behavior is to look for a yyyyddd.nullbaseline file for each yyyyddd.nullspec
file given on the command line. If that fails, nbNullCalib looks for
yyyyddd.nullbline. If neither is found, nbNullCalib uses a built-in default
baseline. By using the -baseline option, the users can override
this search behavior by providing the baseline or bline filename directly.
Note that when multiple nullspec files are given in combination with this option, all of them will use the baseline supplied in <file>.
-quiet:
set nbNullCalib to suppress all informational messages and all but the most
serious warnings and errors. No argument when used.
-shutUp:
the same as quiet.
-normal:
set nbNullCalib to report nominal status, warnings, and errors. This is the
default. No argument when used.
-verbose:
set nbNullCalib to excessively report on status, warnings, and errors.
(This should be used when the user runs into difficulty with nbNullCalib
operation.) No argument when used.
note: nbNullCalib command-line options may be used in any order, interspersed with non-options (filename arguments) and abbreviated to uniqueness (e.g. -h, -verb, -chiW).
nbNullCalib uses the GNU autotools configuration system for configuration, build, and install. All necessary libraries for operation are included and build automatically.
> ./configure
> make
> make check (recommended)
> make install
If configure finds gnuplot in your PATH it will automatically build NullerPlot (package to create standard set of plots from KI nulling data). To stop configure from building and installing NullerPlot, pass it the –enable-nullerplot=no option:
> ./configure --enable-nullerplot=no
To avoid putting software from NExScI
into standard /usr/local/bin
(among other things this requires root access),
the default prefix can be overridden with a
prefix option (/usr/myLocal in the following example) to the
nbNullCalib configure script, as in:
> ./configure --prefix=/usr/myLocal
> make
> make check (recommended)
> make install
Either way, nbNullCalib executables are placed in $prefix/bin, man
pages in $prefix/man, info files in $prefix/info.
Ancillary build targets are (courtesy of GNU autotools):
> make check (automated test)
> make doc
> make clean
> make dist
> make distclean
Finally, several forms of the documentation are available. The default part of the build procedure should make the info format documentation (nbNullCalib.info), but the following additional documentation types (targets) are also available:
> cd doc
> make html
> make ps
> make pdf
nbNullCalib is distributed with the input data for testing (in the test subdirectory). These tests are designed to be run with the the make check target as part of the installation sequence (see Build). The test scripts run nbNullCalib against a pre-packaged calibration script and L-1 data in the test directory and then run the output through diff to check consistency with reference output.
What if "make check" fails?
We have noted two common types of installation problem. In the first, "make check" fails due to round-off differences between the packaging platform and your installation platform. This is usually the case when diff identified only 1 or 2 lines of non-matching output for each test. You may proceed with installation ("make install") and use.
With the second problem, "make check" fails and diff seems to identify all lines from the reference test output files. The likely cause is that the runtime linker could not resolve library dependencies for the executable. We have seen this behavior with gcc3 on Solaris; One of the missing libraries is libstdc++.so. The root cause lies with the installation configuration of your gcc3.
You can check for this linking error on Solaris by typing
> ldd <path/to/your/nbNullCalib>/Calib/nbNullEngine
If you see one or more libraries missing you have two options: 1) set LD_RUN_PATH then "make clean" and "make" again, or 2) set your LD_LIBRARY_PATH when executing
In order for either of these options to work, you will need the path to gcc3 libraries on your system. It some cases the gcc3 libraries and binaries will be installed subdirectories of a gcc3 package location (for example /usr/local/gcc3 might be the package location, with binaries found under /usr/local/gcc3/bin and libraries under /usr/local/gcc3/lib). In other cases binaries and libraries may be installed in generic locations (e.g. /opt/bin and /opt/lib). If you cannot find your gcc3 libraries, contact your local system administrator for help.
We recommend option 1) because it can be done once at installation and no further environment settings are required. If you choose option 2) you must set your LD_LIBRARY_PATH each time you run nbNullCalib or nullCalib (or you may set it in your .cshrc).
Assuming that you have found your gcc3 library path, option 1) proceeds as
> setenv LD_RUN_PATH <your/gcc3/libraries> # for c-shell, OR
> LD_RUN_PATH="/usr/local/gcc3/lib"; export LD_RUN_PATH # for bash
> make clean
> make
> make check
> make install
Option 2) is
> setenv LD_LIBRARY_PATH <your/gcc3/libraries>:${$LD_LIBRARY_PATH} # for c-shell, OR
> LD_LIBRARY_PATH="<your/gcc3/libraries>:$LD_LIRBRARY_PATH"; export LD_LIBRARY_PATH # for bash
Here we provide a list of common nbNullCalib error messages, their interpretation, and suggestions about their correction.
There are a variety of parsing errors that can occur when trying to compose a calibration script. Most of the work can be done with getCal. Use this feature if you can – it'll save you a lot of headaches.
Most of the parsing errors give some sort of diagnostic message, like:
Fatal Error -- Can't open calibration script file bogus
or
Fatal Error -- premature eof encountered in bo
If you're having parsing problems compare your cal script against the example given in Inputs_and_Outputs.
An uncommon error is reporting of delay mismatches in the data:
Warning: at 54234.654, 10.2 m measured/expected delay mismatch on target HDC9939
This has the potential of being serious, because it usually means there's some problem with the baseline model that's being applied, and at a minimum you're guaranteed that the u-v projections that nbNullCalib is calculating will be wrong. Less frequently it is also possible that there is some intrinsic problem with the data, such as incorrect delay line positions or incorrect coordinates in the calibration script.
The first thing to do is to re-run nbNullCalib with the -verbose switch turned on. This will enable all the baseline informational messages, and you can check to see whether there are any irregularities in the baseline information:
## Error detected in /home/bode/pti/data/100/100335.bline; reading baseline model from /home/bode/pti/data/100/100335.baseline (KI_K1K2)
The obvious thing to do in this circumstance is to make sure that the baseline model that gets assigned (e.g. KI_K1K2) is what you think it should be. If not, you should examine the measurement database.
When using multiple calibrators, nbNullCalib checks for system null estimate consistency among the calibrators. Consequently, probably the most common type of warning and error messages you'll see from nbNullCalib will be of the flavor:
# Warning: at 51724.1534 sysNull estimates between calibrators HDC121107 and HDC128167 are inconsistent at the 2.7 sigma level
and
# Reject: at 51724.1680 system null estimates between calibrators HDC121107 and HDC128167 are inconsistent at the 4.5 sigma level
# No suitable calibration found for 51724.1680
As discussed in Arguments, the user can control the thresholds at which these warnings and errors are reported. However, it's not an accident that we put this feature in nbNullCalib, and strong inconsistencies between calibrators can be a sign of either problems with the data on the calibrators or their sizes. Old fashion detective work is about the only general advice we can provide here.
nbNullCalib comes with several "helper" applications that make it a bit more civilized. These can be built at the users option (see Build instructions above).
badScans is a Perl script that allows the user to select certain scans for exclusion in subsequent processing and analyses. badScans is a very general filter for text files: it scans a stdin input text file, and looks for lines that contain one of an input list of exclusion keys. When it finds a line that contains an exclusion key, it prepends the line with a comment character (pound/hash sign – #). The last output from badScans lists the number of exclusions found.
The usage of badScans is:
> badScans xxx.badScans < input.file > output.file
or for instance in the context of a nbNullCalib invocation:
> nbNullCalib [options] cal.myTarg data | badScans myTarg.badScans > myTarg.cal
makeCalScript is a Perl script that allows the user compose a calibration script given a target name and a schedule file.
The usage of makeCalScript is:
> makeCalScript target catfile > outfile
Please note that the NExScI's getCal package also contains a utility for automatic generation of calibration scripts (csAdhoc). Please refer to the getCal documentation (http://nexsci.caltech.edu/software/getCal/index.html#gcCalScript).
NullerPlot is a Perl package which generates a standard set of plots for KI data. NullerPlot will be automatically built and installed if gnuplot is found in your PATH during configuration.
The usage of NullerPlot is:
> NullerPlot -help
usage: NullerPlot filename
options: -all (creates series of plots from sumfile)
-wavelength specfile (plots spectrometer values by wavelength)
-gif produces gif output files
-png produces png output files
-hc produces postscript output files
-size x,y set output image size in pixels [default: 640,480 *requires gnuplot 4+])
-scale r (set output image scale [default: 1 for 640x480, use 1.25 for 800x600])
-font tiny|small|medium|large|giant (set font size [default: medium])
-pointsize r (set point size [default: 1.0])
-spec (creates series of plots from specfile)
-anc ancfile (creates ancilliary plots, used only with all)
-bt btfile (creates block trending plots, used only with all)
-stdin (takes input data from stdin)
-start (set start time in hh.nnn or hh:mm:ss for star plots)
-duration (set duration in seconds for star plots)
-timeseries file (creates time series plots from timeseries.csv)
-xtics n (override number tics on x axis for star plots)
-version (prints version info)
-help (prints this message)
nbNullCalib v1.7
Copyright 2008 California Institute of Technology. For questions or comments about this software, please contact http://nexsci.caltech.edu/contact/help_desk.html. Permission to use, copy and distribute this software and its documentation for academic and/or non-profit research purposes, without fee and without a written agreement is hereby granted, provided that the above copyright notice, this paragraph and the following three paragraphs appear in all copies. Reuse of all or part of the software contained here for commercial purposes is strictly prohibited.
The California Institute of Technology makes no proprietary claims to the results, prototypes, or systems supporting and/or necessary for the use of the research, results and/or prototypes for academic and/or non-profit research uses only. However, to the extent that any software and system built in collaboration with industry partners may incorporate proprietary designs of the industry partners, it is possible that certain restrictions may be imposed on the proprietary information.
In no event shall California Institute of Technology be liable to any party for direct, indirect, special, incidental or consequential damages, including lost profits, arising out of the use of this software and its documentation, even if the California Institute of Technology has been advised of the possibility of such damage.
The California Institute of Technology specifically disclaims any warranties, including the implied warranties or merchantability and fitness for a particular purpose. The software and documentation provided hereunder is on an "as is" basis, and the California Institute of Technology has no obligations to provide maintenance, support, updates, enhancements or modifications.