nullCalib v1.7 – Wide-band Interferometric Nulling Data Calibration
nullCalib is a product of the NASA Exoplanet Science Institute http://nexsci.caltech.edu.
--- The Detailed Node Listing ---
General Description and Features
Algorithm, Internals, and Dependencies
Ancillary Scripts and Programs
NullCalib is a wide-band interferometric nulling data calibration application: it correlates observations on one or more science targets with observations on one or more calibration sources, estimates a model of the system null (see Algorithm below) at the times of the target scan, applies this system null estimate to obtain the calibrated target null amplitude at those times, and optionally computes ancillary geometric information (i.e. u-v coordinates, delays, hour angles). nullCalib was developed for the KI mode by adapting the visibility amplitude program wbCalib. It reads the interferometric data from one or more "nullsum" files (L-1 data). All nullCalib input comes in two (or more) input ASCII files; all data output is ASCII and goes to stdout (with error messages to stderr).
The canonical use case for nullCalib is:
nullCalib [options] xxx.calScript xxx.nullsum [yyy.nullsum...] [> xxx.calData]
where xxx.calScript is a script file that contains object designation and astrometry information in a standard format, and xxx.nullsum (and any number of additional input data files yyy.nullsum) is the standard L-1 nulling reduced data product file. Options for nullCalib are summarized below in Arguments.
The reader unfamiliar with the use of terms like interferometric resolution is referred to any standard interferometric textbook such as Thompson, Moran, and Swenson, the NRAO Imaging Summer School Notes, or lecture notes from the Michelson Summer School. The nulling mode is described at http://nexsci.caltech.edu/software/KISupport/nulling/index.html
nullCalib takes no fewer than two inputs, a calibration script that defines the targets and calibrators of interest in this run, and one or more input data file specifications. Output from nullCalib is written to stdout (with major warnings and errors also written to stderr):
nullCalib xxx.calScript xxx.nullsum [yyy.nullsum...] [zzz.fits] [> xxx.calData]
Various command-line arguments that control the behavior of nullCalib are also available (see Arguments).
The ASCII nullCalib script file contains both science target and calibrator object designations, astrometric information, and calibrator diameter information:
# Simbad Search HD 9939: Type: High proper-motion Star K0IV V=6.99
HDC9939
01 37 25.107 +25 10 03.969 -0.211 -0.208 0.0238 # K0IV V = 7.0, K = 5.0
---
2
# Simbad Search HD 7034: Type: Star F0V V=5.160
## 7 HD7034--F0V 7306 +/- 0 1.79 11 25.82 +/- 0.66 0.52 +/- 0.01 XX
HDC7034 01 11 06.768 +31 25 29.051 -0.009 -0.012 0.00582 0.50 0.05 # F0V V = 5.2, K = 4.5
# Simbad Search HD 7964: Type: Star A3V V=4.752
## 9 HD7964--A3V 8686 +/- 0 9.18 20 47.44 +/- 1.85 0.50 +/- 0.01 XX XXXX
HDC7964 01 19 27.993 +27 15 50.611 0.026 -0.012 0.01049 0.47 0.05 # A3V V = 4.7, K = 4.6
Lines begining with the pound (hash - #) sign are comments and are ignored by nullCalib. The astrometry format is a standard J2000 position (hh mm ss dd mm ss), proper motion (arcsec/yr), plx (arcsec), ancillary info KI-standard format. Targets to be calibrated are designated in the first section of the file, in this example a single target designated HDC9939. On a separate line the astrometry for the target is given. Then separated by a line separator (–), the calibrator section follows. The number of calibrators to use is given, followed by single-line calibrator specifications, indicating the calibrator designation, astrometry (J2000 position, proper motion, and parallax; units are RA: hh mm ss.s of time, Dec: dd mm ss.s of arc, pm: arcsec/yr, parallax arcsec), estimated angular diameter and standard error in such (units of millarcseconds, mas). Take note of the calibrator angular diameter and error fields - they are the critical parameters that determine the scale of the calibrated null.
While the script file can be composed (or edited) by hand, a template calibration script file can be composed by the getCal experiment planning tool suite.
The xxx.nullsum file is the standard L-1 data product. It's format looks like:
NULLSUM 01/17/2008/06:29:23 6.48972222222222 HD23413 4.341976 25.1000 555.6 0.0340 0.0010 25792 11680 11708 18.9 -6.8 8.495 1532.5 93.7 613 250 IN+OUT M1.1 0 KI_K1K2
NULLSUM 01/17/2008/07:16:22 7.27277777777778 HD36780 7.923718 25.1000 540.1 0.0375 0.0014 23131 10186 10183 20.3 -9.3 8.495 1537.5 85.4 615 250 IN+OUT M1.1 0 KI_K1K2
and the fields are:
Tag Date/time Time Source FDL LDL Flux Null_leakage Leakage_error SNR2_XC SNR2_pri SNR2_sec rNN jNN Wavelength #Cycles Gated_cycles Duration Mode Ports L1_prog Calflag Baseline
See http://nexsci.caltech.edu/software/KISupport/nulling/nullsum.shtml for the complete description. nullCalib requires at least one input nullsum file, but will in fact process any (large) number of additional nullsum file specifications on the command line (e.g.):
nullCalib xxx.calScript xxx.nullsum yyy.nullsum zzz.nullsum aaa.nullsum ... [> xxx.calData]
This capability allows a single nullCalib invocation to calibrate large amounts of null data stored separately in individual nullsum files; you can calibrate nights individually, or calibrate 3 years worth of data.
Depending on the user choice of nullCalib verbosity (see Arguments), default output from nullCalib looks like:
(gnomad:27) nullCalib cs.hd123 100343.nullsum
# >> Calibrating star HD123 at 01 37 25.107 +25 10 03.969
# >> with respect to 2 calibrators:
# Simbad Search HD 7034: Type: Star F0V V=5.160
## 7 HD7034--F0V 9021 +/- 232 0.23 11 27.97 +/- 0.40 0.36 +/- 0.02 XX
## 7 HD7034--F0V 7306 +/- 0 1.79 11 25.82 +/- 0.66 0.52 +/- 0.01 XX
# >> star HDC7034 at 01 11 06.768 +31 25 29.051 size 0.5 error 0.05 8.5 degrees away from target
# Simbad Search HD 7964: Type: Star A3V V=4.752
## 9 HD7964--A3V 12760 +/- 559 1.25 20 76.81 +/- 2.14 0.29 +/- 0.02
# >> star HDC7964 at 01 19 27.993 +27 15 50.611 size 0.47 error 0.05 4.5 degrees away from target
# Wideband calibration code settings for this run:
# > Using calibration scans within +/- 1 hr of the target scan
# > Weighting calibration scans WRT temporal proximity, variance doubles in 1 hr
# > Weighting calibration scans WRT sky proximity, variance doubles in 15 deg
# > No minimum calibrated scan uncertainty
# > Flagging calibration warnings when multiple calibrators disagree by more than 2 sigma
# > Rejecting the scan when multiple calibrators disagree by more than 3 sigma
# 24 data lines read from 2008001.nullsum
# Calibration data format:
# name #NullPts MJD m/d/y/h:m:s UT Delay(m) WL(um) CalNull Err RawNull Err SysNull Err #Cal U(m) V(m) HA(hr) IntTime BLName
# Target HD123:
HD123 1 54513.28793 1/17/2008/06:54:37 6.91028 5.638653 8.495 -0.0022382 0.00443179 0.0279 0.00150.0301382 0.00417022 2 39.190259 67.935104 1.48315 100 KI_K1K2
HD123 1 54513.31684 1/17/2008/07:36:15 7.60417 8.19742 8.495 0.00428794 0.00350484 0.0337 0.00090.0294121 0.00338732 1 31.159530 68.715466 2.17894 100 KI_K1K2
HD123 1 54513.34691 1/17/2008/08:19:32 8.32583 9.76943 8.495 0.000654654 0.00339571 0.0284 0.00180.0277453 0.00287938 2 21.717665 69.325632 2.90258 100 KI_K1K2
# 3 calibrated scans output under name HD123
# 3 total calibrated scans output on target HD123
The first section of the output is an extended preamble (comment lines indicated by # symbols) containing the following pieces:
The second section of the file contains a list of calibrated output data. For each specified target/pseudonym there is a designation header, and then a sequence of tab-delimited data lines. nullCalib output data lines have 15 – 19 fields (depending on whether target astrometry has been provided as part of the calibration script). The contents of these data lines is described by the following table:
| Field | Item | Description
|
| 1 | Object | Object Designation (string)
|
| 2 | NSumRecords | The number of nullsum records averaged into this target scan (integer)
|
| 3 | MJD | Modified Julian Date (day) (float)
|
| 4 | UTC Date & Time | UTC Date/time in mm/dd/yy/hh:mm:ss format
|
| 5 | UTC | UTC time (decimal hours) (float)
|
| 6 | Delay | Average delay (m) (float)
|
| 7 | WL | SNR-weighted mean wavelength for scan (microns) (float)
|
| 8 | CalNull | Calibrated wide-band null (dimensionless)
|
| 9 | ErrNull | Std Error in calibrated null (dimensionless)
|
| 10 | RawNull | Raw (L-1) wide-band null (dimensionless)
|
| 11 | ErrRawNull | Std Error in raw wide-band null (dimensionless)
|
| 12 | SysNull | Estimated system null (dimensionless)
|
| 13 | ErrSysNull | Std Error in estimated system null (dimensionless)
|
| 14 | NCal | Number of calibrators used in calibrating this scan (integer)
|
| 15 | uCoord | u * lambda – RA baseline projection (m) (float)
|
| 16 | vCoord | v * lambda – dec baseline projection (m) (float)
|
| 17 | HRangle | Hour angle on target for this scan (decimal hours) (float)
|
| 18 | IntTime | The total integration time in this scan (float)
|
| 19 | Baseline | Baseline designation for this scan (e.g. KI_K1K2) (string)
|
A nullCalib invocation with no arguments generates a short usage reminder.
A nullCalib invocation with argument -help (or -h) outputs the online help.
Additional information and discussion on nullCalib command-line options can be found in Arguments.
The fundamental purpose of nullCalib is to estimate the null depth on a target as measured by an idealized interferometer, N_target, based on the nulling data measurement made by the interferometer, N_meas. The nullCalib algorithm for doing this uses the system null, N_sys and is simply:
(2.1) N_meas = N_target + N_sys
so of course the idealized target null is estimated by inverting Eq. 2.1:
(2.2) N_target = N_meas - N_sys
Note that N_sys may be either positive or negative. The job of nullCalib is to estimate the system null at the time of the target scan by analyzing available calibration measurements, and then to apply (in the sense of Eq 2.2) this system null model to the measured target null to estimate the idealized target null. Described more fully below, the system null estimation algorithms used by nullCalib are designed to characterize spatial and temporal variations in system performance under a variety of calibration circumstances.
The quality of the system null depends in part on knowledge of the size of the calibrator. Unresolved calibrators are ideal, but difficult to find in practice as the point of interferometers is high angular resolution. Since all astronomical sources are of finite surface temperature, depending on the part of sensitivity/spatial frequency space that one is working in, finding an unresolved source at any spatial proximity to your target may or may not be a straightforward exercise. (A related issue is the degree to which one can be certain that a given stellar source is even single; stellar multiplicity is a pervasive phenomenon.) Finite surface temperatures and spatial variations in interferometer and atmospheric conditions unfortunately can lead one to consider calibration sources that are partially resolved by the interferometer. Therefore, nullCalib's calibrator observation modeling can account for resolved calibrators with finite angular size (with the associated requirement of calculating baseline projection effects). The associated planning suite getCal http://nexsci.caltech.edu/software/getCal can assist the user in selecting potential calibrators, estimating their angular sizes, retrieving available ancillary information, and preparing nullCalib input. In the event that calibrators in practice turn out to be less than ideal, nullCalib supports the use (and more to the point, the intercomparison of data from) multiple calibrators to refine the system null estimate and facilitate the identification of troublesome calibrators.
Finally, users find it useful for their calibration application to perform geometric projection (in the parlance, u-v coordinate) calculations. As these geometrical calculations are required to properly account for resolved calibrators, it is straightforward to have nullCalib compute the measurement u-v coordinates if the target astrometry is provided (see Astrometric_Processing).
nullCalib has four main algorithmic components: Data_Management and averaging, System_Null_Estimation and application, Error_Estimation, and Astrometric_Processing. Each of these is discussed in turn.
The majority of the nullCalib code is devoted to getting the right null measurements to the right place at the right time. This is straightforwardly and tediously implemented in nullCalib by creating and traversing a data base of null measurements contained in the input nullsum file set. nullCalib implements this as a doubly-linked list data structure. It consists of a list over the object designations used in the measurement process. Hanging from the nodes of this first list is a time-sorted list of null measurements on the particular object designation. All of the memory allocation for the database is done dynamically as the structure is constructed in the course of reading the data files specified on the command line. Consequently, there is no pre-set size limit as to the maximum amount of data nullCalib can process.
The L-1 data product coming into nullCalib is the average null depth over an entire integration. This is illustrated by the following segment from a typical L-1 nullsum file (this data format was discussed above):
NULLSUM 02/01/2008/06:29:23 6.48972222222222 HD23413 0 0 555.6 0.0340 0.0010 25792 11680 11708 18.9 -6.8 8.495 1532.5 93.7 613 250 IN+OUT M1.1 0 KI_K1K2
NULLSUM 02/01/2008/06:54:37 6.91027777777778 target1 0 0 636.2 0.0279 0.0015 29512 13062 13128 17.8 -7.4 8.495 1532.5 87.2 613 250 IN+OUT M1.1 0 KI_K1K2
NULLSUM 02/01/2008/07:16:22 7.27277777777778 HD36780 0 0 540.1 0.0375 0.0014 23131 10186 10183 20.3 -9.3 8.495 1537.5 85.4 615 250 IN+OUT M1.1 0 KI_K1K2
NULLSUM 02/01/2008/07:36:15 7.60416666666667 target1 0 0 659.1 0.0337 0.0009 33085 14701 14648 22.2 -6.0 8.495 1527.5 90.9 611 250 IN+OUT M1.1 0 KI_K1K2
NULLSUM 02/01/2008/07:56:59 7.94972222222222 HD36780 0 0 569.7 0.0341 0.0009 26910 12002 12018 19.4 -8.2 8.495 1530 92.0 612 250 IN+OUT M1.1 0 KI_K1K2
This example contains two integrations of the science target (target1) bracketed by the calibrators HD23413 and HD36780. Unlike the visiblity amplitude calibration (wbCalib), nullCalib does no averaging of the input data. There will be one output line for every input data line on the target.
As described above in Algorithm, the system null estimate, N_sys, is the point source response of an imperfect interferometer in an imperfect environment. N_sys can be a function of both time and sky location, so the problem at hand in calibrating a given target scan is estimating N_sys at the time of the target scan and sky location of the target. Once that estimate has been made, it is simple to apply Eq. 2.2 to estimate the ideal target nulling data. We shall describe the algorithm used by nullCalib to make this estimate in a series of steps.
First consider an individual scan at time t_i on an unresolved calibrator (one for whom the expected nulling value N_cal is zero). After computing a scan average, the estimation of N_sys at time t_i and the sky location of the calibrator follows straightforwardly from inverting Eq. 2.1:
(2.3) N_sys = N_meas - N_cal => N_meas
However, if this calibrator has the potential to be resolved by the interferometer, N_cal is no longer necessarily zero. nullCalib treats potentially resolved calibrators as uniformly bright disk sources of a given (in fact user-specified) angular diameter theta_cal. N_cal for the uniform disk of diameter theta_cal is computed using the Bessel function to compute the visibility:
V_cal = 2 J_1(pi * theta_cal * B_proj / lambda ) / (pi * theta_cal * B_proj / lambda)
and then calculating the null depth
(2.4) N_cal = (1 - V_cal)/(1 + V_cal)
With B_proj the projected baseline on the calibrator object (see Astrometric_Processing), lambda the operating wavelength, and J_1 the first order Bessel function of the first kind.
As our fundamental goal is to estimate (and apply) the system null at the time t_target and sky location of a given target scan, we can use our knowledge of N_sys at t_i as the basis for that estimate. In general, the system null N_sys evolves with time and sky position; we are left with the problem of formulating an estimator for N_sys at t_target and sky location of the target given some set of calibrator observations taken at t_i. Speaking momentarily of just a single calibrator, it is simple to imagine a weighted averaging scheme that would take a set of calibrator observations at times t_i and estimate the system null at time t_target:
(2.5) N_sys(t_target) = Sum(N_sys(t_i) w_i) / Sum(w_i)
but this begs the question of what are the weights w_i? A simple weighted averaging scheme (derived by a minimum variance optimality criterion) uses weights as the inverse variance of the quantity estimate:
(2.6) w_i = [sigma^2_N_sys(t_i)]^-1
The quantity sigma^2_N_sys(t_i) is calculated from the uncertainty given in the nullsum file for the calibrator scan at t_i. A second, related question is to ask what about Eq. 2.5 makes it specific to time t_target? There are two possible answers to that question – both related to the postulate that the instrumental and environmental conditions tend to be more correlated on small timescales than on long timescales. The first answer is to sample calibrator scans over a time interval symmetric around the science target scan time – that way any temporal variability in the system null (N_sys(t)) is at least sampled in an approximately unbiased way, and a weighted averaging process (Eq. 2.5) estimating the system visibility will similarly be unbiased. nullCalib always constructs a symmetric time window around the science scan time in order to select calibrator scans. The user can control the size of this time window; the default value is one hour.
The second variant on the weighted averaging scheme used by nullCalib is to manipulate the weights (Eq. 2.6) used in the averaging process so as to more heavily weight calibrator scans temporarily near the science scan time. For example, doubling the error variance in one hour works well; this is easily accomplished with a slight modification to Eq. 2.6:
w_i = [sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2)]^-1
Here, T_C is the by-default one-hour scan time window, but it is under the control of the the user. So, nullCalib can supply a continuum of weighting schemes (including a nearly flat weighting scheme where T_C is large). This time weighting scheme can be turned off with the -notimeWeighting argument (see Arguments).
As well as being a function of time, the system null can be a function of sky location. Thus, one is motivated to choose calibration sources that are close to the science target; this is the primary operating model for getCal. Like the temporal argument, nullCalib provides a variance weighting scheme to emphasize spatially-near calibrators over those more distant:
(2.7) w_i = [sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C)]^-1
By default, Ang_C is set to 15 degrees, but like T_C, Ang_C is controlled by the user, so a nearly flat weighting scheme is again possible with Ang_C large. As before, this angle weighting scheme can be turned off with the -noangleProxWeighting switch (see Arguments).
Error estimation in nullCalib follows from standard error propagation calculations. From Eq. 2.2 the estimated calibrated scan variance is given by:
(2.8) sigma^2_N_target = sqrt(sigma^2_N_meas + sigma^2_N_cal)
The target null measurement variance is simply the square of the uncertainty taken from the nullsum file. The system null variance is computed by the canonical weighted average recipe adopted in Eq. 2.5 (with weights as computed in Eq. 2.7):
(2.9) sigma^2_N_sys = [Sum(1 / w_i)]^-1
= [Sum([sigma^2_N_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C)]^-1)]^-1
The estimated calibrator null calculations for nullCalib (Eq. 2.4) call for the projection of the interferometer baseline on the calibrator direction unit vector s_hat:
(2.10) B_proj = B \dot s_hat
where B is the instantaneous baseline 3-vector. nullCalib uses the Navy NOVAS astrometric subroutine package to evaluate target astrometry (see Dependencies), and it is convenient to evaluate Eq. 2.10 in true equitorial intertial coordinates. That implies that we have to evaluate B – fixed to the rotating earth. nullCalib uses the NExScI baseline class library to evaluate the instantaneous baseline vector (see Dependencies).
Further, nullCalib users find it useful to have projected interferometer baseline coordinates (u-v coordinates times the operating wavelength) on the calibration target packaged with along the calibrated target scan – this feature is more a desirement rather than a requirement. So (in instances where users provide target astrometry in their calibration script files – in all the examples provided herein), nullCalib computes u-v coordinates for calibrated target scans. This calculation is straightforward:
(2.11) u = B \dot ra_hat
v = B \dot dec_hat
with ra_hat and dec_hat as unit vectors in the instantaneous directions of increasing right ascention and declination at the target location respectively:
ra_hat = (z_hat \cross s_hat) / | z_hat \cross s_hat |
dec_hat = s_hat \cross ra_hat
and z_hat a unit vector in the direction of the Earth's angular momentum vector (e.g. (0,0,1) in equitorial inertial coordinates).
nullCalib uses (requires) the libraries below for its build. These libraries are packaged and distributed with nullCalib.
The command line arguments to nullCalib are:
-help:
print a short help message and exit. No argument, overrides all
other options when used.
-calTimeWindow argument:
set the calibration scan acceptance time window size (hr). Default
value is 1 hr. Argument required when used.
-[no]timeWeighting:
set the [non] use of time proximity weighting of calibration scans.
Default is to use time weighting. No argument when used.
-timeErrorVal argument:
if timeWeighting, set calibration scan time weighting value (hr), the
time over which the calibrator null measurement variance is
assumed to double. Default value is 1 hr. Argument required when used.
-[no]angleProxWeighting:
set the [non] use of sky proximity weighting of calibration scans.
Default is to use sky proximity weighting. No argument when
used.
-angleErrorVal argument:
if angleProxWeighting, set target/calibrator sky separation weighting
value (deg), the angle over which the calibrator null measurement
variance is assumed to double. Default value is 15 deg. Argument
required when used.
-[no]minScanUncertainty argument:
sets the [non] use of a minimum floor to the calibrated null
uncertainty. Default is to have no artificial minimum
uncertainty. Argument required when used to set, no argument required
when not using minimum.
-chiWarning argument:
set multiple calibrator system null inconsistency threshold for
triggering a warning (sigmas – dimensionless). Default is to report
system null estimate inconsistencies at or greater than the two
(2) sigma level (assuming that reporting is on, see verbosity options
below). Argument required when used.
-chiReject argument:
set multiple calibrator system null inconsistency threshold for
rejecting a target scan (sigmas – dimensionless). Default is to reject
the science scan if multiple system null estimates are
inconsistent at or greater than the three (3) sigma level. Argument
required when used.
-delayCheck[=argument]:
check delay values for targets and calibrators, and report warnings
when data delay value is not in agreement with model calculation.
Default is to check delay values. Optional floating point argument
specifies delay check tolerance (units of meters) – default value is
0.1 m (approximately 3 arcmin on a 100 m baseline).
-nodelayCheck:
do not check delay values for targets and calibrators. Default is to
check delay values.
-doCalibrators:
causes the calibrator observations to also be calibrated.
-baseline <file>: Use the baseline supplied in <file>. Normal
behavior is to look for a yyyyddd.nullbaseline file for each yyyyddd.nullsum
file given on the command line. If that fails, nullCalib looks for
yyyyddd.nullbline. If neither is found, nullCalib uses a built-in default
baseline. By using the -baseline option, the users can override
this search behavior by providing the baseline or bline filename directly.
Note that when multiple nullsum files are given in combination with this option, all of them will use the baseline supplied in <file>.
-quiet:
set nullCalib to suppress all informational messages and all but the most
serious warnings and errors. No argument when used.
-shutUp:
the same as quiet.
-normal:
set nullCalib to report nominal status, warnings, and errors. This is the
default. No argument when used.
-verbose:
set nullCalib to excessively report on status, warnings, and errors.
(This should be used when the user runs into difficulty with nullCalib
operation.) No argument when used.
note: nullCalib command-line options may be used in any order, interspersed with non-options (filename arguments) and abbreviated to uniqueness (e.g. -h, -verb, -chiW).
nullCalib uses the GNU autotools configuration system for configuration, build, and install. All necessary libraries for operation are included and build automatically.
> ./configure
> make
> make check (recommended)
> make install
If configure finds gnuplot in your PATH it will automatically build NullerPlot (package to create standard set of plots from KI nulling data). To stop configure from building and installing NullerPlot, pass it the –enable-nullerplot=no option:
> ./configure --enable-nullerplot=no
To avoid putting software from NExScI
into standard /usr/local/bin
(among other things this requires root access),
the default prefix can be overridden with a
prefix option (/usr/myLocal in the following example) to the
nullCalib configure script, as in:
> ./configure --prefix=/usr/myLocal
> make
> make check (recommended)
> make install
Either way, nullCalib executables are placed in $prefix/bin, man
pages in $prefix/man, info files in $prefix/info.
Ancillary build targets are (courtesy of GNU autotools):
> make check (automated test)
> make doc
> make clean
> make dist
> make distclean
Finally, several forms of the documentation are available. The default part of the build procedure should make the info format documentation (nullCalib.info), but the following additional documentation types (targets) are also available:
> cd doc
> make html
> make ps
> make pdf
nullCalib comes distributed with the required input for a test case (the test subdirectory). This test is designed to be run with the the make check target as part of the installation sequence (see Build). The test scripts run nullCalib against a pre-packaged calibration script and L-1 data in the test directory and then run the output through diff to check consistency with reference output.
What if "make check" fails?
We have noted two common types of installation problem. In the first, "make check" fails due to roundoff differences between the packaging platform and your installation platform. This is usually the case when diff identified only 1 or 2 lines of non-matching output for each test. You may proceed with installation ("make install") and use.
With the second problem, "make check" fails and diff seems to identify all lines from the reference test output files. The likely cause is that the runtime linker could not resolve library dependencies for the executable. We have seen this behavior with gcc3 on Solaris; One of the missing libraries is libstdc++.so. The root cause lies with the installation configuration of your gcc3.
You can check for this linking error on Solaris by typing
> ldd <path/to/your/nullCalib>/Calib/nullEngine
If you see one or more libraries missing you have two options: 1) set LD_RUN_PATH then "make clean" and "make" again, or 2) set your LD_LIBRARY_PATH when executing.
In order for either of these options to work, you will need the path to gcc3 libraries on your system. In some cases the gcc3 libraries and binaries will be installed subdirectories of a gcc3 package location (for example /usr/local/gcc3 might be the package location, with binaries found under /usr/local/gcc3/bin and libraries under /usr/local/gcc3/lib). In other cases, binaries and libraries may be installed in generic locations (e.g. /opt/bin and /opt/lib). If you cannot find your gcc3 libraries, contact your local system administrator for help.
We recommend option 1) because it can be done once at installation and no further environment settings are required. If you choose option 2) you must set your LD_LIBRARY_PATH each time you run nbnullCalib or nullCalib (or you may set it in your .cshrc).
Assuming that you have found your gcc3 library path, option 1) proceeds as
> setenv LD_RUN_PATH <your/gcc3/libraries> # for c-shell, OR
> LD_RUN_PATH="/usr/local/gcc3/lib"; export LD_RUN_PATH # for bash
> make clean
> make
> make check
> make install
Option 2) is
> setenv LD_LIBRARY_PATH <your/gcc3/libraries>:${LD_LIBRARY_PATH} # for c-shell, OR
> LD_LIBRARY_PATH="<your/gcc3/libraries>:$LD_LIRBRARY_PATH"; export LD_LIBRARY_PATH # for bash
Here we provide a list of common nullCalib error messages, their interpretation, and suggestions about their correction.
There are a variety of parsing errors that can occur when trying to compose a calibration script. Most of the work can be done with getCal. Use this feature if you can – it'll save you a lot of headaches.
Most of the parsing errors give some sort of diagnostic message, like:
Fatal Error -- Can't open calibration script file bogus
or
Fatal Error -- premature eof encountered in bo
If you're having parsing problems, compare your cal script against the example given in Inputs_and_Outputs.
Another possible error is reporting of delay mismatches in the data:
Warning: at 54234.654, 10.2 m measured/expected delay mismatch on target HDC9939
This has the potential of being serious, because it usually means there's some problem with the baseline model that's being applied, and at a minimum you're guaranteed that the u-v projections that nullCalib is calculating will be wrong. Less frequently it is also possible that there is some intrinsic problem with the data, such as incorrect delay line positions or incorrect coordinates in the calibration script.
The first thing to do is to re-run nullCalib with the -verbose switch turned on. This will enable all the baseline informational messages, and you can check to see whether there are any irregularities in the baseline information:
## Error detected in /home/bode/pti/data/100/100335.bline; reading baseline model from /home/bode/pti/data/100/100335.baseline (KI_K1K2)
The obvious thing to do in this circumstance is to make sure that the baseline model that gets assigned (e.g. KI_K1K2) is what you think it should be. If not, you should examine the measurement database.
When using multiple calibrators, nullCalib checks for system null estimate consistency among the calibrators. Consequently, probably the most common type of warning and error messages you'll see from nullCalib will be of the flavor:
# Warning: at 51724.1534 sysNull estimates between calibrators HDC121107 and HDC128167 are inconsistent at the 2.7 sigma level
and
# Reject: at 51724.1680 system null estimates between calibrators HDC121107 and HDC128167 are inconsistent at the 4.5 sigma level
# No suitable calibration found for 51724.1680
As discussed in Arguments, the user can control the thresholds at which these warnings and errors are reported. However, it's not an accident that this feature appears in nullCalib, and strong inconsistencies between calibrators can be a sign of either problems with the data on the calibrators or their sizes. Old fashion detective work is about the only general advice we can provide here.
nullCalib comes with several "helper" applications that make it a bit more civilized. These can be built at the users option (see Build instructions above).
badScans is a Perl script that allows the user to select certain scans for exclusion in subsequent processing and analyses. badScans is a very general filter for text files: it scans a stdin input text file, and looks for lines that contain one of an input list of exclusion keys. When it finds a line that contains an exclusion key, it prepends the line with a comment character (pound/hash sign – #). The last output from badScans lists the number of exclusions found.
The usage of badScans is:
> badScans xxx.badScans < input.file > output.file
or for instance in the context of a nullCalib invocation:
> nullCalib [options] cal.myTarg data | badScans myTarg.badScans > myTarg.cal
makeCalScript is a Perl script that allows the user compose a calibration script given a target name and a schedule file.
The usage of makeCalScript is:
> makeCalScript target catfile > outfile
Please note that the NExScI's getCal package also contains a utility for automatic generation of calibration scripts (csAdhoc). Please refer to the getCal documentation (http://nexsci.caltech.edu/software/getCal/index.html#gcCalScript).
NullerPlot is a Perl package which generates a standard set of plots for KI data. NullerPlot will be automatically built and installed if gnuplot is found in your PATH during configuration.
The usage of NullerPlot is:
> NullerPlot -help
usage: NullerPlot filename
options: -all (creates series of plots from sumfile)
-wavelength specfile (plots spectrometer values by wavelength)
-gif produces gif output files
-png produces png output files
-hc produces postscript output files
-size x,y set output image size in pixels [default: 640,480 *requires gnuplot 4+])
-scale r (set output image scale [default: 1 for 640x480, use 1.25 for 800x600])
-font tiny|small|medium|large|giant (set font size [default: medium])
-pointsize r (set point size [default: 1.0])
-spec (creates series of plots from specfile)
-anc ancfile (creates ancilliary plots, used only with all)
-bt btfile (creates block trending plots, used only with all)
-stdin (takes input data from stdin)
-start (set start time in hh.nnn or hh:mm:ss for star plots)
-duration (set duration in seconds for star plots)
-timeseries file (creates time series plots from timeseries.csv)
-xtics n (override number tics on x axis for star plots)
-version (prints version info)
-help (prints this message)
nullCalib v1.7
Copyright 2008 California Institute of Technology. For questions or comments about this software, please fill out the contact form at http://nexsci.caltech.edu/contact/help_desk.html. Permission to use, copy and distribute this software and its documentation for academic and/or non-profit research purposes, without fee and without a written agreement is hereby granted, provided that the above copyright notice, this paragraph and the following three paragraphs appear in all copies. Reuse of all or part of the software contained here for commercial purposes is strictly prohibited.
The California Institute of Technology makes no proprietary claims to the results, prototypes, or systems supporting and/or necessary for the use of the research, results and/or prototypes for academic and/or non-profit research uses only. However, to the extent that any software and system built in collaboration with industry partners may incorporate proprietary designs of the industry partners, it is possible that certain restrictions may be imposed on the proprietary information.
In no event shall California Institute of Technology be liable to any party for direct, indirect, special, incidental or consequential damages, including lost profits, arising out of the use of this software and its documentation, even if the California Institute of Technology has been advised of the possibility of such damage.
The California Institute of Technology specifically disclaims any warranties, including the implied warranties or merchantability and fitness for a particular purpose. The software and documentation provided hereunder is on an "as is" basis, and the California Institute of Technology has no obligations to provide maintenance, support, updates, enhancements or modifications.