NExScI Home
Planetquest
Keck
Exoplanet Exploration
Caltech     IPAC           JPL           NASA
nbCalib -- Narrow-band Interferometric Visibility Calibration

nbCalib -- Narrow-band Interferometric Visibility Calibration

Next: , Previous: (dir), Up: (dir)

nbCalib – Narrow-band Optical Interferometric Visibility Amplitude Calibration

nbCalib v1.4.4 – Narrow-band Interferometric Visibility Calibration

Table of Contents

nbCalib is a product of the NASA Exoplanet Science Institute.

./images/NExScI_FullColorLogo_WithText.png

--- The Detailed Node Listing ---

Description

Algorithm

Ancillary

Next: , Previous: Top, Up: Top

1 General Description and Features

nbCalib is a narrow-band interferometric (squared) visibility amplitude (modulus) calibration application: it correlates observations on one or more science targets with observations on one or more calibration sources, estimates a model of the system visibility (see Algorithm below) at the times of the target scan, applies this system visibility estimate to estimate the calibrated target visibility amplitude at time, and optionally computes ancillary geometric information (i.e. u-v coordinates, delays, hour angles). nbCalib was developed in the context of PTI, where it reads the interferometric data from one or more vis-output "spec" files (L-1 data). All nbCalib input comes in two (or more) input ASCII files; all data output is ASCII and goes to stdout (with error messages to stderr).

The canonical use case for nbCalib is: 

     nbCalib [options] xxx.calScript xxx.spec [yyy.spec...] [> xxx.calData]

where xxx.calScript is a script file that contains object designation and astrometry information in a standard format, and xxx.spec (and any number of additional input data files yyy.spec) is the standard L-1 narrow-band V^2 reduced data product file. Options for nbCalib are summarized below in Arguments. nbCalib is the sister application of wbCalib, the corresponding wide-band visibility calibration package. Readers familier with wbCalib will find most of the information in this documentation redundant.

The reader unfamilier with the use of terms like interferometric visibility and interferometric resolution is refered to any standard interferometric textbook such as Thompson, Moran, and Swenson, the NRAO Imaging Summer School Notes, or lecture notes from the Michelson Summer School. In particular it is important (particularly for radio interferometrists) to note that nbCalib deals with the calibration of normalized squared visibility modulus, which optical interferometrists notationally describe as V^2, and often call squared visibilty amplitude, or visibility amplitude, or sometimes (unfortunately) just visibility. V^2 is a dimensionless quantity – unlike the corresponding currency in radio astronomy.

Next: , Previous: Description, Up: Description

1.1 Inputs and Outputs

nbCalib takes no fewer than two inputs, a calibration script that defines the targets and calibrators of interest in this run, and one or more input data file specifications. Output from nbCalib is written to stdout (with major warnings and errors also written to stderr): 

     nbCalib xxx.calScript xxx.spec [yyy.spec...] [> xxx.calData]

Various command-line arguments that control the behavior of nbCalib are also available (see Arguments).

1.1.1 Calibration Script File

The ASCII nbCalib script file contains both science target and calibrator object designations, astrometric information, and calibrator diameter information: 

     # Simbad Search HD 9939: Type: High proper-motion Star  K0IV V=6.99
     HDC9939
     01 37 25.107   +25 10 03.969 -0.211 -0.208 0.0238 # K0IV  V = 7.0, K = 5.0
     ---
     2
     # Simbad Search HD 7034: Type: Star  F0V V=5.160
     ##  7 HD7034--F0V      7306 +/-   0    1.79  11    25.82 +/- 0.66  0.52 +/- 0.01  XX
     HDC7034   01 11 06.768   +31 25 29.051  -0.009 -0.012 0.00582 0.50 0.05   # F0V   V = 5.2, K = 4.5
     # Simbad Search HD 7964: Type: Star  A3V V=4.752
     ## 9 HD7964--A3V      8686 +/-   0    9.18  20    47.44 +/- 1.85  0.50 +/- 0.01  XX         XXXX
     HDC7964   01 19 27.993   +27 15 50.611   0.026 -0.012 0.01049 0.47 0.05   # A3V   V = 4.7, K = 4.6

Lines begining with the pound (hash – #) sign are comments and are ignored by nbCalib. The astrometry format is a standard J2000 position (hh mm ss dd mm ss), proper motion (arcsec/yr), plx (arcsec), ancillary info PTI-standard format. Targets to be calibrated are designated in the first section of the file, in this example a single target designated HDC9939. On a separate line the astrometry for the target is given. Then separated by a line separator (—), the calibrator section follows. The number of calibrators to use is given, followed by single-line calibrator specifications, indicating the calibrator designation, astrometry (J2000 position, proper motion, and parallax; units are RA: hh mm ss.s of time, Dec: dd mm ss.s of arc, pm: arcsec/yr, parallax arcsec), estimated angular diameter and standard error in such (units of millarcseconds, mas). Take note of the calibrator angular diameter and error fields – they are the critical parameters that determine the scale of the calibrated visibilities.

While the script file can be composed (or edited) by hand, a template calibration script file can be composed by the getCal experiment planning tool suite.

1.1.2 L-1 Narrow-band Visibility File

The xxx.spec file is the standard L-1 data product from the PTI/KI V^2 processing code (Vis/Kvis). It's format looks like: 

     SPEC: 06/28/100 HDC144208    5.661736  -13.6204672  5 12.460  2.396  0.581  0.687  1.012 19.722  2.299  0.571  0.700  1.014 18.364  2.203  0.523  0.654  1.009 15.065  2.106  0.488  0.619  1.024  5.854  2.009  0.479  0.476  1.047
     SPEC: 06/28/100 HDC144208    5.668814  -13.5713159  5 13.250  2.396  0.660  0.683  1.012 21.017  2.299  0.649  0.745  1.014 20.065  2.203  0.594  0.655  1.009 16.232  2.106  0.620  0.707  1.024  6.230  2.009  0.660  0.864  1.047
     SPEC: 06/28/100 HDC144208    5.676736  -13.5164837  5 11.759  2.396  0.611  0.762  1.012 18.606  2.299  0.542  0.655  1.014 17.321  2.203  0.559  0.688  1.009 13.921  2.106  0.533  0.674  1.024  5.318  2.009  0.448  0.500  1.047
     ...

nbCalib requires at least one input spec file, but will in fact process any (large) number of additional spec file specifications on the command line (e.g.): 

     nbCalib xxx.calScript xxx.spec yyy.spec zzz.spec aaa.spec ... [> xxx.calData]

This capability allows a single nbCalib invocation to calibrate large amounts of visibility data stored separately in individual spec files; you can calibrate nights individually, or calibrate 3 years worth of data.

L1 data files can also be used as input in FITS format, with no restrictions on file name or extension. Please refer to L1 FITS specification for details on the KI V2 L1 FITS format.

1.1.3 nbCalib Output

Depending on the user choice of nbCalib verbosity (see Arguments), output from nbCalib looks like: 

     (gnomad:8) nbCalib cs.hd195987 100180.spec
     # Calibrating star HDC195987  at 20 32 51.642 +41 53 54.522
     
     #  with respect to 4 calibrators:
     #  star HDC195194  at 20 28 25.791 +39 19 44.250   size 0.67  error 0.1  2.7 degrees away from target
     #  star HDC200031  at 20 59 42.912 +38 49 22.504   size 0.6  error 0.1    6 degrees away from target
     #  star HDC177196  at 19 01 26.368 +46 56 05.325   size 0.7  error 0.06   17 degrees away from target
     #  star HDC185395  at 19 36 26.538 +50 13 15.970   size 0.84  error 0.04   13 degrees away from target
     
     # Calibration code settings for this run:
     # > Calibrating all wavebands
     # > Calibrating all frame rates
     # > Calibrating incoherent spectrometer V2 data
     # > Applying jitter correction
     # > Using calibration scans within +/- 1 hr of the target scan
     # > Weighting calibration scans WRT temporal proximity, variance doubles in 1 hr
     # > Weighting calibration scans WRT sky proximity, variance doubles in 15 deg
     # > No minimum calibrated scan uncertainty
     # > Flagging calibration warnings when multiple calibrators disagree by more than 2 sigma
     # > Rejecting the scan when multiple calibrators disagree by more than 3 sigma
     
     # 271 data lines read from 100180.spec
     # Calibration data format:
     # name	#VisPts	MJD	m/d/y/h:m:s	UT	Delay(m)	#Chan	WL(um)	CalVis	Err	RawVis	Err	SysVis	Err...	#Cal	U(m)	V(m)	HA(hr)	BLName
     
     # Target HDC195987:
     # Warning: at 51723.3761 sysVis estimates between calibrators HDC200031 and HDC185395 are inconsistent at the 2.2 sigma level
     HDC195987	5	51723.37614	6/28/100/9:1:38	9.0274	-32.2712	5	2.396	0.717954	0.185974	0.5772	0.147111	0.803951	0.0371808	2.299	0.818887	0.054887	0.5996	0.0299239	0.732213	0.0327609	2.203	1.04916	0.110158	0.7392	0.0530713	0.704562	0.0539792	2.106	0.671904	0.162046	0.4934	0.114967	0.734331	0.0456897	2.009	-0.260067	1.31634	-0.1524	0.77102	0.586004	0.0904048	3	-49.483712	-96.068125	-0.860713	PTI_NS
     
     # Warning: at 51723.3896 sysVis estimates between calibrators HDC200031 and HDC185395 are inconsistent at the 2.0 sigma level
     HDC195987	5	51723.38956	6/28/100/9:20:58	9.34947	-29.2717	5	2.396	1.10414	0.200846	0.8818	0.155426	0.798628	0.0358986	2.299	0.964866	0.0867675	0.7028	0.0555928	0.728391	0.0311573	2.203	0.774638	0.0653161	0.538	0.0226627	0.694518	0.0507289	2.106	0.684614	0.13061	0.4916	0.0891933	0.718068	0.0423484	2.009	-0.373657	0.55363	-0.2216	0.326886	0.593057	0.0824321	3	-45.105387	-98.740076	-0.537763	PTI_NS
     
     HDC195987	5	51723.41871	6/28/100/10:2:56	10.0491	-23.8127	5	2.396	0.52008	0.132474	0.4102	0.103244	0.788724	0.0308713	2.299	0.2174	0.0561052	0.1638	0.0417344	0.75345	0.030926	2.203	0.253686	0.0403794	0.1822	0.0260799	0.718212	0.0500006	2.106	0.452483	0.0691282	0.3332	0.0462402	0.736381	0.0470445	2.009	0.596225	0.731455	0.3798	0.462217	0.637007	0.0986268	3	-34.528602	-103.637869	0.163832	PTI_NS
     
     HDC195987	5	51723.43261	6/28/100/10:22:57	10.3827	-21.7606	5	2.396	-0.00461744	0.0983485	-0.0036	0.0766775	0.779652	0.0312187	2.299	0.30862	0.0763539	0.2338	0.0570873	0.757566	0.0302001	2.203	0.430719	0.0761984	0.309	0.0511038	0.717405	0.0450594	2.106	0.954759	0.208171	0.705	0.147509	0.738406	0.0452858	2.009	1.67607	0.317735	1.075	0.0991383	0.641382	0.10623	3	-29.053836	-105.498135	0.49831	PTI_NS
     
     HDC195987	5	51723.44924	6/28/100/10:46:54	10.7817	-19.76	5	2.396	0.00610819	0.20137	0.0048	0.158242	0.78583	0.0289093	2.299	0.645381	0.0640445	0.4968	0.0460756	0.769778	0.0271728	2.203	0.85441	0.0904047	0.6162	0.0560193	0.721199	0.0390439	2.106	0.919917	0.0583453	0.7062	0.0244818	0.767678	0.0407729	2.009	0.197098	0.460153	0.1392	0.3243	0.706249	0.106816	3	-22.216218	-107.292969	0.898412	PTI_NS
     
     HDC195987	5	51723.46329	6/28/100/11:7:7	11.1189	-18.4922	5	2.396	0.333942	0.16019	0.2616	0.12501	0.783369	0.0327602	2.299	0.881791	0.0537716	0.6778	0.0316443	0.768663	0.0301538	2.203	0.873385	0.105202	0.6254	0.0666951	0.716065	0.0401008	2.106	0.830083	0.190091	0.6388	0.141957	0.769562	0.0425597	2.009	-1.68011	0.81481	-1.1426	0.516616	0.680074	0.11929	3	-16.245651	-108.430393	1.23649	PTI_NS
     
     HDC195987	5	51723.47423	6/28/100/11:22:53	11.3814	-17.7801	5	2.396	0.700943	0.364691	0.5476	0.283807	0.781233	0.0357119	2.299	0.897403	0.198883	0.688	0.149749	0.766657	0.0319817	2.203	0.72824	0.255886	0.518	0.179534	0.711304	0.0411092	2.106	0.194606	0.292886	0.15	0.225575	0.770786	0.0460331	2.009	-0.933437	1.56292	-0.6264	1.04169	0.671068	0.130862	3	-11.505750	-109.069313	1.49976	PTI_NS
     
     HDC195987	5	51723.48849	6/28/100/11:43:25	11.7238	-17.2193	5	2.396	1.48957	0.651106	1.0504	0.416038	0.70517	0.130388	2.299	0.898156	0.131502	0.653	0.0736451	0.727045	0.0678831	2.203	0.365418	0.148217	0.2484	0.0987838	0.679769	0.0542553	2.106	0.268877	0.267829	0.2112	0.209466	0.78549	0.0727343	2.009	15.3734	49.7315	1.779	1.36827	0.11572	0.363609	2	 -5.245604	-109.572417	1.8431	PTI_NS
     
     HDC195987	5	51723.49570	6/28/100/11:53:48	11.8969	-17.0957	5	2.396	2.67978	1.60863	1.8438	1.04061	0.688042	0.140695	2.299	0.878223	0.25584	0.641	0.175399	0.729883	0.072951	2.203	0.364603	0.195515	0.2486	0.131617	0.681837	0.0580732	2.106	0.0546386	0.465463	0.043	0.36629	0.786989	0.0765758	2.009	-12.6906	42.6868	-1.569	2.05818	0.123635	0.382937	2	 -2.062488	-109.683289	2.01662	PTI_NS
     
     # 9 calibrated scans output under name HDC195987
     # 9 total calibrated scans output on target HDC195987

The first section of the output is an extended preamble (comment lines indicated by # symbols) containing the following pieces:

The second section of the file containes a list of calibrated output data. For each specified target/pseudonym there is a designation header, and then a sequence of tab-delimited data lines. nbCalib output data lines have a variable number of fields depending on the number of spectrometer channels in the input data, and whether target astrometry has been provided as part of the calibration script to support u-v calculations. The contents of these data lines is described by the following table:

Field Item Description


1 Object Object Designation (string)
2 NSpecRecords The number of sum records averaged into this target scan (integer)
3 MJD Modified Julian Date (day) (float)
4 UTC Date & Time UTC Date/time in mm/dd/yy/hh:mm:ss format
5 UTC UTC time (decimal hours) (float)
6 Delay Average delay (m) (float)
7 NChannels The number of spectrometer channels (integer)
8 + 7*(i-1) WL i Channel i wavelength (microns) (float)
9 + 7*(i-1) CalVis i Channel i calibrated visibility (dimensionless) (float)
10 + 7*(i-1) ErrVis i Channel i std Error in calibrated visibility (dimensionless) (float)
11 + 7*(i-1) RawVis i Channel i Raw (L-1) visibility (dimensionless) (float)
12 + 7*(i-1) ErrRawVis 1Channel i std Error in raw visibility (dimensionless) (float)
13 + 7*(i-1) SysVis i Channel i system visibility (dimensionless) (float)
14 + 7*(i-1) ErrSysVis iChannel i std Error in estimated system visibility (dimensionless)
15 + 7 * (NChannels-1) NCal Number of calibrators used in calibrating this scan (integer)
16 + 7 * (NChannels-1) uCoord u * lambda – RA baseline projection (m) (float)
17 + 7 * (NChannels-1) vCoord v * lambda – dec baseline projection (m) (float)
18 + 7 * (NChannels-1) HRangle Hour angle on target for this scan (decimal hours) (float)
19 + 7 * (NChannels-1) Baseline Baseline designation for this scan (e.g. PTI_NS) (string)

where the channel index i runs from 1 to NChannels.

The nbCalib output can also be in FITS format, if the -fits option is used. An optional argument may be used to specify the output file name (e.g. -fits=myFITSfile). If no argument is used, the default file name nbCalib-out.fits is used. The L2 FITS format conforms to the IAU working group standard for optical/infrared calibrated visibilities, please refer to IAU standard page for details. Please note that the use of the -fits and -doCalibrators options are mutually exclusive in the current release.

Previous: Inputs_and_Outputs, Up: Description

1.2 Additional Features

A nbCalib invocation with no arguments generates a short usage reminder.

A nbCalib invocation with argument -help (or -h) outputs the online help.

Additional information and discussion on nbCalib command-line options can be found in Arguments.

Next: , Previous: Description, Up: Top

2 Algorithm

The fundamental purpose of nbCalib is to estimate the squared visibility modulus (herein refered to a visibility amplitude or visibility) on a target as measured by an idealized interferometer V^2_target based on the visibility amplitude measurement made by the interferometer V^2_meas. The nbCalib algorithm for doing this is very simple; following Mozurkewich91 (see References), we model the visibility amplitude performance of the interferometer as linearly degraded by a multiplicative factor refered to as the system visibility V^2_sys:

(2.1) V^2_meas = V^2_target * V^2_sys

so of course the idealized target visibility is estimated by inverting Eq. 2.1:

(2.2) V^2_target = V^2_meas / V^2_sys

Eq. 2.1 illustrates the physical interpretation of V^2_sys: since the ideal visibility amplitude for an unresolved point source is 1, V^2_sys represents the point source response of the interferometer. This observation further suggests the methodology for calibrating the system visibility – namely to periodically look at calibration point sources with the interferometer to quantify the interferometer response (duh)! Therefore, the job of nbCalib is to estimate the system visibility at the time of the target scan by analyzing available calibration measurements, and then to apply (in the sense of Eq 2.2) this system visibility model it to the measured target visibility at epoch to estimate the idealized target visibility at epoch.

This prescription sounds straightforward, but it leaves open (at least) two important questions. The first and most fundamental of these questions is to consider why the response of the interferometer isn't perfect (quantitatively, why isn't V^2_sys = 1)? (It is in fact the essence of the experimental art to appreciate and correct for the shortcomings in one's measurement apparatus.) A detailed answer to the question of interferometer performance imperfections is well beyond the scope of this manual; for our purposes suffice it to say that visibility degradations arise both from imperfections in the interferometer itself, and because the interferometer finds itself in an imperfect environment (namely a turbulent and opaque atmosphere). Both interferometer and enviromental imperfections can be both temporally and spatially (with sky position) varying; therefore calibration observations must be made on both a timescale and a spatial scale which capture these variations, whatever they may be. It is the job of the experiment designer to design an observing strategy that can capture the important variations in system performance while efficiently utilizing observing resources. Described more fully below, the system visibility estimation algorithms used by nbCalib are designed to characterize spatial and temporal variations in system performance under a varieity of calibration circumstances.

The second important question to consider is where does one go to find an unresolved source? The point of interferometers is high angular resolution. Since all astronomical sources are of finite surface temperature, depending on the part of sensitivity/spatial frequency space that one is working in, finding an unresolved source at any spatial proximity to your target may or may not be a straightforward exercise. (A related issue is the degree to which one can be certain that a given stellar source is even single; stellar multiplicity is a pervasive phenomenon.) Finite surface temperatures and spatial variations in interferometer and atmospheric conditions unfortunately can lead one to consider calibration sources that are partially resolved by the interferometer. Therefore, nbCalib's calibrator observation modeling can account for resolved calibrators with finite angular size (with the associated requirement of calculating baseline projection effects). The associated planning suite getCal can assist the user in selecting potential calibrators, estimating their angular sizes, retrieving available ancillary information, and preparing nbCalib input. In the event that calibrators in practice turn out to be less than ideal, nbCalib supports the use (and more to the point, the intercomparison of data from) multiple calibrators to refine the system visibility estimate and facilitate the identification of troublesome calibrators.

Finally, users find it useful for their calibration application to perform geometric projection (in the parlance, u-v coordinate) calculations. As these geometrical calculations are required to properly account for resolved calibrators, it is straightforward to have nbCalib compute the measurement u-v coordinates if the target astrometry is provided (see Astrometric_Processing).

Next: , Previous: Algorithm, Up: Algorithm

2.1 Algorithm Details

nbCalib has four main algorithmic components: Data_Management and averaging, System_Visibility_Estimation and application, Error_Estimation, and Astrometric_Processing. Each of these is discussed in turn.

Next: , Previous: Algorithm_Details, Up: Algorithm

2.2 Data Management

The majority of the nbCalib code is devoted to getting the right visibility measurements to the right place at the right time. This is straightforwardly and tediously implemented in nbCalib by creating and traversing a data base of visibility measurements contained in the input visibility (sum) file set. nbCalib implements this as a doubly-linked list data structure; a list over the object designations used in the measurement process, and hanging from the nodes of this first list a time sorted list of visibility measurements on the particular object designation. All of the memory allocation for the database is done dynamically as the structure is constructed in the course of reading the data files specified on the command line. Consequently there is no pre-set size limit as to the maximum amount of data nbCalib can process. A typical year's (2000) worth of L-1 PTI data is typically on the order of six Mbytes in size, so nbCalib can comfortably handle several years worth of such visibility measurement loads on modern computer hardware without really breaking a sweat.

The L-1 data product coming into nbCalib is usually two or more visibility averages over a block of (typically) 25 seconds, resulting in a scan that is typically 90 – 130 seconds in length. This is illustrated by the following segment from a typical L-1 (sum) visibility file (this data format was discussed above): 

     SUM: 10/05/100 HDC182488    2.078131  -10.7919001   185.0  0.368  0.342    8.9  0.650  0.767  0.66   1.00   1.00    75.3   272.2   210.7   86.6   95.3   91.1  2.219   0.79   7  0  OK 19.39278221 33.22196579 100
     SUM: 10/05/100 HDC182488    2.085947  -10.7238601   183.5  0.387  0.362    8.1  0.643  0.705  0.60   1.05   1.05    75.3   272.2   210.7   86.6   95.3   91.1  2.219   0.92   1  0  OK 19.39278221 33.22196579 100
     SUM: 10/05/100 HDC182488    2.092986  -10.6627497   191.4  0.415  0.388    8.6  0.693  0.768  0.59   1.04   1.04    75.3   272.2   210.7   86.6   95.3   91.1  2.218   0.98   0  0  OK 19.39278221 33.22196579 100
     SUM: 10/05/100 HDC182488    2.099931  -10.6026391   186.5  0.379  0.350    8.5  0.650  0.735  0.64   1.05   1.04    75.3   272.2   210.7   86.6   95.3   91.1  2.220   0.98   1  0  OK 19.39278221 33.22196579 100
     SUM: 10/05/100 HDC182488    2.106878  -10.5426376   183.7  0.357  0.332    8.5  0.652  0.757  0.66   1.05   1.04    75.3   272.2   210.7   86.6   95.3   91.1  2.220   0.84   5  0  OK 19.39278221 33.22196579 100
     SUM: 10/05/100 HDC174881    2.164375    2.0741479   433.5  0.211  0.201   20.8  0.299  0.330  0.53   1.13   1.12    75.3   508.7   336.8   86.5  104.3   98.4  2.220   0.94   2  0  OK 18.85997009 28.78367043 100
     SUM: 10/05/100 HDC174881    2.171444    2.1225823   429.2  0.182  0.171   21.4  0.279  0.309  0.55   1.13   1.13    75.3   508.7   336.8   86.5  104.3   98.4  2.220   0.86   4  0  OK 18.85997009 28.78367043 100
     SUM: 10/05/100 HDC174881    2.178542    2.1710274   425.4  0.193  0.182   19.2  0.305  0.322  0.51   1.13   1.13    75.3   508.7   336.8   86.5  104.3   98.4  2.222   0.98   0  0  OK 18.85997009 28.78367043 100
     SUM: 10/05/100 HDC174881    2.185486    2.2182318   417.7  0.191  0.179   19.3  0.294  0.324  0.56   1.13   1.13    75.3   508.7   336.8   86.5  104.3   98.4  2.221   0.94   2  0  OK 18.85997009 28.78367043 100
     SUM: 10/05/100 HDC174881    2.192431    2.2652693   427.4  0.169  0.159   20.6  0.300  0.325  0.56   1.13   1.13    75.3   508.7   336.8   86.5  104.3   98.4  2.221   0.94   2  0  OK 18.85997009 28.78367043 100
     SUM: 10/05/100 HDC199763    2.256900  -21.3263380   276.1  0.350  0.319   10.4  0.603  0.662  0.71   1.56   1.56    75.3   351.6   266.4   86.4   96.3   94.3  2.224   0.92   2  0  OK 20.97099304 30.39593887 100
     SUM: 10/05/100 HDC199763    2.264236  -21.2728906   272.2  0.316  0.286   11.0  0.573  0.629  0.72   1.56   1.56    75.3   351.6   266.4   86.4   96.3   94.3  2.220   0.94   0  0  OK 20.97099304 30.39593887 100
     SUM: 10/05/100 HDC199763    2.271181  -21.1396865   272.4  0.350  0.322   11.4  0.605  0.666  0.71   1.56   1.56    75.3   351.6   266.4   86.4   96.3   94.3  2.220   0.94   1  0  OK 20.97099304 30.39593887 100
     SUM: 10/05/100 HDC199763    2.278158  -21.0486798   275.4  0.260  0.234   10.9  0.571  0.665  0.80   1.56   1.56    75.3   351.6   266.4   86.4   96.3   94.3  2.222   0.88   4  0  OK 20.97099304 30.39593887 100
     SUM: 10/05/100 HDC199763    2.285208  -20.9568405   266.0  0.377  0.351   10.7  0.647  0.688  0.60   1.56   1.56    75.3   351.6   266.4   86.4   96.3   94.3  2.221   0.98   0  0  OK 20.97099304 30.39593887 100

Here three sequential 130-second visibility scans on a calibrator (HDC182488), target (HDC174881), calibrator (HDC199763) sequence are represented by multiple (five) 25-second block averages for each of the visibility scan on the objects. nbCalib produces calibrated visibility estimates for these visibility data scans, and so it is responsible for consolidating the constituent blocks into scan averages. The time window over which these blocks are averaged is by default 150 sec, but this can be controlled by the command-line argument scanThreshold (see Arguments). This process is straightforward, and results in scans being represented by unweighted averages and sample standard deviations in the visibility data. nbCalib performs all subsequent visibility arithmetic in terms of these scan averages and standard deviations.

Next: , Previous: Data_Management, Up: Algorithm

2.3 System Visibility Estimation

As described above in Algorithm, the system visibility estimate V^2_sys is the point source response of an imperfect interferometer in an imperfect environment. We generally consider V^2_sys to be a function of both time and sky location, so the problem at hand in calibrating a given target scan is estimating V^2_sys at the time of the target scan, and sky location of the target. Once that estimate has been made, it is simple to apply Eq. 2.2 to estimate the ideal target visibility amplitude. We shall describe the algorithm used by nbCalib to make this estimate in a series of steps.

First consider an individual scan at time t_i on an unresolved calibrator (one for whom the expected visibility amplitude V^2_cal is one). After computing a scan average, the estimation of V^2_sys at time t_i and the sky location of the calibrator of course follows straightforwardly from inverting Eq. 2.1:

(2.3) V^2_sys = V^2_meas / V^2_cal => V^2_meas

However, if this calibrator has the potential to be resolved by the interferometer V^2_cal is no longer necessarily one, and the denominator in Eq. 2.3 becomes nontrivial. nbCalib treats potentially resolved calibrators a uniformly bright disk sources of a given (in fact user specified) angular diameter theta_cal. It is straightforward to compute V^2_cal for the uniform disk of diameter theta_cal (a tedious derivation is given in http://olbin.jpl.nasa.gov/iss1999/coursenotes/chapts1and2.pdf):

(2.4) V^2_cal = (2 J_1(pi * theta_cal * B_proj / lambda ) / (pi * theta_cal * B_proj / lambda))^2

With B_proj the projected baseline on the calibrator object (see Astrometric_Processing), lambda the operating wavelength, and J_1 the first order Bessel function of the first kind. With such a V^2_cal, the last equality in Eq. 2.3 falls away, and we are left with V^2_sys = V^2_meas / V^2_cal as our estimate for V^2_sys at t_i and the calibrator sky position.

As our fundamental goal is to estimate (and apply) the system visibility at the time t_target and sky location of a given target scan, we can use our knowledge of V^2_sys at t_i as the basis for that estimate. In general the system visibility V^2_sys evolves with time and sky position; we are left with the problem of formulating an estimator for V^2_sys at t_target and sky location of the target given some set of calibrator observations taken at t_i. Speaking momentarily of just a single calibrator, it is simple to imagine a weighted averaging scheme that would take a set of calibrator observations at times t_i and estimate the system visibility at time t_target:

(2.5) V^2_sys(t_target) = Sum(V^2_sys(t_i) w_i) / Sum(w_i)

but this begs the question of what are the weights w_i? A simple weighted averaging scheme (derived by a minimum variance optimality criterion) uses weights as the inverse variance of the quantity estimate:

(2.6) w_i = [sigma^2_V^2_sys(t_i)]^-1

The quantity sigma^2_V^2_sys(t_i) can be straightforwardly estimated from the scan averaging on and variance of the calibrator scan at t_i (see discussion in Data_Management). A second, related question is to ask what about Eq. 2.5 makes it specific to time t_target? There are two possible answers to that question – both related to the postulate that the instrumental and environmental conditions tend to be more correlated on small timescales than on long timescales. The first answer is to sample calibrator scans over a time interval symmetric around the science target scan time – that way any temporal variability in the system visibility (V^2_sys(t)) is at least sampled in an approximately unbiased way, and a weighted averaging process (Eq. 2.5) estimating the system visibility will similarly be unbiased. nbCalib always constructs a symmetric time window around the science scan time in order to select calibrator scans. The user can control the size of this time window; the default value is one hour.

The second variant on the weighted averaging scheme used by nbCalib is to manipulate the weights (Eq. 2.6) used in the averaging process so as to more heavily weight calibrator scans temporarily near the science scan time. Our experience at PTI indicates that a scheme that doubles the error variance in one hour works well; this is easily accomplished with a slight modification to Eq. 2.6:

w_i = [sigma^2_V^2_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2)]^-1

Here T_C is the by default the one hour scan time window, but it is under the control of the the user, so nbCalib can supply a continuum of weighting schemes (including a nearly flat weighting scheme where T_C is large). This time weighting scheme can be turned off with the -notimeWeighting argument (see Arguments).

As we argued above, as well as being a function of time the system visibility can be a function of sky location. Thus one is motivated to chose calibration sources that are close to the science target; this is the primary operating model for getCal. Like the temporal argument, nbCalib provides a variance weighting scheme to emphasize spatially-near calibrators over those more distant:

(2.7) w_i = [sigma^2_V^2_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C]^-1

By default, Ang_C is set to 15 degrees, but like T_C, Ang_C is controled by the user, so a nearly flat weighting scheme is again possible with Ang_C large. As before, this angle weighting scheme can be turned off with the -noangleProxWeighting switch (see Arguments).

Next: , Previous: System_Visibility_Estimation, Up: Algorithm

2.4 Error Estimation

Error estimation in nbCalib follows from standard error propagation calculations. From Eq. 2.2 the estimated calibrated scan variance is given by:

(2.8) sigma^2_V^2_target = (V^2_meas / V^2_sys)^2 sigma^2_V^2_meas + ((V^2_meas / (V^2_sys)^2)^2 sigma^2_V^2_sys

The target visibility measurement variance is simply that calculated in the scan averaging process described above in Data_Management. The system visibility variance is compute by the canonical weighted average recipe adopted in Eq. 2.5 (with weights as computed in Eq. 2.7):

(2.9) sigma^2_V^2_sys = [Sum(1 / w_i)]^-1

= [Sum([sigma^2_V^2_sys(t_i) * (1 + (t_i - t_target)^2 / T_C^2) * (1 + angBetween(s_target,s_i) / Ang_C]^-1)]^-1

In our experience with PTI the errors computed by nbCalib are correct relative to each other, and approximately correct in an absolute sense (in the sense that models fit to the calibrated data typically have chi-squareds close to one. In particular we are NOT by default weighting the computed scan errors by root-Nsamples in the input data – however this option is available if the user desires with the -useRootNWeighting command-line argument (see Arguments).

Previous: Error_Estimation, Up: Algorithm

2.5 Astrometric Processing

The estimated calibrator visibility calculations for nbCalib (Eq. 2.4) call for the projection of the interferometer baseline on the calibrator direction unit vector s_hat:

(2.10) B_proj = B \dot s_hat

where B is the instantaneous baseline 3-vector. nbCalib uses the Navy NOVAS astrometric subroutine package to evaluate target astrometry (see Dependencies), and it is convenient to evaluate Eq. 2.10 in true equitorial intertial coordinates. That implies that we have to evaluate B – fixed to the rotating earth. nbCalib uses the MSC baseline class library to evaluate the instantaneous baseline vector (see Dependencies).

Further, nbCalib users find it useful to have projected interferometer baseline coordinates (u-v coordinates times the operating wavelength) on the calibration target packaged with along the calibrated target scan – this feature is more a desirement rather than a requirement. So (in instances where users provide target astrometry in their calibration script files – in all the examples provided herein) nbCalib computes u-v coordinates for calibrated target scans. This calculation is straightforward:

(2.11) u = B \dot ra_hat

v = B \dot dec_hat

with ra_hat and dec_hat as unit vectors in the instaneous directions of increasing right ascention and declination at the target location respectively:

ra_hat = (z_hat \cross s_hat) / | z_hat \cross s_hat |

dec_hat = s_hat \cross ra_hat

and z_hat a unit vector in the direction of the Earth's angular momentum vector (e.g. (0,0,1) in equitorial intertial coordinates).

Please note that these u-v coordinates are instantaneous – that is at the epoch of observation. At the time of this writing the upgrade to rotate these coordinates to a single common epoch (e.g. Jan 1, 2000) is planned but not yet completed (see our nbCalib ToDo list).

Next: , Previous: Algorithm, Up: Top

3 Dependencies

nbCalib uses (requires) the libraries below for its build. Beginning with version 1.0, the libraries are packaged and distributed with nbCalib.

The optional FITS interface requires external C and Perl libraries for FITS file manipulation. These are available from:

Next: , Previous: Dependencies, Up: Top

4 nbCalib Command-Line Arguments

The command line arguments to nbCalib are:

note: nbCalib command-line options may be used in any order, interspersed with non-options (filename arguments) and abbreviated to uniqueness (e.g. -h, -ver, -jitter, -K).

Next: , Previous: Arguments, Up: Top

5 nbCalib Build/Install

nbCalib uses the GNU autotools configuration system for configuration, build, and install. nbCalib requires the set of libraries contained in mscLib (see Dependencies). Previously it was necessary to install this package seperately and then define the environment variables MSCLIB (MSCPATH/lib) and MSCINCLUDE (MSCPATH/include) to point at the libraries and headers for mscLib. With V2Calib version 1.0, nbCalib, wbCalib, and mscLib are packaged and built together. It is no longer necessary to define the environment variables or use the prescribed mscSoftware tree. Users who previously used the tree may continue to do so by defining the environment variables as they have before; for more information see http://nexsci.caltech.edu/software/mscSoftwareTree/index.html. 

     > ./configure
     > make
     > make check (recommended)
     > make install

If configure finds gnuplot in your PATH it will automatically build KvisPlot (package to create standard set of plots form KI data). To stop configure from building and installing KvisPlot, pass it the –enable-kvisplot=no option: 

     > ./configure --enable-kvisplot=no

I don't like to put NExScI software in into standard /usr/local/bin (among other things this requires root access), so new in wbCalib v0.58dev the configure script attempts to autodetect the root of the NExScI software tree (MSCPATH – /home/bode/NExScISoftware in my installation), and to use this location as the installation prefix (if MSCPATH is not found the configuation reverts to the default /usr/local). As always, either default prefix can by overridden with a prefix option (/usr/myLocal in the following example) to the nbCalib configure script, as in: 

     > ./configure --prefix=/usr/myLocal
     > make
     > make check (recommended)
     > make install

Either way, nbCalib executables are placed in $prefix/bin, man pages in $prefix/man, info files in $prefix/info.

Ancillary build targets are (courtesy of GNU autotools): 

     > make check (automated test)
     > make doc
     > make clean
     > make dist
     > make distclean

Finally, several forms of the documentation are available. The default part of the build procedure should make the info format documentation (nbCalib.info), but the following additional documentation types (targets) are also available: 

     > cd doc
     > make html
     > make ps
     > make pdf

NOTE: The L2 FITS output capability requires that the CFITSIO library and the Astro::FITS::CFITSIO Perl extension be installed in your system. During installation, the configure process will determine whether or not this is the case, and if it is not, the installation will not be affected, and all other features will run. Please see http://heasarc.gsfc.nasa.gov/docs/software/fitsio/fitsio.html for information on the CFITSIO library and http://search.cpan.org/~pratzlaff/ for information on the Perl extension module.

Next: , Previous: Build, Up: Top

6 nbCalib Test Cases

nbCalib is distributed with the input data for testing (in the test subdirectory). These tests are designed to be run with the the make check target as part of the installation sequence (see Build). The test scripts run nbCalib against a pre-packaged calibration script and L-1 data in the test directory, and run the output through diff to check consistency with reference output.

What if "make check" fails?

We have noted two common types of installation problem. In the first "make check" fails due to roundoff differences between the packaging platform and your installation platform. This is usually the case when diff identified only 1 or 2 lines of non-matching output for each test. You may proceed with installation ("make install") and use.

With the second problem "make check" fails and diff seems to identify all lines from the reference test output files. The likely cause is that the runtime linker could not resolve library dependencies for the executable. We have seen this behavior with gcc3 on Solaris; One of the missing libraries is libstdc++.so. The root cause lies with the installation configuration of your gcc3.

You can check for this linking error on Solaris by typing 

     > ldd <path/to/your/V2Calib>/Calib/wbcEngine

If you see one or more libraries missing you have two options: 1) set LD_RUN_PATH then "make clean" and "make" again, or 2) set your LD_LIBRARY_PATH when executing

In order for either of these options to work, you will need the path to gcc3 libraries on your system. It some cases the gcc3 libraries and binaries will be installed subdirectories of a gcc3 package location (for example /usr/local/gcc3 might be the package location, with binaries found under /usr/local/gcc3/bin and libraries under /usr/local/gcc3/lib). In other cases binaries and libraries may be installed in generic locations (e.g. /opt/bin and /opt/lib). If you cannot find your gcc3 libraries, contact your local system administrator for help.

We recommend option 1) because it can be done once at installation and no further environment settings are required. If you choose option 2) you must set your LD_LIBRARY_PATH each time you run nbCalib or wbCalib (or you may set it in your .cshrc).

Assuming that you have found your gcc3 library path, option 1) proceeds as 

     > setenv LD_RUN_PATH <your/gcc3/libraries>               # for c-shell, OR
     > LD_RUN_PATH="/usr/local/gcc3/lib"; export LD_RUN_PATH  # for bash
     > make clean
     > make
     > make check
     > make install

Option 2) is 

     > setenv LD_LIBRARY_PATH <your/gcc3/libraries>:${$LD_LIBRARY_PATH}    # for c-shell, OR
     > LD_LIBRARY_PATH="<your/gcc3/libraries>:$LD_LIRBRARY_PATH"; export LD_LIBRARY_PATH # for bash

Next: , Previous: Test, Up: Top

7 Common Errors & Troubleshooting

Here we provide a list of common nbCalib error messages, their interpretation, and suggestions about their correction.

Next: , Previous: Common_Errors, Up: Top

8 Ancillary Scripts & Programs

nbCalib comes with several "helper" applications that make it a bit more civilized. These can be built at the users option (see Build instructions above).

Previous: Ancillary, Up: Ancillary

8.1 badScans

badScans is a Perl script that allows the user to select certain scans for exclusion in subsequent processing and analyses. badScans is a very general filter for text files: it scans a stdin input text file, and looks for lines that contain one of an input list of exclusion keys. When it finds a line that contains an exclusion key, it prepends the line with a comment character (pound/hash sign – #). The last output from badScans lists the number of exclusions found.

The usage of badScans is:

> badScans xxx.badScans < input.file > output.file

or for instance in the context of a wbCalib invocation:

> wbCalib [options] cal.myTarg data | badScans myTarg.badScans > myTarg.cal

Previous: Ancillary, Up: Ancillary

8.2 makeCalScript

makeCalScript is a Perl script that allows the user compose a calibration script given a target name and a schedule file.

The usage of makeCalScript is:

> makeCalScript target catfile > outfile

Please note that the NExScI package getCal, also contains an utility for automatic generation of calibration scripts (csAdhoc). Please refer to the getCal documentation (http://nexsci.caltech.edu/software/getCal/index.html#gcCalScript).

Previous: Ancillary, Up: Ancillary

8.3 KvisPlot

KvisPlot is a Perl package which generates a standard set of plots for KI data. KvisPlot will be automatically built and installed if gnuplot is found in your PATH during configuration.

The usage of KvisPlot is: 

     > KvisPlot -help
     usage: KvisPlot filename
       options:  -all (creates series of plots from sumfile)
                 -one (creates one plot of specified columns from any Kvis file)
                 -gif produces gif output files
                 -png produces png output files
                 -hc produces postscript output files
                 -spec (creates series of plots from specfile)
                 -anc ancfile (creates ancilliary plots, used only with all)
                 -bt btfile (creates block trending plots, used only with all)
                 -stdin (takes input data from stdin)
                 -help  (prints this message)

Next: , Previous: Ancillary, Up: Top

9 ToDo

Next: , Previous: ToDo, Up: Top

10 Copyright

nbCalib v1.4.4

Copyright 2007 California Institute of Technology. For questions or comments about this software, please contact http://nexsci.caltech.edu/contact/help_desk.html. Permission to use, copy and distribute this software and its documentation for academic and/or non-profit research purposes, without fee and without a written agreement is hereby granted, provided that the above copyright notice, this paragraph and the following three paragraphs appear in all copies.

The California Institute of Technology makes no proprietary claims to the results, prototypes, or systems supporting and/or necessary for the use of the research, results and/or prototypes for academic and/or non-profit research uses only. However, to the extent that any software and system built in collaboration with industry partners may incorporate proprietary designs of the industry partners, it is possible that certain restrictions may be imposed on the proprietary information.

In no event shall California Institute of Technology be liable to any party for direct, indirect, special, incidental or consequential damages, including lost profits, arising out of the use of this software and its documentation, even if the California Institute of Technology has been advised of the possibility of such damage.

The California Institute of Technology specifically disclaims any warranties, including the implied warranties or merchantability and fitness for a particular purpose. The software and documentation provided hereunder is on an "as is" basis, and the California Institute of Technology has no obligations to provide maintenance, support, updates, enhancements or modifications.

Previous: Copyright, Up: Top

11 References

Privacy Policy | Site Version: 0.99