%-----------------------------------------------------------------------
%;  Copyright (C) 2017-2018, 2024
%;  Associated Universities, Inc. Washington DC, USA.
%;
%;  This program is free software; you can redistribute it and/or
%;  modify it under the terms of the GNU General Public License as
%;  published by the Free Software Foundation; either version 2 of
%;  the License, or (at your option) any later version.
%;
%;  This program is distributed in the hope that it will be useful,
%;  but WITHOUT ANY WARRANTY; without even the implied warranty of
%;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%;  GNU General Public License for more details.
%;
%;  You should have received a copy of the GNU General Public
%;  License along with this program; if not, write to the Free
%;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
%;  MA 02139, USA.
%;
%;  Correspondence concerning AIPS should be addressed as follows:
%;          Internet email: aipsmail@nrao.edu.
%;          Postal address: AIPS Project Office
%;                          National Radio Astronomy Observatory
%;                          520 Edgemont Road
%;                          Charlottesville, VA 22903-2475 USA
%-----------------------------------------------------------------------
\documentclass[twoside]{article}
\usepackage{palatino}
\renewcommand{\ttdefault}{cmtt}
% Highlight new text.
\usepackage{color}
\usepackage{alltt}
\usepackage{graphicx,xspace,wrapfig}
\usepackage{pstricks}  % added by Greisen
\definecolor{hicol}{rgb}{0.7,0.1,0.1}
\definecolor{mecol}{rgb}{0.2,0.2,0.8}
\definecolor{excol}{rgb}{0.1,0.6,0.1}
\newcommand{\Hi}[1]{\textcolor{hicol}{#1}}
%\newcommand{\Hi}[1]{\textcolor{black}{#1}}
\newcommand{\Me}[1]{\textcolor{mecol}{#1}}
%\newcommand{\Me}[1]{\textcolor{black}{#1}}
\newcommand{\Ex}[1]{\textcolor{excol}{#1}}
%\newcommand{\Ex}[1]{\textcolor{black}{#1}}
\newcommand{\No}[1]{\textcolor{black}{#1}}
\newcommand{\hicol}{\color{hicol}}
%\newcommand{\hicol}{\color{black}}
\newcommand{\mecol}{\color{mecol}}
%\newcommand{\mecol}{\color{black}}
\newcommand{\excol}{\color{excol}}
%\newcommand{\excol}{\color{black}}
\newcommand{\hblack}{\color{black}}
%
\newcommand{\AIPS}{{$\cal AIPS\/$}}
\newcommand{\eg}{{\it e.g.},}
\newcommand{\ie}{{\it i.e.},}
\newcommand{\etal}{{\it et al.}}
\newcommand{\tablerowgapbefore}{-1ex}
\newcommand{\tablerowgapafter}{1ex}
\newcommand{\keyw}[1]{\hbox{{\tt #1}}}
\newcommand{\sub}[1]{_\mathrm{#1}}
\newcommand{\degr}{^{\circ}}
\newcommand{\vv}{v}
%\newcommand{\vv}{\varv}
\newcommand{\eq}{\hbox{\hspace{0.6em}=\hspace{0.6em}}}
\newcommand{\newfig}[2]{\includegraphics[width=#1]{data.fig#2}}
%\newcommand{\putfig}[1]{\includegraphics{data.fig#1.eps}}
\newcommand{\putfig}[1]{\includegraphics{#1.eps}}
\newcommand{\whatmem}{\AIPS\ Memo \memnum}
\newcommand{\boxit}[3]{\vbox{\hrule height#1\hbox{\vrule width#1\kern#2%
\vbox{\kern#2{#3}\kern#2}\kern#2\vrule width#1}\hrule height#1}}
%
\newcommand{\memnum}{123 \Hi{revised}}
\newcommand{\memtit}{New Pulse-cal Capabilities for VLBI in \AIPS}
\title{
   \vskip -35pt
   \fbox{{\large\whatmem}} \\
   \vskip 28pt
   \vskip 10pt
%   \fbox{{\Huge \Me{D R A F T}}}
%   \vskip 10pt
   \memtit \\}
\author{Eric W. Greisen}
%
\parskip 4mm
\linewidth 6.5in                     % was 6.5
\textwidth 6.5in                     % text width excluding margin 6.5
\textheight 9.0 in                  % was 8.81
\marginparsep 0in
\oddsidemargin .25in                 % EWG from -.25
\evensidemargin -.25in
\topmargin -0.4in
%\topmargin 0.2in
\headsep 0.25in
\headheight 0.25in
\parindent 0in
\newcommand{\normalstyle}{\baselineskip 4mm \parskip 2mm \normalsize}
\newcommand{\tablestyle}{\baselineskip 2mm \parskip 1mm \small }
%
%
\begin{document}

\pagestyle{myheadings}
\thispagestyle{empty}

\newcommand{\Rheading}{\whatmem \hfill \memtit \hfill Page~~}
\newcommand{\Lheading}{~~Page \hfill \memtit \hfill \whatmem}
\markboth{\Lheading}{\Rheading}
%
\vskip -.5cm
\pretolerance 10000
\listparindent 0cm
\labelsep 0cm
%
%

\vskip -30pt
\maketitle

\normalstyle
\begin{abstract}
The DiFX correlator has acquired the capability of measuring and
recording large numbers of pulse-cal tones, typically at intervals of
1 MHz through each spectral window.  These pulse-cal tones have very
high signal-to-noise, but show a roll-off at the edges of each
spectral window and usually have one or more single channels which
have amplitude and residual phase differing significantly from the
average for the spectral window.  \AIPS\ has acquired the ability to
deal with these data including new tasks to display and edit them and
to derive calibration from them.  \Hi{This memo was revised in December
  2024 to add additional descriptions of typical many-tone {\tt PC}
  tables.}
\end{abstract}

\renewcommand{\floatpagefraction}{0.75}
\typeout{bottomnumber = \arabic{bottomnumber} \bottomfraction}
\typeout{topnumber = \arabic{topnumber} \topfraction}
\typeout{totalnumber = \arabic{totalnumber} \textfraction\ \floatpagefraction}

\section{Introduction}

Pulse cal tones have been measured and recorded for a long time in
continuum VLBI observations.  Typically, there have been only a few
tones recorded.  \AIPS\ task {\tt PCCOR} would use only two tones to
measure a delay for the particular antenna.  A simplified delay
solution on the visibilities of a strong calibration scan is used to
remove the lobe ambiguities from the pulse-cal values.  The DiFX
software correlator has changed this.  DiFX can measure and record a
great many pulse-cal tones, typically one each MHz across each
spectral window.  These data are not typically written in the FITS-IDI
data set as yet, but are recorded in text files available to the user.
This will probably change to have the new, large PC tables written in
the FITS-IDI file normally.  The pulse-cal data show a bandpass-like
roll-off at the edges of each spectral window.  They also, typically,
have a single interior channel which has amplitude and phase
systematically different from the average amplitude and residual phase
of the spectral window.  Furthermore, there are times where the
pulse-cal amplitudes are abnormally low (and sometimes high) compared
to the average over the observing run or when the rms of the amplitude
in the spectral window is abnormally high or low.  It has been found
that editing the {\tt PC} table for these defects is necessary before
using the data for calibration.

\AIPS\ has acquired a number of new and revised tasks to handle the
large number of tones in the new {\tt PC} tables.  If the many-tone
pulse-cal data did not appear in the FITS-IDI file, they may be found
in text files recorded with the data.  {\tt PCLOD} can read in such
text files to make a large {\tt PC} table.  Then {\tt PCPLT} will plot
such data, {\tt PCEDT} and {\tt PCFLG} will edit the pulse-cal data
interactively, {\tt PCAVG} will average them over time, {\tt PCFIT}
will write an {\tt SN} table with changes in delay and phase found in
the pulse-cal data, {\tt PCRMS} will edit them automatically from the
amplitude rms and discrepant channels found, {\tt PCHIS} will plot
histograms from text files written by {\tt PCRMS}, and {\tt PCASS}
will write a bandpass table to be applied to the visibilities from the
spectral shapes in the pulse-cal data.

This memo is intended to provide instruction in the use of these
tasks.  A ``standard practice'' for applying these tasks has yet to be
determined, although one is suggested at the end.  Note that the test
data set used in all examples in this memo has four spectral windows
each with 64 pulse-cal tones and two polarizations.  The ten antennas
of the VLBA were used.  There are about 33000 times in the input data
set (24 hours of 2-second samples observing about 75\%\ of the total
elapsed time).  This is a large data set which poses some challenges
and renders some of the tasks less than useful.  But others rise to
the challenge.  The data set consisted primarily of observations of
stronger sources.   I was able therefore to compare the results of
{\tt PCFIT} to those, lower signal-to-noise, results from {\tt FRING}
to discover that the pulse-cal tables should be able to provide
calibration of delay variations and jumps even for data sets where the
sources are not strong enough for {\tt FRING}\@.  Be careful of the
sort order. Resorting the table from antenna-time to time-antenna can
be expensive.  Each task has a required sort order and will sort the
table if necessary.  The text below will identify the required sort
orders.

\hicol
\section{The data phases and delays}

The initial version of this memo emphasized instructions on how to use
the then-new tasks in \AIPS\ and did not adequately describe the data
themselves.  The data set mentioned above is BM413Z3 observed 8--9 June,
2017.  The last revision of this memo was in November 2018, at which
time the full many-tone {\tt PC} table was still present on disk.
Unfortunately, since then, the copy of the data on disk was lost.  While
the visibility data and conventional 2-tone {\tt PC} table are available
from the archive, the many-tone text files used by {\tt PCLOD} are no
longer accessible.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCPLT.phase}}
\caption{\hicol{\tt PCPLT} of unedited {\tt PC} table showing phases at one
  single 2-second time sample.}
\label{fig:PCPLTphase}
\end{center}
\end{figure}

Therefore, this section will be based on the data set TA035I observed on
27 April 2021 at a frequency of 5 GHz.  The data are comprised of four
scans on four different sources.  There are 4 spectral windows each 32
MHz wide.  Thus there are 32 tones in each spectral window and two
polarizations.  The phases before any calibration show quite a range of
apparent delays; {\tt PCFIT} reveals that these delays are slightly
different in each IF and polarization but quite different between
antennas.  Figure~\ref{fig:PCPLTphase} illustrates this.  That the
phases are quite constant over this rather short time is illustrated in
Figure~\ref{fig:PCFITdelay}.  The scan average delays found by {\tt
  PCFIT} for this data set are listed in Table~\ref{tab:PCFIT.delay}.
The changes with scan are mostly well less than 0.1 ns.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{SNPLT.delay}}
\caption{\hicol{\tt SNPLT} of {\tt PCFIT} fit delays from an unedited {\tt PC}
  table showing delays from {\tt BR}, {\tt FD}, and {\tt HN}\@.
  Spectral window one is red, 2 yellow, 3 cyan, and 4 blue.}
\label{fig:PCFITdelay}
\end{center}
\end{figure}

\begin{table}
\begin{center}
\hicol\begin{tabular}{|rr|rrrr|rrrr|}
\hline
Ant & Src & R 1 & R 2 & R 3 & R 4 & L 1 & L 2 & L 3 & L 4 \\
\hline
 1 & 1 &  112.6 &  110.6 &  110.1 &  110.8 &  113.4 &  113.6 &  109.6 &  110.9 \\
 1 & 2 &  112.6 &  110.6 &  110.1 &  110.8 &  113.4 &  113.6 &  109.6 &  110.9 \\
 1 & 3 &  112.6 &  110.6 &  110.1 &  110.8 &  113.4 &  113.6 &  109.6 &  110.9 \\
 1 & 4 &  112.6 &  110.6 &  110.1 &  110.8 &  113.4 &  113.7 &  109.6 &  111.0 \\
\hline
 2 & 1 & -166.2 & -167.8 & -170.1 & -168.7 & -171.3 & -169.7 & -170.6 & --169.9 \\
 2 & 2 & -166.2 & -167.7 & -170.1 & -168.7 & -171.4 & -169.7 & -170.6 & --169.9 \\
 2 & 3 & -166.2 & -167.7 & -170.1 & -168.7 & -171.4 & -169.7 & -170.5 & --169.9 \\
 2 & 4 & -166.2 & -167.7 & -170.1 & -168.7 & -171.4 & -169.7 & -170.6 & --169.9 \\
\hline
 3 & 1 & -402.1 & -405.3 & -406.6 & -406.1 & -406.1 & -405.1 & -408.7 & -409.1 \\
 3 & 2 & -402.1 & -405.3 & -406.6 & -406.0 & -406.0 & -405.1 & -408.8 & -409.2 \\
 3 & 3 & -402.1 & -405.3 & -406.6 & -406.0 & -406.0 & -405.1 & -408.8 & -409.2 \\
 3 & 4 & -402.1 & -405.3 & -406.6 & -406.0 & -406.1 & -405.1 & -408.8 & -409.2 \\
\hline
 4 & 1 &   87.1 &   85.1 &   84.5 &   84.6 &   87.3 &   86.2 &   83.6 &   80.9 \\
 4 & 2 &   87.1 &   85.1 &   84.5 &   84.6 &   87.3 &   86.2 &   83.6 &   80.9 \\
 4 & 3 &   87.1 &   85.1 &   84.5 &   84.6 &   87.3 &   86.1 &   83.6 &   80.9 \\
 4 & 4 &   87.1 &   85.1 &   84.5 &   84.6 &   87.2 &   86.0 &   83.6 &   81.0 \\
\hline
 5 & 1 &  -43.0 &  -45.8 &  -46.5 &  -45.8 &  -40.9 &  -45.7 &  -44.1 &  -41.9 \\
 5 & 2 &  -43.0 &  -45.8 &  -46.5 &  -45.8 &  -40.9 &  -45.7 &  -44.1 &  -41.9 \\
 5 & 3 &  -43.0 &  -45.8 &  -46.5 &  -45.8 &  -40.9 &  -45.7 &  -44.1 &  -41.9 \\
 5 & 4 &  -43.0 &  -45.8 &  -46.5 &  -45.8 &  -40.9 &  -45.7 &  -44.1 &  -41.9 \\
\hline
 6 & 1 &  119.3 &  117.7 &  116.3 &  116.2 &  119.0 &  116.9 &  111.1 &  118.5 \\
 6 & 2 &  119.3 &  117.7 &  116.3 &  116.2 &  119.1 &  117.0 &  111.0 &  118.4 \\
 6 & 3 &  119.3 &  117.7 &  116.3 &  116.2 &  119.1 &  117.0 &  111.0 &  118.4 \\
 6 & 4 &  119.3 &  117.7 &  116.3 &  116.2 &  119.1 &  116.9 &  111.0 &  118.5 \\
\hline
 7 & 1 &  217.4 &  215.6 &  215.1 &  214.7 &  216.4 &  219.3 &  215.6 &  219.8 \\
 7 & 2 &  217.4 &  215.6 &  215.1 &  214.7 &  216.4 &  219.3 &  215.6 &  219.8 \\
 7 & 3 &  217.4 &  215.6 &  215.1 &  214.7 &  216.4 &  219.3 &  215.7 &  219.8 \\
 7 & 4 &  217.4 &  215.6 &  215.1 &  214.7 &  216.4 &  219.4 &  215.7 &  219.8 \\
\hline
 8 & 1 &  172.9 &  170.0 &  170.6 &  171.3 &  165.9 &  170.3 &  168.9 &  169.5 \\
 8 & 2 &  172.9 &  170.0 &  170.6 &  171.3 &  165.8 &  170.4 &  169.0 &  169.6 \\
 8 & 3 &  172.9 &  170.0 &  170.6 &  171.3 &  165.8 &  170.4 &  169.0 &  169.6 \\
 8 & 4 &  172.9 &  170.0 &  170.6 &  171.3 &  165.8 &  170.3 &  169.0 &  169.6 \\
\hline
 9 & 1 & -421.5 & -423.8 & -424.4 & -423.9 & -414.5 & -418.5 & -417.2 & -413.1 \\
 9 & 2 & -421.5 & -423.8 & -424.4 & -423.9 & -414.5 & -418.5 & -417.2 & -413.1 \\
 9 & 3 & -421.5 & -423.8 & -424.4 & -423.9 & -414.5 & -418.5 & -417.2 & -413.1 \\
 9 & 4 & -421.5 & -423.8 & -424.4 & -423.9 & -414.5 & -418.6 & -417.1 & -413.1 \\
\hline
10 & 1 & -401.1 & -403.1 & -404.3 & -404.0 & -409.6 & -407.7 & -409.4 & -404.1 \\
10 & 2 & -401.1 & -403.1 & -404.3 & -404.0 & -409.6 & -407.7 & -409.3 & -404.1 \\
10 & 3 & -401.1 & -403.1 & -404.3 & -404.0 & -409.5 & -407.7 & -409.3 & -404.3 \\
10 & 4 & -401.1 & -403.0 & -404.3 & -404.0 & -409.5 & -407.7 & -409.3 & -404.4 \\
\hline
\end{tabular}
\caption{{\tt PCFIT} scan average delays in nanoseconds.  The stated
  delay errors are less than or about 0.1 ns in most cases.}
\label{tab:PCFIT.delay}
\end{center}
\end{table}

The two {\tt PC} tables provided with these data are somewhat different
in more ways than just the number of tones.  The two-tone data are
pre-averaged to 30 seconds centered on 15 and 45 seconds.  They are at
frequencies matching tones 5 and 27 in the many-tone data set.  The
many-tone data are averaged to 2 seconds centered on 1, 3, 5, and so
forth.  The values appear to be different, but applying {\tt PCAVG} to
the 2-second data produced pulse-cal values that matched the 2-tone data
in phase but with amplitudes a factor of 2 larger.

\hblack
\section{PCLOD}

Use {\tt FITLD} to read the FITS-IDI file for your current
observations.  This will make a $uv$ data set with an index ({\tt NX})
table containing the scan structure of your data and, at present, a
small {\tt PC} table containing at most a few tones.  It is possible
that this will change and the {\tt PC} table will already contain all
available tones.  If it does, then {\tt PCLOD} is not needed.  If it
does not, then you will need to acquire a text file (or files)
containing the pulse-cal data.  The current situation is somewhat
cumbersome and not directly available to the normal user.  The data
reside in numerous text files located in Socorro.  These files can be
made available by contacting the VLBA data analysts to those needing
the information for testing.  You may read each of these {\tt PCAL}
text files into \AIPS\ with {\tt PCLOD}, setting {\tt OUTVER} to a
constant value so that all data end up in the same table.
Alternatively, you may concatenate many of these text files together
into one, possibly large, text file and then read that file with {\tt
  PCLOD}\@.

{\tt PCLOD} offers the {\tt SUBARRAY} option, but it is recommended
that you read in the pulse-cal data before possibly separating the
data into multiple subarrays (if that is needed).  The {\tt DOKEEP}
option allows you to read in pulse-cal data that occur outside the
scans currently shown in the index table.  There may be reasons to
invoke this option, such as a desire to look at pulse-cal data from
multiple bands while having visibilities from only one band.  However,
for calibration purposes, loading data with {\tt DOKEEP} false is to
be preferred.  Note that the $uv$ data set also provides the start
date used in computing times in the {\tt PC} table.  The {\tt PC}
table written by {\tt PCLOD} will not have a sort order specified and
may be seriously unsorted depending on the text files used.

\section{PCPLT}

{\tt PCPLT} is a task similar to {\tt BPLOT} to plot pulse-cal spectra
graphically.  Each page of output contains either all antennas at a
single time ({\tt SORT = 'TA'}) or all selected times for a single
antenna (any other value of {\tt SORT})\@.  For a small pulse-cal
table, this might be quite useful and may be used on large tables with
{\tt DOTV = 1} to get some sense of the contents of the table before
editing.  Note that this task will sort the input {\tt PC} table to
the specified {\tt SORT}, either {\tt 'AT'} or {\tt 'TA'}\@.  The two
possible plot types are illustrated in Figure~\ref{fig:PCPLT}.

\begin{figure}
\begin{center}
\centerline{\resizebox{3.65in}{!}{\putfig{PCPLT.at}}\hskip 1em%
   \resizebox{3.3in}{!}{\putfig{PCPLT.ta}}}
\caption{{\tt PCPLT} of unedited {\tt PC} table; left: a few times for
  antenna 1, ({\tt SORT='AT'}), right: single time for all antennas
  ({SORT='TA'}).}
\label{fig:PCPLT}
\end{center}
\end{figure}

\section{PCEDT}

{\tt PCEDT} is a graphical editor very similar to {\tt BPEDT} and not
unlike {\tt EDITA} and {\tt EDITR}\@.  The principal difference is
that it is designed to flag times and channels in a {\tt PC} table,
writing a new and improved table, rather than flagging the visibility
data.  In this way it is similar to {\tt SNEDT}\@.  Because {\tt
  PCEDT} displays one antenna and time to be edited, it is more
suitable for small pulse-cal tables or for a quick examination of a
larger table.  Nonetheless, for completeness, it will be described
here.

{\tt PCEDT} requires that the output {\tt PC} table be in {\tt 'TA'}
sort order.  Figure~\ref{fig:PCEDT} shows a typical screen shot seen
when running {\tt PCEDT}\@.  The larger area at the bottom in yellow
is the area in which editing may be performed and currently shows
pulse-cal amplitude.  The smaller area above, also in yellow, shows a
second physical parameter for the same antenna, in this case the
residual phase after a large delay has been solved and removed.  The
amplitude spectra for two other antennas are shown in the green
colored panels above.  The display is for a single time.

You select a particular option by moving the cursor to that menu
option (it will change color) and hitting ``buttons'' {\tt A}, {\tt
  B}, or {\tt C}\@.  Button {\tt D} causes a display in your terminal
window of some helpful information about the option.  When you are
expected to enter numeric values, a prompt will appear in that
terminal window and you type in the values there.

The left-hand menu begins with a choice of flagging methods.  You can
flag one channel at a time, a range of channels, all values below or
above an interactive level, all points with rectangular areas, or one
point at a time with or without a button push.  You can enter the
display range for all three types of display.  You can list the
current flag commands, undo some of the flag commands, and then redo
the remaining ones.  You can recompute the delays and residual phases
taking into account the flagged samples.  You can set a screen zoom
for use during flagging and reset it.  You can pause the automatic
updating of the screen and then resume it and you can re-plot the
current screen taking into account any new flags.  You can exit {\tt
  PCEDT} applying the flags and keeping the output {\tt PC} table or
you can ``abort'' deleting the flag information and the new output
table.

The right hand menu allows you to switch between displayed
polarizations.  It allows you to change which data are flagged when a
flag is generated, one or all polarizations, times, sources, and
antennas.  It allows you to enter the next antenna you wish to edit or
simply select the next one.  You can plot all {\tt PC} tones or
interactively select a smaller range of these channels or select a
single IF (spectral window) or the next IF or previous IF\@.  You can
move to the next or previous time for the current antenna.  And you
can choose what parameter appears in the edit window (amplitude, input
phase, or residual phase) and what parameter appears in the second,
comparison window.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCEDT}}
\caption{{\tt PCEDT} of unedited {\tt PC} table; antenna 1 is being
  edited in amplitude with residual phase also shown.  Amplitude
  spectra for antennas 2 and 3 are shown for comparison.  Any flags
  generated will apply only to one time, one polarization, and one
  antenna.}
\label{fig:PCEDT}
\end{center}
\end{figure}

\section{PCFLG}

Unless you wish to trust an automatic editor (see {\tt PCRMS} below),
the task you will use on any good-sized {\tt PC} table is {\tt
  PCFLG}\@.  It is a task much like {\tt SPFLG} in that it displays
pulse-cal amplitude, input phase, or residual phase as a grey-scale
image with time on the vertical axis and pulse-cal tone (channel) on
the horizontal axis.  If there are more times or channels than will
fit on the TV display, the data are averaged over time and every
$n^{\rm th}$ channel is displayed.  One may then select windows in
time and in channel and have every time and channel be displayed.
The example data set fits well on the screen in channels, but grossly
overruns the time axis.  One can then display one ``piece'' of the
time axis at a time and step through the pieces.  Note that {\tt
  PCFLG} will copy the input table to a new output table, sorting if
the input is not in {\tt 'TA'} order.

With the 24-hour example {\tt PC} table, {\tt PCFLG} finds 36503
separate times to grid from 325 scans.  It chooses an averaging
interval of 53, but, because of scan breaks, this results in an image
with 1828 rows which is much too big for my display.  It is necessary
to first enter a much larger smooth time (\eg\ 250) and re-load the
image.  Then set a small vertical window and load it, select a smooth
time of one, and re-load.  Turning on axis labeling, we have the
image shown in Figure~\ref{fig:PCFLG}.  Note the low amplitudes at the
edges of each spectral window and in the initial 5 times.  Note also
the discrepant amplitudes in channel 40 of IF 1 and channel 16 of the
remaining IFs.  We should flag the edge and discrepant channels and
also the times of low amplitude.

The first column of the menu may contain
\begin{center}
\begin{tabular}{|l|l|}\hline
{\tt OFFZOOM    } &   turn off any zoom magnification \\
{\tt OFFTRANS   } &   turn of any black \&\ white enhancement\\
{\tt OFFCOLOR   } &   turn of any pseudo-coloring\\
{\tt TVFIDDLE   } &   allows zoom, pseudo-color contours or black
                      \&\ white enhancement\\
{\tt TVTRANSF   } &   black and white enhancement as in {\tt AIPS}\\
{\tt TVPSEUDO   } &   many pseudo-colorings as in {\tt AIPS}\\
{\tt DO WEDGE ? } &   switches choice of displaying a step wedge\\
{\tt LOAD $xxxx$ } &  switch TV load transfer function to $xxxx$\\
{\tt LIST FLAGS } &   list selected range of flag commands\\
{\tt UNDO FLAGS } &   remove flags by number from {\tt FC} table and
                      from the master grid\\
{\tt REDO FLAGS } &   reapply all current flags to master grid\\
{\tt REDO DELAY } &   recompute the fits for delays and center phases and
                      recompute the\\
                  &   corrected (residual) {\tt PC} values\\
{\tt VIEW RESID } &   to switch from viewing the input data to viewing the
                      residual data\\
                  &   (only phase differs)\\
{\tt VIEW INPUT } &   to switch from viewing the residual data to viewing
                      the input data\\
                  &   (only phase differs)\\
{\tt DO LABEL ? } &   turns on/off axis labeling
\\ \hline
\end{tabular}
\end{center}
Note: when a flag is undone, all cells in the master grid which were
first flagged by that command are restored to use.  Flag commands done
after the one that was undone may also, however, have applied to some of
those cells.  To check this and correct any improperly unflagged pixels,
use the {\tt REDO FLAGS} option. This option even redoes {\tt CLIP}
operations!  After an {\tt UNDO} or {\tt REDO FLAGS} operation, the TV
is automatically reloaded if needed. Note that the {\tt UNDO}
operation is one that reads and writes the full master grid.

The load to the TV for all non-phase displays may be done with all
standard transfer functions: {\tt LIN}ear, {\tt LOG}, {\tt SQRT}, and
{\tt LOG2} (more extreme log).  The menu shows the next one in the
list through which you may cycle.  The TV is reloaded immediately when
a new transfer function is selected.  Only one of the {\tt VIEW} {\it
  yyyyy} options will appear, depending on which form you are
currently viewing.  Note that the two forms differ only in phase, but
that that affects vector RMS and amplitude of the vector difference as
well as the phase display types.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCFLG}}
\caption{{\tt PCFLG} of unedited {\tt PC} table; antenna 1 is being
  edited in amplitude.  The averaging over time has been turned off
  and only a small piece of the total time range is displayed.}
\label{fig:PCFLG}
\end{center}
\end{figure}

Column 2 offers type-in controls of the TV display and controls of which
data are to be flagged.  In general, the master grid may be too large
to display on the TV screen in its entirety.  The program begins by
loading every $n^{\rm th}$ channel and time smoothing by $m$ time
intervals in order to fit the full image on the screen.  However, you
may select a sub-window in order to see the data in more detail.  You
may also control the range of intensities displayed (like the adverb
{\tt PIXRANGE} in {\tt TVLOD} inside {\tt AIPS}).  The averaging time
to smooth the data for the TV display may be chosen, as may the
averaging time for the ``scan average'' used in some of the displays.
Which correlators are to be flagged by the next flagging command may
be typed in.  All of the standard {\tt STOKES} values, plus any 4-bit
mask may be entered (see HELP file above).  The antenna number may be
typed in.  Flagging may be done only for the current antenna and IF
and source, or it may be done for all antennas and/or IFs and/or
sources.  Note that these controls affect the next {\tt LOAD}s to the
TV or the flagging commands prepared after the parameter is changed.
When the menu of options is displayed at the top of the TV, the
current selections are shown along the bottom.  If some will change on
the next load, they are shown with an asterisk following.

\begin{center}
\begin{tabular}{|l|l|}\hline
{\tt ENTER BLC          } & Type in a BLC on the terminal\\
{\tt ENTER TRC          } & Type in a TRC on the terminal\\
{\tt ENTER AMP PIXRANGE } & Type in the intensity range to be used for
                         loading amplitude images to the TV\\
{\tt ENTER PHS PIXRANGE } & Type in the phase range to be used for loading
                         phase images to the TV\\
{\tt ENTER RMS PIXRANGE } & Type in the intensity range to be used for
                         loading images of the rms to the TV\\
{\tt ENTER R/M PIXRANGE } & Type in the value range to be used for loading
                         rms/mean images to the TV\\
{\tt ENTER SMOOTH TIME  } & To enter the time smoothing length in units of
                         the master grid cell size\\
{\tt ENTER SCAN TIME    } & To enter the time averaging length for the
                         "scan average" in units of the master\\
                          & grid cell size\\
{\tt ENTER ANTENNA      } & To enter a desired antenna using the terminal\\
{\tt ENTER STOKES FLAG  } & To type in the 4-character string which will
                         control which correlators\\
                          & (polarizations) are flagged.  Note: this
                         will apply only to subsequent flagging\\
                          & commands. It will change whenever a
                          different Stokes is displayed.\\
{\tt ENTER CH SMOOTH    } & To type in the FWHM of a Gaussian smoothing
                         in spectral channels in the\\
                          & data type being loaded.\\
{\tt SWITCH SOURCE FLAG } & To switch between having all sources flagged by
                         the current flag commands\\
                          & and having only those sources included in this
                          execution of {\tt PCFLG} flagged.\\
{\tt SWITCH ALLANT FLAG } & To switch between flagging only the current
                         antenna and flagging all antennas.\\
{\tt SWITCH ALL-IF FLAG } & To rotate the flag all IFs status from one IF
                         to a range of IFs, to all IFs.  Applies\\
                          & to subsequent flag commands.\\ \hline
\end{tabular}
\end{center}

The third column of options is used to control which data are displayed
and to cause the TV display to be updated.  The master grid must be
converted from complex to amplitude or phase for display.  Using
either scalar or vector averaging, it may be converted to the rms of
the amplitude or the rms divided by the mean of the amplitude.  It may
also be converted to the amplitude of the vector difference between
the current observation and the "scan average" as defined by the
selected ``scan'' time, or the absolute value of the difference in
amplitude with the scalar-average amplitude or the absolute value of
the difference in phase with the vector scan average.  This column has
the options:

\begin{center}
\begin{tabular}{|l|l|}\hline
{\tt DISPLAY AMPLITUDE  } & To display amplitudes on the TV\\
{\tt DISPLAY PHASE      } & To display phases on the TV\\
{\tt DISPLAY RMS        } & To display scalar amplitude rms on the TV\\
{\tt DISPLAY RMS/MEAN   } & To display scalar amplitude rms/mean on the TV\\
{\tt DISPLAY VECT RMS   } & To display vector amplitude rms on the TV\\
{\tt DISPLAY VRMS/VAVG  } & To display vector amplitude rms/mean on the TV\\
{\tt DISPLAY AMP V DIFF } & To display the amplitude of the difference
                         between the data and a running\\
                          &  (vector) "scan average"\\
{\tt DISPLAY AMPL DIFF  } & To display the abs(difference) of the amplitude
                         of the data and a running\\
                          & scalar average of the amplitudes in the "scan"\\
{\tt DISPLAY PHASE DIFF } & To display the abs(difference) of the phase of
                         the data and the phase of a\\
                          & running (vector) "scan average"\\
{\tt DISPLAY STOKES xx  } & To switch to Stokes type xx (where xx can be
                         RR, LL).\\
{\tt OFF WINDOW + LOAD  } & Reset the window to the full image and reload
                         the TV\\
{\tt SET WINDOW + LOAD  } & Interactive window setting (like TVWINDOW)
                         followed by reloading the TV.\\
{\tt LOAD LAST ANTENNA  } & Reload TV with the current parameters and the
                         previous antenna in sequence.\\
{\tt LOAD NEXT ANTENNA  } & Reload TV with the current parameters and the
                         next antenna in sequence.\\
{\tt LOAD LAST PIECE    } & Load the previous overlapping piece of the
                         data\\
{\tt LOAD NEXT PIECE    } & Load the next overlapping piece of the data\\
{\tt LOAD               } & Reload TV with the current parameters.\\ \hline
\end{tabular}
\end{center}
{\tt SET WINDOW + LOAD} is "smarter" than {\tt TVWINDOW} and will not
let you set a window larger than the basic image.  Therefore, if you
wish to include all pixels on some axis, move the TV cursor outside
the image in that direction.  The selected window will be shown.  {\tt
  LOAD LAST PIECE} and {\tt LOAD NEXT PIECE} appear only when there
are too many times to fit on the TV screen at the current smoothing
parameter.  It lets you load one piece at a time in sequence.  The
pieces will overlap somewhat.

The fourth column is used to select the type of flagging to be done.
During flagging, a TV graphics plane is used to display the current
pixel much like {\tt CURVALUE} in {\tt AIPS}\@.  Buttons {\tt A} and
{\tt B} do the flagging (except {\tt A} switches corners for the area
and time range modes).  Button {\tt C} also does the flagging, but the
program then returns to the main menu rather than prompting for more
flagging selections.  Button {\tt D} exits back to the menu without
doing any additional flagging.  Another graphics plane is used to show
the current area/time/antenna being flagged.  All flagging commands
can create zero, one, two, or more entries in the flagging list; hit
button {\tt D} at any time.  There are also two clipping modes, an
interactive one and one in which the user enters the clip limits from
the terminal.  In both, the current image computed for the TV (with
user-set windows and data type, but not any other windows or alternate
pixels etc. required to fit the image on the TV) is examined for
pixels which fall outside the allowed intensity range.  Flagging
commands are prepared and the master file blanked for all such pixels.
In the interactive mode, buttons {\tt A} and {\tt B} switch between
setting the lower and upper clip limits, button {\tt C} causes the
clipping to occur followed by a return to the main menu, and button
{\tt D} exits to the menu with no flagging.  The options are
%\vfill\eject

\begin{center}
\begin{tabular}{|l|l|}\hline
{\tt FLAG PIXEL      } & To flag single pixels\\
{\tt FLAG/CONFIRM    } & To flag single pixels, but request a yes or no on
                      the terminal before proceeding\\
{\tt FLAG AREA       } & To flag a rectangular area in Channel-T\\
{\tt FLAG TIME RANGE } & To flag all channels for a range of times\\
{\tt FLAG CHANNEL-DT } & To flag a channel for a range of times\\
{\tt FLAG TIME       } & To flag all channels for a specific time\\
{\tt FLAG A CHANNEL  } & To flag all times for a specific channel\\
{\tt CLIP BY SET \#S } & To enter from the terminal a clipping range for
                      the current mode and then clip\\
{\tt CLIP INTERACTIV } & To enter with the cursor and LUTs a clipping range
                      for the current mode and\\
                       & then clip the data.\\
{\tt CLIP BY FORM    } & To clip selected antennas using the "method"
                      and clipping range of some\\
                       & previous clip operation.\\ \hline
\end{tabular}
\end{center}
The last operation allows you to apply a clipping method already used on
one antenna to other antennas and/or Stokes.  {\tt CLIP BY FORM} asks
for a command number (use {\tt LIST FLAGS}) and applies its display
type (amp, phase, rms, rms/mean), averaging interval and clip levels
to a range of antennas and Stokes (as entered from the terminal).  To
terminate the operation, doing nothing, enter a letter instead of one
of the requested antenna numbers.  To omit a Stokes, reply, if
requested for a flag pattern, with a blank line.  You may watch the
operation being carried out on the TV as it proceeds.

The final column selects an exit from the task.  If flag commands were
prepared, the task will ask if you wish to apply them.  Answer ``no''
twice to cause all editing to be abandoned and the grid file deleted
(unless you selected {\tt DOCAT} true when starting {\tt PCFLG})\@.
In that case, you may restart {\tt PCFLG} specifying the cataloged
work file to resume editing where you left off.

\section{PCAVG}

{\tt PCAVG} sorts the input data set into antenna-time order if needed
and then writes out a new {\tt PC} table averaging over time while
honoring scan breaks.  You may choose to do this to improve the
already very good signal-to-noise of the pulse-cal data and to reduce
the size of the pulse-cal table.  It is probably best if you do the
flagging on the input table, since you do not want to average times of
bad data with times of good data.

{\tt PCAVG} suffers from the same issues that affect all time
averaging tasks.  If {\tt SOLINT}$ > 0$ each interval begins with a
data sample for the current antenna and lasts until the first time
more than {\tt SOLINT} later (or the scan ends, the source changes,
etc.).  This means that the different antennas will end up with
different averaged times if there has been any editing or drop outs.
{\tt PCFLG} on such a data set will be (way) less than desirable.
Thus {\tt PCAVG} offers a second averaging scheme, indicated by
setting {\tt SOLINT} negative, in which each scan is broken into
intervals of approximate length {\tt ABS (SOLINT)}.  Each interval is
labeled with its central time and any data falling in the interval is
averaged.  This means that all antennas will have records at the same
times, but those records may contain differing amounts of data.  Of
course, the other averaging scheme also has the same issue. {\tt
  PCFLG} will be very much happier with the fixed interval method.

\section{PCFIT}

{\tt PCFIT} uses the same fitting routine found in {\tt PCPLT}, {\tt
  PCEDT}, and {\tt PCFLG} to fit each spectral window for a delay and
phase.  It remembers the initial delay and phase for each antenna and
IF and polarization and writes a solution ({\tt SN}) table usually
containing the differences between the currently fit values and the
initial ones.  If the delay changes by too much (defined by {\tt
  CUTOFF}), then the change is ignored by resetting the ``initial''
value to the new value after the change.  {\tt PCFIT} writes a new
{\tt PC} table containing the pulse-cal spectra after the fit delays
and phases are removed.  It does this on the cadence of the input
table, even if time averaging is used in finding the delays and
phases.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCFIT}}
\caption{{\tt PCFIT} of edited {\tt PC} table delay solutions for 5 of
  the antennas.  BR and NL (as well as FD and SC) had no jumps but
  delays still wander around smoothly with time.  All other antennas
  had jumps in delay associated with periods of time in which the
  antenna was not observing.  KP actually also had a jump of about 600
  ns which was ``ignored'' by {\tt PCFIT}\@.}
\label{fig:PCFIT}
\end{center}
\end{figure}

{\tt PCFIT} offers the option of time averaging using the irregular
grid method, since the {\tt SN} table plus {\tt CLCAL} handle that
well.  It has an option to combine spectral windows before solving,
but it is best to leave {\tt NPIECE} at 0 (solves each spectral window
separately).  If you like looking at lots of numbers, you may set {\tt
  PRTLEV} greater than zero to view solutions with error bars and even
to view the initial guesses.  The sort order used by {\tt PCFIT} is
antenna-time is {\tt SOLINT} is $> 0$ and time-antenna otherwise.

The delays written by {\tt PCFIT} in the {\tt SN} table are shown for
four representative antennas in Figure~\ref{fig:PCFIT}.

\section{PCRMS}

{\tt PCRMS} is a task designed to spare the user from the interactive
editing in {\tt PCFLG}\@.  It sorts the input {\tt PC} table into
antenna-time order.  Then, in the first pass through the input {\tt
  PC} table, the task computes the amplitude and phase statistics of
the pulse-cal values in each IF and polarization for each antenna,
after the fit delays and phases have been removed.  It then looks for
channels that deviate from the mean and channels that stick out
compared to their immediate neighbors.  If a channel sticks out from
its neighbors often enough, it is scheduled for deletion.  In the
second pass through the data, the channel in each IF, polarization,
and antenna that sticks out is flagged along with the edge channels.
The delays may be recomputed ({\tt APARM(5) > 0}) which will allow
phase statistics to be reported along with amplitude ones.  The phase
statistics are not currently used for flagging, so the expense of a
second round of delay fitting is only for information purposes.  The
average amplitude and rms over time are stored along with the rmses of
these parameters for each antenna, polarization, and IF\@.

If {\tt APARM(4)} $\leq 0$, a third pass through the input {\tt PC}
table is performed with a new {\tt PC} table being written containing
a flagged version of the input.  The average amplitude and amplitude
rms after flagging the channels that stuck out and ignoring the edge
channels and is compared to the overall average and rms for that IF,
polarization, and antenna.  Values that deviate too much ({\tt
  APARM(1)} and {\tt APARM(2)}) are flagged, edge channels are flagged
when {\tt APARM(6)} $\leq 0$, and the new, edited {\tt PC} table is
written.  At the end of the third pass, summaries of the number of
samples flagged for bad channels and the number of spectral windows
flagged for bad amplitude and rms are reported.  Adverb {\tt PRTLEV}
controls the amount of displayed information in each pass.  Leaving
the edge channels unflagged in the output is useful for the bandpass
shape task {\tt PCASS}; set {\tt APARM(6)} $> 0$.

If you would like to examine the spectral-window average amplitude and
rms for every time, {\tt PCRMS} will write a text file when {\tt
  APARM(3)} $> 0$ and {\tt OUTFILE} is specified.  The file contains
16 columns, 2 polarizations for 4 spectral windows of average
amplitude followed by the same for rms of the average.  One row is
written for every time for the specified antenna.  This file is really
too large for much human examination.  A task, described next, was
written to plot the histogram and statistics of any one of the
columns.  Such plots may help in deciding what the best values for
{\tt APARM(1)} and {\tt APARM(2)} might be in {\tt PCRMS}\@.  If you
are using this option, you may wish to avoid writing a new {\tt PC}
table by setting {\tt APARM(4)} $> 0$.

\begin{figure}
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCHIS}}
\caption{{\tt PCHIS} plot from antenna 9 (PT) spectral window 2, L
  polarization.  The plus sign identifies the mean, the interior ticks
  show $n$ times the rms from the mean (up to 5) and the labeling
  indicates that 57 times had excessively low average amplitude while
  no times had excessively large amplitude.  Note that the vertical
  axis is the log base 10 of the counts in each histogram cell.  The
  logarithm of zero is taken as -1.}
\label{fig:PCHIS}
\end{center}
\end{figure}
%\vfill\eject
\section{PCHIS}

{\tt PCHIS} will plot a histogram of the values found in one column of
a many-column numerical text file identified by adverb {\tt INTEXT}\@.
{\tt APARM(5)} identifies which column will be used for the histogram
and the number of samples deviating from the mean by more than {\tt
  APARM(6)} will be reported.  Figure~\ref{fig:PCHIS} shows a sample
plot.

\section{PCASS}

Finally, {\tt PCASS} is a task which averages pulse-cal data and
writes a bandpass calibration table.  It can use scalar or vector
averaging for the amplitude, it can compute phases or set the bandpass
phase to zero, and it can average all calibration scans together,
writing one {\tt BP} record for each antenna, or it can write separate
records for each antenna and each scan.  Calibration scans are
signified by giving the source names in the usual way, with {\tt
  CALSOUR = ' '} signifying all sources.  The output of {\tt PCRMS}
with {\tt APARM(6)} $> 0$ is recommended since bad interior channels
and bad times will be flagged but the edge channels remain to allow
fitting of the edges of the bandpass.  Note that {\tt PCASS}
interpolates the bandpass function from the frequencies of the pulse
cals to the frequencies of the visibility spectral channels.  The
signal-to-noise is excellent, as illustrated in
Figure~\ref{fig:PCASS}, but it is not clear that this bandpass shape
actually applies to the visibility data.

\begin{figure}
\begin{center}
\resizebox{4.8in}{!}{\putfig{PCASS}}
\caption{{\tt PCASS} output bandpass function for antenna 1 (BR).}
\label{fig:PCASS}
\end{center}
\end{figure}

\section{Suggestion}

A ``standard practice'' for applying these tasks needs to be devised
and vetted by VLBI scientists.  This has yet to be done.  The
author's suggestion is first to have the large pulse-cal tables
written in the FITS-IDI data files.  Executing {\tt PCRMS} twice, first
on the input {\tt PC} table and then on the output table from {\tt
 PCRMS}, should remove up to two bad channels per spectral window
and all times when the pulse-cal amplitudes are either low, high, or
erratic.  Then {\tt PCFIT}, perhaps with some time averaging, will
determine the relative delays and phases and write them into an {\tt
  SN} table. Task {\tt CLCAL} can then be used to apply the {\tt SN}
table values to the current {\tt CL} table, writing a new {\tt CL}
table.

\end{document}

\begin{figure}[ht]
\begin{center}
\resizebox{6.5in}{!}{\putfig{PCFLG.phase}}
\caption{{\tt PCFLG} of unedited {\tt PC} table showing phases on one
  antenna over time.  The color scheme is circular so that phases of
  -180 and 180 are the same color.  Antenna 1 was chosen to show
  multiple loops in phase in each spectral window but not some many as
  to render the plot less than legible (\eg\ antennas 9 or 10).}
\label{fig:PCFLGphase}
\end{center}
\end{figure}
