%-----------------------------------------------------------------------
%;  Copyright (C) 2010
%;  Associated Universities, Inc. Washington DC, USA.
%;
%;  This program is free software; you can redistribute it and/or
%;  modify it under the terms of the GNU General Public License as
%;  published by the Free Software Foundation; either version 2 of
%;  the License, or (at your option) any later version.
%;
%;  This program is distributed in the hope that it will be useful,
%;  but WITHOUT ANY WARRANTY; without even the implied warranty of
%;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%;  GNU General Public License for more details.
%;
%;  You should have received a copy of the GNU General Public
%;  License along with this program; if not, write to the Free
%;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
%;  MA 02139, USA.
%;
%;  Correspondence concerning AIPS should be addressed as follows:
%;          Internet email: aipsmail@nrao.edu.
%;          Postal address: AIPS Project Office
%;                          National Radio Astronomy Observatory
%;                          520 Edgemont Road
%;                          Charlottesville, VA 22903-2475 USA
%-----------------------------------------------------------------------
%Body of intermediate AIPSletter for 31 December 2009 version

\documentclass[twoside]{article}
\usepackage{graphics}

\newcommand{\AIPRELEASE}{June 30, 2010}
\newcommand{\AIPVOLUME}{Volume XXX}
\newcommand{\AIPNUMBER}{Number 1}
\newcommand{\RELEASENAME}{{\tt 31DEC10}}
\newcommand{\NEWNAME}{{\tt 31DEC10}}
\newcommand{\OLDNAME}{{\tt 31DEC09}}

%macros and title page format for the \AIPS\ letter.
\input LET98.MAC
%\input psfig

\newcommand{\MYSpace}{-11pt}

\normalstyle

\section{General developments in \AIPS}

\subsection{\Aipsletter\ publication}

We have decided to discontinue paper copies of the \Aipsletter\ other
than for libraries and NRAO staff.  The \Aipsletter\ will be available
in PostScript and pdf forms as always from the web site listed above.
It will be announced in the NRAO e-News mailing and on the bananas
list server.

\subsection{Current and future releases}

We have formal \AIPS\ releases on an annual basis.  While all
architectures can do a full installation from the source files,
Linux (32- and 64-bit), Solaris, and MacIntosh OS/X (PPC and Intel)
systems may install binary versions of recent releases.  The last,
frozen release is called \OLDNAME\ while \RELEASENAME\ remains under
active development.  You may fetch and install a copy of these
versions at any time using {\it anonymous} {\tt ftp} for source-only
copies and {\tt rsync} for binary copies.  This \Aipsletter\ is
intended to advise you of improvements to date in \RELEASENAME\@.
Having fetched \RELEASENAME, you may update your installation whenever
you want by running the so-called ``Midnight Job'' (MNJ) which copies
and compiles the code selectively based on the changes and
compilations we have done.  The MNJ will also update sites that have
done a binary installation.  There is a guide to the install script
and an \AIPS\ Manager FAQ page on the \AIPS\ web site.

The MNJ serves up \AIPS\ incrementally using the Unix tool {\tt cvs}
running with anonymous ftp.  The binary MNJ also uses the tool {\tt
rsync} as does the binary installation.  Linux sites will almost
certainly have {\tt cvs} installed; other sites may have installed it
along with other GNU tools.  Secondary MNJs will still be possible
using {\tt ssh} or {\tt rcp} or NFS as with previous releases.  We
have found that {\tt cvs} works very well, although it has one quirk.
If a site modifies a file locally, but in an \AIPS-standard directory,
{\tt cvs} will detect the modification and attempt to reconcile the
local version with the NRAO-supplied version.  This usually produces a
file that will not compile or run as intended.

\AIPS\ is now copyright \copyright\ 1995 through 2010 by Associated
Universities, Inc., NRAO's parent corporation, but may be made freely
available under the terms of the Free Software Foundation's General
Public License (GPL)\@.  This means that User Agreements are no longer
required, that \AIPS\ may be obtained via anonymous ftp without
contacting NRAO, and that the software may be redistributed (and/or
modified), under certain conditions.  The full text of the GPL can be
found in the \texttt{15JUL95} \Aipsletter, in each copy of \AIPS\
releases, and on the web at {\tt http://www.aips.nrao.edu/COPYING}.

\vfill\eject

\section{Improvements of interest in \RELEASENAME}

We expect to continue publishing the \Aipsletter\ approximately every
six months along with the annual releases.  Henceforth, this
publication will be primarily electronic.  There have been several
significant changes in \RELEASENAME\ in the last six months.  New
tasks include {\tt UVRFI} to apply RFI mitigation techniques rather
than simple flagging, {\tt EVASN} and {\tt EVAUV} to examine the
remains after pipeline reduction to determine how well things went,
{\tt REWAY} to estimate the weight of data from the rms found across
spectral channels within the given polarization and IF, {\tt RSPEC} to
determine the spectra of the rms in a data cube, and {\tt RLDLY} to
determine the right-left delay difference.  {\tt IMAGR} and {\tt
  MCUBE} now scale the units of each image plane to match those
implied by the image header and {\tt CONVL} now uses different
convolving beams for each channel to achieve identical resolution for
the whole image cube.  New verbs {\tt ADDDISK} and {\tt REMDISK} allow
the addition/removal of remote host disks within the local area net,
while support to run \AIPS\ batch jobs on other computers within the
local net was added.

\OLDNAME\ contains a significant change in the format of the antenna
files, which will cause older releases to do wrong things to data
touched by \OLDNAME\ or \RELEASENAME\@.  {\tt 31DEC08} contains major
changes to the display software.  Older versions may use the {\tt
  31DEC08} display ({\tt XAS}), but {\tt 31DEC08} code may not use
older versions of {\tt XAS}\@.   Magnetic tape logical unit numbers
changed with {\tt 31DEC04}\@.  You are encouraged to use a relatively
recent version of \AIPS, whilst those with EVLA data to reduce must
get the latest release.

\subsection{EVLA UV-data calibration and handling}

A great many changes have been stimulated by the commissioning effort
now underway with the new EVLA WIDAR correlator.  A new appendix (E)
to the \Cookbook\ has been written to describe EVLA data reduction.
It has been available on the web and is now incorporated into the
\AIPS\ release.  Note that this document is a work in progress and it
will of necessity evolve with time.

There are five new tasks which fit into this area.  {\tt RLDLY} is
based on {\tt FRING} but is specifically designed to solve only for
the difference in delay between the right- and left-hand
polarizations.  It replaces the horribly kludged procedure {\tt
  VLBACPOL}, but uses many of the ideas pioneered in that procedure.
It solves for the difference in delays separately for all baselines to
a user-specified reference antenna and averages the result.  It
requires a reasonable signal in the RL and LR cross-hand
polarizations, but instrumental polarization alone may provide that on
a strong source.

{\tt UVRFI} is a new task that attempts to separate the signal at the
fringe-stopping point from interfering signals at other fringe
frequencies, saving only the former.  It has had some interesting
results although real RFI as seen by the EVLA is rather complicated.
See the new \AIPS\ Memo 116 by Leonia Kogan and Fraser Owen described
elsewhere in this \AIPSLETTER.

{\tt REWAY} is a new task that began with the simple proposition that
the noise in a visibility spectrum should be able to be estimated from
the rms of the real and imaginary parts.  The new EVLA provides a lot
of spectral channels in each polarization and IF and, at the moment
anyway, no other estimate of data weight.  The result was found to be
rather noisy, so a variety of algorithms were added to use data over a
moving window in time to provide a more stable solution.  The results
have been a bit puzzling in that they appear to show real time
variability where little is expected.  What is clear is that the task
is good at detecting, and now at flagging, data with apparent rms way
below or above that expected.

{\tt EVASN} and {\tt EVAUV} are two tasks designed to evaluate the
results of a pipeline data reduction.  The former looks at the
amplitude rms and phase stability in the {\tt CL} tables used.  The
latter evaluates the signal-free rms of the image and computes the
$uv$ data with the model subtracted and divided and then does
statistics on these.  Both have been tested and improved, but neither
is yet incorporated into a released pipeline procedure.

A major bug was found in {\tt CVEL}, the task that shifts visibility
spectra for the motion of the Earth when that shift is not done
on-line.  For single-source files having an index table (which most
now do), the task only shifted the data of the first scan.  This fact
was obscured by another error which truncated the message which should
have pointed out the error.  An infinite loop of recent origin was
also corrected.  For a period of time, the shift for the VLBA was also
done incorrectly.  It should have been Earth centered and the same for
all antennas, but it was not.

A major effort by Rick Perley and Brian Butler has produced new
estimates of the spectra of the primary flux calibration sources.
These have been incorporated in {\tt SETJY} and result in rather
different fluxes for some of the sources at the higher frequencies.
Older source fluxes remain available in {\tt SETJY} if requested.
The same effort produced improved gain curves for the VLA antennas at
some of the frequencies.  These curves have been added to the standard
tables used by {\tt FILLM} and {\tt INDXR}\@.  The tasks were
corrected to use a generic antenna gain whenever an antenna-specific
gain is not available.  Josh Marvil has done a detailed study of
opacity as a function of frequency and his model is now incorporated
in these two tasks.  It computes a K-band opacity by the old
seasonal/weather-based methods and then uses Josh's tables to
determine the opacity at each observed IF frequency.  The model is
described in EVLA Memo 143.  {\tt INDXR} was improved to use by
default the methods now widely used elsewhere in \AIPS\ to determine
when scan breaks occur.

Traditionally, spectral-line cubes have been constructed in {\tt
  IMAGR} and {\tt MCUBE} from image planes made at the ``natural''
resolution resulting from the frequency of that plane.  However, the
cube has contained the information about the Clean beam parameters for
only one of the planes.  Thus, any conversion from brightness in
Jy/beam to flux could only be correct in one plane.  {\tt IMAGR}, {\tt
  MCUBE} and {\tt FQUBE} have been changed so that each plane is
scaled to be in Jy/beam for the beam listed in the header while the
resolution used is kept in a new {\tt CG} table.  {\tt CONVL} has been
changed to employ the {\tt CG} table to achieve the same spatial
resolution and intensity scaling for all channels of an image cube
including suitable scaling for the residuals.

\begin{description}
\myitem{CALIB} was changed to offer three modes of data averaging.
        The new mode is vector over channels at each time plus scalar
        over time.  This is suitable for data with serious atmospheric
        phase variation.
\myitem{BPASS} was changed in the interpretation of the {\tt
        BPASSPRM(5)} parameter.  Now $+1$ means no channel zero
        division, $0$ means record-by-record channel zero division in
        amplitude and phase, $-1$ means record-by-record division but
        only in phase, and $-2$ means amplitude and phase division
        using the time-averaged visibilities.
\myitem{Calibration} routines were changed to forbid calibration of
        data weights for the EVLA, at least for the moment.  They were
        also corrected to calibrate all channels needed for spectral
        smoothing rather than just {\tt BCHAN} through {\tt ECHAN}.
        Amplitudes should be corrected for delay only for any spectral
        averaging after correlation, so that correction was removed
        except for the VLBA which keeps track of such things.
\myitem{Table} copying was addressed.  When calibration is applied, a
        copy of the old {\tt CL} table 1 is made with null (1,0) gains
        to avoid having {\tt INDXR} create a new {\tt CL} table with
        opacity and other corrections.  Index tables are now never
        copied, they must be generated as the data are copied or by
        {\tt INDXR} later.
\myitem{VLANT} was changed to handle the new form of station names and
        the necessary data files for the EVLA were created and
        populated with some early values.
\myitem{FITLD} and {\tt UVLOD} were changed to flag out and report any
        visibility spectra that are pure 0.0 in value for all
        channels.  They were also changed to compute the apparent
        coordinates in all cases since they are frequently in error on
        input from other software packages.
\myitem{UVFLG} was corrected to do flagging for shadowing properly.
\myitem{FRING} now offers the option {\tt CHINC} to average groups of
        channels before fitting for delay.  Additional issues with
        times were addressed.
\myitem{LOCIT} was changed to be with respect to the longitude of the
        array, which is Greenwich for the EVLA.  Changes to name forms,
        allowing previous corrections to be read and added in, and
        doing another plot of corrections along arms were also added.
\myitem{SUFIX} was changed to allow the user to specify the details of
        the new source, allowing the ``new'' source to be one of the
        existing sources.
\myitem{ELINT} was corrected to work on 64-bit computers.
\myitem{PRTAN} was changed to handle both old and new forms of antenna
        and station names and to display missing antennas on the North
        arm in the Y plot.  It now offers the option to display the
        antenna locations a second time, changed to be with respect to
        an array center such as that previously used for the VLA.
\end{description}

\subsection{Other UV-data matters}

\begin{description}
\myitem{REAMP} was changed to allow scaling factors to depend on
        subarray permitting all subarrays to be changed to the same
        flux scale.
\myitem{CVEL} lost the flag telling it to use the center of the Earth
        for the VLBA.  This error arose from September 20, 2009 until
        April 7, 2010.
\myitem{LISTR} was changed to allow the {\tt SCAN} option to do what
        it can when source and/or index tables are absent.  Previously
        it refused.
\myitem{FITTP} had the dangerous {\tt DOSTOKES} and {\tt DONEWTAB}
        options removed.  The latter did nothing interesting.
\myitem{ACCOR} was changed to have enough memory for current arrays
        and its use was clarified in the help file.  It is needed for
        the old hardware and new software VLBA correlators.
\myitem{FITLD} was given a new option to allow users to specify the
        antenna numbering order through the adverb {\tt ANTNAME}\@.
        This applies only when reading data in FITS IDI format.
\myitem{SPLIT} and {\tt SPLAT} corrected the alternate reference pixel
        for {\tt BCHAN} twice.
\end{description}

\subsection{Image analysis matters}

\begin{description}
\myitem{RSPEC} is a new task based on {\tt ISPEC} that determines and
        plots a spectrum of the plane-by-plane rms.  The rms may be
        found by robust methods or by a more expensive median method.
\myitem{FARS} is a Faraday rotation measure synthesis task that is
        under active development.  The option to do a one-dimensional
        Clean of the Fourier transform at each spatial point,
        including convolution of the components, was added.
\myitem{RM} was changed to accept data as a cube with an {\tt FQID}
        axis, as produced by {\tt MCUBE} and {\tt FQUBE}, and a second
        solution method due to F. Zhou was added.
\myitem{GAL} was changed so that it computes residual images from
        input parameters in the same way it did when it is first
        fitted those parameters.  Output formats and defaults were
        also improved.
\myitem{SAD,} {\tt JMFIT}, and {\tt IMFIT} were given the option of a
        special compact output format used at other institutions.  The
        {\tt MF} file was given more columns to record primary beam
        and bandwidth smearing correction factors supported by these
        tasks and by {\tt MFPRT}\@.  {\tt SAD} was given further mark
        characters in its output including indicating outside primary
        beam and the help file was improved to explain them.
\myitem{LGEOM,} {\tt HGEOM}, and {\tt PGEOM} were changed to use as
        much dynamic memory as they might wish rather than having a
        finite limit.  They still blank things more than {\tt OGEOM}
        and {\tt OHGEO} since the interpolation method cannot handle
        edges and blanked input pixels well.
\myitem{IMEAN} and {\tt IMSTAT} were changed to count pixels with
        double precision floating point variables; 32-bit integers are
        not adequate to count pixels with EVLA image cubes.
\end{description}

\subsection{Display matters}

\begin{description}
\myitem{XYRATIO} was added to tasks {\tt ISPEC}, {\tt VPLOT}, {\tt
        ANBPL}, {\tt CLPLT}, {\tt CAPLT}, and {\tt UVPLT} with default
        to fill the TV on {\tt DOTV} true and 1.0 for plot files.
\myitem{POSSM} was changed to reset {\tt NPLOTS} if there are fewer
        than {\tt NPLOTS} to be plotted and to know accurately when
        the end of plotting arises.
\myitem{VPLOT} was also changed to know accurately when the last plot
        is done so that it may be labeled correctly.  It had options
        to plot scan boundaries and log of amplitude added.
\myitem{BPLOT} was corrected to skip fully flagged {\tt BP} records
        and to know accurately when it is done plotting.
\myitem{LWPLA} was changed so that plot output sizes greater than 33
        inches, i.e. posters, are possible.
\myitem{PLOTR} was given the option to fit orthogonal polynomials to
        the input data, plotting the result.
\end{description}

\subsection{System-wide matters}

\AIPS\ has long allowed users to specify {\tt da=}{\it hostname} on
the command line to include the \AIPS\ disks of {\it hostname} along
with the \AIPS\ disks of the current computer in an {\tt AIPS}
session.  New verbs {\tt ADDDISK} and {\tt REMDISK} have been added to
allow this operation dynamically within the {\tt AIPS} session.  You
may gain access to the \AIPS\ disks of the computer named in the {\tt
  REMHOST} adverb and release that use when you no longer need them.
This works only with computers named in the {\tt DADEVS} files used
when you started up {\tt AIPS}, presumably those that share the local
\AIPS\ installation.

Similarly, {\tt REMHOST} and {\tt REMQUE} adverbs were added to verb
{\tt SUBMIT} and also to {\tt UNQUE}, {\tt QUEUES}, and {\tt JOBLIST}.
This will allow the submission of batch jobs to be executed on another
computer within the local \AIPS\ installation.  This required the
creation a new script in {\tt \$AIPS\_ROOT} called {\tt START\_QMNGR}
which {\tt AIPSC} will invoke when initiating a remote batch job.
This capability was added to begin a simple capability, so far solely
in the user's hands, to run jobs in parallel.

\section{Patch Distribution for \OLDNAME}

Important bug fixes and selected improvements in \OLDNAME\ can be
downloaded via the Web beginning at:

\begin{center}
\vskip -10pt
{\tt http://www.aoc.nrao.edu/aips/patch.html}
\vskip -10pt
\end{center}

Alternatively one can use {\it anonymous} \ftp\ to the NRAO server
{\tt ftp.aoc.nrao.edu}.  Documentation about patches to a release is
placed on this site at {\tt pub/software/aips/}{\it release-name} and
the code is placed in suitable sub-directories below this.  As bugs in
\NEWNAME\ are found, they are simply corrected since \NEWNAME\ remains
under development.  Corrections and additions are made with a midnight
job rather than with manual patches.  Since we now have many binary
installations, the patch system has changed.  We now actually patch
the master version of \OLDNAME, which means that a MNJ run on
\OLDNAME\ after the patch will fetch the corrected code and/or
binaries rather than failing.  Also, installations of \OLDNAME\ after
the patch date will contain the corrected code.

The \OLDNAME\ release has had a number of important patches:
\begin{enumerate}
  \item\ {\tt IMEAN} and {\tt IMSTAT} must count pixels in double
       precision for large cubes. {\it 2010-02-05}
  \item\ {\tt DOBAND} must not use edge channels to scale weights for
       EVLA. {\it 2010-01-18}
  \item\ {\tt FITLD} messed up the polarization type code in the
       antenna file for FITS IDI files. {\it 2010-01-25}
  \item\ Amplitudes were corrected for delay corrections which should
       be done only if channel averaging is done post correlation.
       {\it 2010-02-11}
  \item\ {\tt APCAL} read every other row of the weather table and
       could go off the end. {\it 2010-02-11}
  \item\ {\tt CVEL} could go into an infinite loop and would not shift
       scans $> 1$ for single-source files having an index ({\tt NX})
       table {\it 2010-03-01}
  \item\ {\tt FITLD} issued alarming but harmless messages about
       reference date with EVN FITS-IDI data files. {\it 2010-03-05}
  \item\ {\tt UVCON} did not produce correct models from images {\it
       2010-03-17}
  \item\ {\tt SPLIT} and {\tt SPLAT} corrected the alternate reference
       pixel for {\tt BCHAN} twice {\it 2010-03-17}
  \item\ {\tt CVEL} shifted VLBA antennas wrongly, using the antenna
       location rather than the center of the Earth {\it 2010-04-05}
  \item\ {\tt INDXR} when making a new CL table 1 for VLA data made
       mistakes likely to affect P, KA, and Q bands {\it 2010-04-08}
  \item\ {\tt BPASS} aborted when trying to shift VLBA bandpasses {\it
       2010-04-21}
  \item\ {\tt UVFLG} failed to flag shadowed antennas correctly {\it
       2010-03-05 and 2010-06-23}
\end{enumerate}

\vfill\eject
\section{\AIPS\ Distribution}

We are now able to log apparent MNJ accesses and downloads of the tar
balls.  We count these by unique IP address.  Since some systems
assign the same computer different IP addresses at different times,
this will be a bit of an over-estimate of actual sites/computers.
However, a single IP address is often used to provide \AIPS\ to a
number of computers, so these numbers are probably an under-estimate
of the number of computers running current versions of \AIPS\@. In
2010, there have been a total of 1433 IP addresses so far that have
accessed the NRAO cvs master.  Each of these has at least installed
\AIPS\ and {\bf 340} appear to have run the MNJ on \RELEASENAME\ at
least occasionally.  During 2010 more than 205 IP addresses have
downloaded the frozen form of \OLDNAME, while more than 692 IP
addresses have downloaded \RELEASENAME\@.  The binary version was
accessed for installation or MNJs by 369 sites in \OLDNAME\ and 644
sites in \RELEASENAME\@.  The attached figure shows the cumulative
number of unique sites, cvs access sites, and binary and tar-ball
download sites known to us as a function of week --- so far --- in
2010.  These numbers are about the same as those reported one
year ago for last year's releases.

\centerline{\resizebox{!}{3.1in}{\includegraphics{FIG/PLOTIT10a.PS}}}

\section{Recent \AIPS\ and related Memoranda}

All \AIPS\ Memoranda are available from the \AIPS\ home page.  There
is one new memoranda in the last six months.

\begin{tabular}{lp{5.8in}}
{\bf 116} & {\bf RFI Mitigation in \AIPS; the new task UVRFI}\\
   &  Leonid Kogan \&\ F. Owen (NRAO)\\
   &  June 2010\\
   &  Recently Ramana Athrea published a new algorithm (2009) based on
      the difference at fringe rates of a source in the sky and
      ground-based RFI.  His algorithm works only for ground-based and
      constant-amplitude RFI during a solution interval. We modified
      his algorithm to include a possible change of the RFI's
      amplitude during the solution interval and developed another
      algorithm based on H\"{o}gbom CLEANing of the Fourier transform
      of the time series of the SOURCE+RFI visibilities. These
      algorithms allow us to mitigate RFI originating from more than
      one source moving with different nonzero speeds relatively the
      array (e.g. ground-based and satellite-based RFI). The new
      algorithms are implemented in AIPS in the task UVRFI. The result
      of testing this task is demonstrated using the EVLA data at L
      and 4 band. It is also shown that self-averaging of RFI can
      reduce its impact on imaging even if the solution interval in
      the correlator is too small to allow self-averaging before
      imaging.
\end{tabular}

\begin{figure}
\centering
\resizebox{\hsize}{!}{\includegraphics{FIG/OrionA.eps}%
\hspace{0.5cm}\includegraphics{FIG/OrionB.eps}}
\caption{Early EVLA result: the spectrum of the hot core of Orion A at
  K band.  Three separate observations of 8192 channels each 0.125 MHz
  wide were made using 12 antennas in the D array.  Two hours total
  telescope time went into each of the two lower thirds of the
  spectrum and 1 hour was used for the highest third.  The plot was
  made using {\tt ISPEC} over a 54 by 60 arc second area.  Line
  identifications provided by Karl Menten.}
\label{fig:OrionKband}
\end{figure}

%\hphantom{.}
%\vfill
%\centerline{This page deliberately left blank}
\vfill
\eject

% mailer page
% \cleardoublepage
\pagestyle{empty}
 \vbox to 4.4in{
  \vspace{12pt}
%  \vfill
\centerline{\resizebox{!}{3.2in}{\includegraphics{FIG/Mandrill.eps}}}
%  \centerline{\rotatebox{-90}{\resizebox{!}{3.5in}{%
%  \includegraphics{FIG/Mandrill.color.plt}}}}
  \vspace{12pt}
  \centerline{{\huge \tt \AIPRELEASE}}
  \vspace{12pt}
  \vfill}
\phantom{...}
\centerline{\resizebox{!}{!}{\includegraphics{FIG/AIPSLETS.PS}}}

\end{document}
