%-----------------------------------------------------------------------
%;  Copyright (C) 2019
%;  Associated Universities, Inc. Washington DC, USA.
%;
%;  This program is free software; you can redistribute it and/or
%;  modify it under the terms of the GNU General Public License as
%;  published by the Free Software Foundation; either version 2 of
%;  the License, or (at your option) any later version.
%;
%;  This program is distributed in the hope that it will be useful,
%;  but WITHOUT ANY WARRANTY; without even the implied warranty of
%;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%;  GNU General Public License for more details.
%;
%;  You should have received a copy of the GNU General Public
%;  License along with this program; if not, write to the Free
%;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
%;  MA 02139, USA.
%;
%;  Correspondence concerning AIPS should be addressed as follows:
%;          Internet email: aipsmail@nrao.edu.
%;          Postal address: AIPS Project Office
%;                          National Radio Astronomy Observatory
%;                          520 Edgemont Road
%;                          Charlottesville, VA 22903-2475 USA
%-----------------------------------------------------------------------
%Body of intermediate AIPSletter for 31 December 2018 version

\documentclass[twoside]{article}
\usepackage{graphics}

\newcommand{\AIPRELEASE}{June 30, 2019}
\newcommand{\AIPVOLUME}{Volume XXXIX}
\newcommand{\AIPNUMBER}{Number 1}
\newcommand{\RELEASENAME}{{\tt 31DEC19}}
\newcommand{\NEWNAME}{{\tt 31DEC19}}
\newcommand{\OLDNAME}{{\tt 31DEC18}}

%macros and title page format for the \AIPS\ letter.
\input LET98.MAC
%\input psfig

\newcommand{\MYSpace}{-11pt}

\normalstyle

\section{Happy 40$^{\rm th}$ birthday \AIPS\ and FITS}

The FITS format was devised in a meeting at the VLA site on March 27
and 28, 1979.  Don Wells (then of NOAO) and Eric Greisen (still of
NRAO) were the primary authors while Barry Clark and other NRAO staff
members provided valuable suggestions.  The FITS format, enhanced by
later extensions, is still in wide spread use in astronomy and is the
envy of other scientific fields.  Then the FITS format colored the
design of the \AIPS\ internal formats.  The \AIPS\ project was begun
officially on July 1, 1979 and has stood the test of time remarkably
well.

\subsection{\Aipsletter\ publication}

We have discontinued paper copies of the \Aipsletter\ other than for
libraries and NRAO staff.  The \Aipsletter\ will be available in
PostScript and pdf forms as always from the web site listed above.
New issues will be announced in the NRAO eNews mailing and on the
bananas and mnj list server.

\subsection{Current and future releases}

We have formal \AIPS\ releases on an annual basis.  While all
architectures can do a full installation from the source files,
Linux (32- and 64-bit), and MacIntosh OS/X (Intel) systems may install
binary versions of recent releases.  Binary versions for Solaris and
MacIntosh (PPC) are available for the {\tt 31DEC17} release but not
for later releases.  The last, ``frozen'' release is called \OLDNAME\
while \RELEASENAME\ remains under active development.  You may fetch
and install a copy of these versions at any time using {\it anonymous}
{\tt ftp} for source-only copies and {\tt rsync} for binary copies.
This \Aipsletter\ is intended to advise you of improvements to date in
\RELEASENAME\@.  Having fetched \RELEASENAME, you may update your
installation whenever you want by running the so-called ``Midnight
Job'' (MNJ) which copies and compiles the code selectively based on
the changes and compilations we have done.  The MNJ will also update
sites that have done a binary installation.  There is a guide to the
install script and an \AIPS\ Manager FAQ page on the \AIPS\ web site.

The MNJ for binary versions of \AIPS\ now uses solely the tool {\tt
  rsync} as does the initial installation.  For locally compiled
(``text'') installations, the Unix tool {\tt cvs} running with
anonymous ftp is used for the MNJ\@. Linux sites will almost
certainly have {\tt cvs} installed; but other sites may have to
install it from the web.  Secondary MNJs will still be possible using
{\tt ssh} or {\tt rcp} or NFS as with previous releases.  We have
found that {\tt cvs} works very well, although it has one quirk. If a
site modifies a file locally, but in an \AIPS-standard directory,
{\tt cvs} will detect the modification and attempt to reconcile the
local version with the NRAO-supplied version.  This usually produces a
file that will not compile or run as intended.  For local versions,
use a copy of the task and its help file in a private disk area
instead.

\AIPS\ is now copyright \copyright\ 1995 through 2019 by Associated
Universities, Inc., NRAO's parent corporation, but may be made freely
available under the terms of the Free Software Foundation's General
Public License (GPL)\@.  This means that User Agreements are no longer
required, that \AIPS\ may be obtained via anonymous ftp without
contacting NRAO, and that the software may be redistributed (and/or
modified), under certain conditions.  The full text of the GPL can be
found in the \texttt{15JUL95} \Aipsletter, in each copy of \AIPS\
releases, and on the web at {\tt http://www.aips.nrao.edu/COPYING}.


\section{Improvements of interest in \RELEASENAME}

We expect to continue publishing the \Aipsletter\ approximately every
six months, but the publication is now primarily electronic.  There
have been several significant changes in \RELEASENAME\ in the last six
months.  Some of these were in the nature of bug fixes which were
applied to \OLDNAME\ before and after it was frozen.  If you are
running \OLDNAME, be sure that it is up to date; pay attention to the
patches and run a MNJ any time a patch relevant to you appears.
There is one new task in \RELEASENAME\ called {\tt FGPRT}\@.  It
prints a matrix of the number of flags applying to each baseline.  The
counts can be restricted to those applying to selected IFs, channels,
times, sources, and the like.  There is also one new procedure called
{\tt VLBAMPHC} to calibrate and image data for secondary
phase-stopping centers.

{\tt 31DEC14} contains a change to the ``standard'' random parameters
in $uv$ data and adds columns to the {\tt SN} table.  Note, however,
that the random parameters written to FITS files have not been changed.
Older releases of \AIPS\ cannot handle the new {\it internal} $uv$
format and might be confused by the {\tt SN} table as well.  {\tt
  31DEC09} contains a significant change in the format of the antenna
files, which will cause older releases to do wrong things to data
touched by {\tt 31DEC09} and later releases.  You are encouraged to
use a relatively recent version of \AIPS, whilst those with recent VLA
data to reduce should get release \OLDNAME\ or, preferably, the latest
release.

\subsection{VLBI}

\subsubsection{EOPs}

The web site at Goddard Space Flight Center long used to provide data
on the location of the Earth's pole stopped working late last Fall.
Thanks to dedicated GSFC employees, we were able to obtain these data
by a back channel to support the VLA and VLBA correlators.  We also
made these files available on an NRAO (LBO) web page.  The standard
{\tt VLBAEOPS} procedure was changed to allow the user to provide the
data file instead of automatically downloading it.  Finally, in May, a
new web site was created to supply the EOP data files and \AIPS\
procedures were changed to use it.  That web site is\\ {\tt
\centerline{ftp://cddis.gsfc.nasa.gov/vlbi/gsfc/ancillary/solve\_apriori/usno\_finals.erp}}

Using 500-day rather than full data files during this time raised an
issue in {\tt CLCOR}\@.  The code was changed to make sure that the
observation date did not precede the valid data in the file.

\subsubsection{VLBARUN}

{\tt VLBARUN} is a procedure which invokes many other procedures in
the {\tt VLBARUN} file and in the {\tt VLBAUTIL} file to load VLBA
data to disk and then calibrate and image it.  Optionally, {\tt
  VLBARUN} creates an html file filled with plots generated during
the calibration and imaging.  It is easy to skim through this html
page looking for issues, or simply confirming that everything went
well.  Starting with the 2020A observing semester, with proposal
deadline Aug 1, 2019, NRAO will offer staff assistance to novice VLBA
users with the goal of broadening the user base and reducing the
steepness of the learning curve.  Therefore, a re-examination of
{\tt VLBARUN} was undertaken.

First, the \POPS\ language processor was given an increase in
memory by a factor of four.  This simplified the development of the
run files.  Then {\tt VLBAUTIL} was reviewed, but all it required were
changes to the typing to make the code and output messages more
readable.  Then the code of {\tt VLBARUN} was overhauled.  New
capabilities were added to allow a-priori flags to be read in at the
start from a text file and, before imaging, to run {\tt RFLAG} on each
source individually.  Several changes, reported below, were made to
{\tt RFLAG} to enhance its performance in this environment.

More technical changes made to {\tt VLBARUN} include changing the
imaging of line data to make a separate image cube for each IF and to
plot only a selected number of channels in {\tt KNTR}\@.  The
output plots and html are now written to a dated sub-directory below
the directory to which {\tt OUTFILE} points.  The messages from {\tt
  VLBARUN} are now printed to a text file which is accessible from the
{\tt vlbarun.html} file.  The background color of the gif files was
changed from black to a light grey so that they may be displayed
outside of the html environment.  The plots appear two across in the
html file, making comparisons easier and the file shorter.
%  The plots
%from {\tt RFLAG} now appear in the html file in living color.  The
The VLBA tutorial found at {\tt https://go.nrao.edu/vlba-tut2} was
updated for the revised {\tt VLBARUN} and the link to the {\tt
  vlbarun.html} file was changed to a modern version.

\subsubsection{VLBAMPHC}

{\tt VLBAMPHC} is a new procedure to support DiFX's ability to
correlate an observation at multiple phase-stopping centers.  DiFX
writes the calibrator data, which does not use multiple centers, and
the zero-offset position of the multiple center source into the
``primary'' data file.  It then writes additional, smaller data files,
one for each ``secondary'' phase stopping center.  After the user
calibrates the primary data file, by hand or with {\tt VLBARUN}, {\tt
  VLBAMPHC} may be used to copy the highest {\tt CL} and {\tt BP}
tables from the primary data set to a secondary data set.  Applying
this calibration, {\tt RFLAG} may be run.  Then the calibration and
new flags are applied to the secondary data in {\tt SPLIT} and the
result imaged in either continuum or line modes.  {\tt RFLAG} plots
in living color and the message file are available from a {\tt
  vlbamphc.html} file.

\subsection{UV-data}

\subsubsection{{\tt RFLAG}}

{\tt RFLAG} was given two new options, both of which are used in the
new and revised VLBA procedures.  The first, hidden away as {\tt
  FPARM(17)}, flags all times for a channel if more than {\tt
  FPARM(17)} of the times have been flagged.  Unlike normal flags
written by {\tt RFLAG}, such flags are restricted to the specified
source.  The second option, {\tt FPARM(18)} requests that the clip
levels, usually set in {\tt NOISE} and {\tt SCUTOFF}, be instead
computed and used on a baseline basis.  This requires a new attached
table, called {\tt NS}, which like the adverbs is computed in the {\tt
  DOPLOT > 0} pass of {\tt RFLAG} and then applied in the {\tt
  DOPLOT < 0} pass.  The computation of the table and its application
can be done in one pass if {\tt DOPLOT > 0} and {\tt DOFLAG > 0}.  The
usual plots and adverbs, averaged over baselines, are still output as
well (but not used) even when {\tt FPARM(18)} is invoked.

A subtle bug was found in the code which could in some cases make very
non-subtle results (NaNs and infinities in {\tt NOISE} and {\tt
  SCUTOFF})\@.  The bug also caused a large number of useless flags to
be written.  Masking portions of the summing arrays when a baseline
has no data in a time interval solved the issue.  Applying the
pre-clip parameter {\tt FPARM(13)} is also a good idea.  The help file
recommends it now and the VLBA procedures use it.

\subsubsection{Miscellaneous}

\begin{description}
\myitem{FGPRT} is a new task to count flags by baseline applying to
        user specified ranges of IF, channels, sources, times, etc.
        A matrix display is produced.
\myitem{Planets} are now supported wherever a coordinate is required
        so long as a proper {\tt PO} table is available to describe
        the time-varying coordinates.
\myitem{Linear} polarization required a reversal of the sign of the
        parallactic rotation formulae when converting linear
        polarization data to Stokes.  We believe we finally have the
        sign correct.
\myitem{FRING} was changed for EVLA data to have the default for {\tt
        DPARM(9)} be to not compute rates.  EVLA users can overlook
        this important option and then be very puzzled by the wild
        swings in the phases.
\myitem{TYSMO} has a new option {\tt DOSCALE} to normalize the values
        in each scan for the median-window filter clipping.  That
        clipping was changed to be applied both before and after the
        median window operation.  Psum was flagged mysteriously due to
        testing on Psys, when no such test was requested.  Corrected
        to flag only when requested.
\myitem{The} {\tt SY} table format was changed some time ago to add a
        solar data indicator.  Several tasks, especially {\tt TYSMO}
        when flagging the {\tt SY} table, were corrected to handle the
        new format properly.
\myitem{SPLIT} was given a {\tt ROTMEAS} option to apply a rotation
        measure correction to the cross-hand polarization data
        written.  The value used is added to the output file's header
        keywords.  This option will allow averaging of spectral
        channels in the cross hands.
\myitem{BDF2AIPS} has a new option {\tt DOSTOKES} to select the
        desired data when the desired configuration contains data with
        different numbers of Stokes.
\myitem{SETJY} has a new option {\tt SRCALIAS} to give source name
        aliases so that a user's name for a source does not have to
        match the official name(s) of the source in {\tt 'CALC'}\@.
        {\tt SRCALIAS({\it i})} must be an official name for the
        {\tt SOURCES({\it i})}\@.
\myitem{BPWAY} was given the {\tt NORMALIZ} option to avoid the rms
        spectrum normalization if desired.  It was corrected to write
        the correct {\it uvw}'s when {\tt BIF > 1}.
\myitem{BPASS} has additional normalization options ({\tt
        BPASSPRM(10)}) to use the median rather than the mean.
\myitem{VBCAL} was corrected to write out valid data.  The default
        values for the {\tt {\it x}PARM} adverbs were corrected and
        the help file improved.
\myitem{IM2CC} was given the {\tt IMSIZE} option to allow the output
        image to be bigger than the minimum size.  The image size
        controls the accuracy of gridded modeling and larger is much
        better.
\myitem{OOSUB} was corrected in its division mode for an error that
        zeroed channels too early.  Other corrections were made to
        zero only the current channels and Stokes when computing a
        {\tt 'MODL'}\@.   The DFT routine and lower-level Q routines
        were changed to keep better track of whether a normal or a
        {\tt 'MODL'} computation was being done.  The decision on DFT
        versus GRID mode was also improved.
\end{description}

\subsection{Display and imaging}

\begin{description}
\myitem{IMAGR} was given {\tt NEXT FIELD} and {\tt LAST FIELD} options
         in the TV menu to make examining a large number of fields
         easier.  The {\tt STOP FLD} and {\tt ALLOW FLD} options now
         appear with all values of {\tt OVERLAP}\@.
\myitem{FILIT} was corrected to enable the menu after loading an
         image, rather than simply loading the images to the display
         in an infinite loop.  The roam function was also corrected.
\myitem{RESCALE} was given a {\tt DOCC} option to apply the re-scale
         factor to any {\tt CC} files as well as the image.
\myitem{Plots} from several basic tasks labeled brightnesses as
         ``flux''.  This was corrected to ``brightness.''
\end{description}

\subsection{General}

\begin{description}
\myitem{The} memory for the \POPS\ language processor was
         increased by a factor of 4.  {\tt POPSGN} and the {\tt SAVE}
         and {\tt GET} pseudo-verbs were improved to handle such
         changes in a manner that will be mostly transparent to the
         user.
\myitem{QCREATE} was changed so that the default is to create disk
         files quickly.  Many users are unaware of this adverb to {\tt
           GO} and suffer needless delays in their processing.  It may
         still be set negative to force a new file to be filled with
         zeros thereby insuring that the disk space is available when
         needed.
\myitem{\Cookbook} updates were made for the labeling of figures plus
         changes described here and new tasks from last year {\tt
           TVIEW} and {\tt SNBLP}\@.
\myitem{MOVE} was given a new {\tt OPCODE = 'COPD'} to copy a
         catalog entry, allowing duplicate names, and then delete the
         input file.
\end{description}

\section{Patch Distribution for \OLDNAME}

Important bug fixes and selected improvements in \OLDNAME\ can be
downloaded via the MNJ or from the Web beginning at:
\hspace{3em}{\tt http://www.aoc.nrao.edu/aips/patch.html}\\
Alternatively one can use {\it anonymous} \ftp\ to the NRAO server
{\tt ftp.aoc.nrao.edu}.  Documentation about patches to a release is
placed on this site at {\tt pub/software/aips/}{\it release-name} and
the code is placed in suitable sub-directories below this.  As bugs in
\NEWNAME\ are found, they are simply corrected since \NEWNAME\ remains
under development.  Corrections and additions are made with a midnight
job rather than with manual patches.  Because of the many binary
installations, we now actually patch the master version of \OLDNAME,
meaning that a MNJ run on \OLDNAME\ after the patch will fetch the
corrected code and/or binaries rather than failing.  Also,
installations of \OLDNAME\ after the patch date will contain the
corrected code.  The \OLDNAME\ release has had a number of patches:
\begin{enumerate}
  \item\ {\tt VBCAL} did not write the output data set. {\it
      2019-01-18}
  \item\ {\tt TYCOP} and other tasks had the old number of columns for
    the SY table. {\it 2019-02-13}
  \item\ Model computation had problems when computing multiple CC
    models. {\it 2019-03-05}
  \item\ Conversion of linear polarization to Stokes had a wrong sign.
    {\it 2019-03-05}
  \item\ {\tt TYSMO} applied clipping and flags inconsistently. {\it
      2019-03-05}
  \item\ {\tt FILIT} had an infinite loop displaying the image facets
    and did not handle the roam function properly {\it 2019-03-28}
  \item\ {\tt KNTR} and other tasks truncated one of the "brightness"
    strings {\it 2019-04-03}
  \item\ {\tt OOSUB} did not do channel dependent division correctly.
    {\it 2019-04-10}
  \item\ {\tt SPIXR} did not handle sub-images with an {\tt FQ} axis
    correctly. {\it 2019-05-03}
  \item\ {\tt RFLAG} did not handle missing data correctly, making too
    many flags. {\it 2019-06-14}
  \item\ {\tt KNTR} computed the number of planes to plot poorly {\it
      2019-06-14}
  \item\ {\tt BPWAY, RLCOR} did not change uvw for {\tt BIF} greater
    than one.{\it 2019-06-14}
  \item\ {\tt TAPLT} tested for no data backwards and so did not do
    good plots. {\it 2019-06-21}
\end{enumerate}

\section{\AIPS\ Distribution}

We are now able to log apparent MNJ accesses and downloads of the tar
balls.  We count these by unique IP address.  Since some systems
assign the same computer different IP addresses at different times,
this will be a bit of an over-estimate of actual sites/computers.
However, a single IP address is often used to provide \AIPS\ to a
number of computers, so these numbers are probably an under-estimate
of the number of computers running current versions of \AIPS\@.  So
far in 2019 more than 263 IP addresses have downloaded the frozen form
of \OLDNAME, while more than 396 IP addresses have downloaded
\RELEASENAME\@.  The binary version was accessed for installation or
MNJs by 253 sites for \OLDNAME\ and 352 sites for \RELEASENAME\@.  A
total of 671 different IP addresses have appeared in one of our
transaction log files.  The numbers for \RELEASENAME\ are essentially
the same as last year while the total and \OLDNAME\ numbers are
slightly lower than last year at this time.

\vfill\eject

% mailer page
% \cleardoublepage
\pagestyle{empty}
 \vbox to 4.4in{
  \vspace{12pt}
%  \vfill
\centerline{\resizebox{!}{3.2in}{\includegraphics{FIG/Mandrill.eps}}}
%  \centerline{\rotatebox{-90}{\resizebox{!}{3.5in}{%
%  \includegraphics{FIG/Mandrill.color.plt}}}}
  \vspace{12pt}
  \centerline{{\huge \tt \AIPRELEASE}}
  \vspace{12pt}
  \vfill}
\phantom{...}
\centerline{\resizebox{!}{!}{\includegraphics{FIG/AIPSLETS.PS}}}

\end{document}
