%-----------------------------------------------------------------------
%;  Copyright (C) 1995
%;  Associated Universities, Inc. Washington DC, USA.
%;
%;  This program is free software; you can redistribute it and/or
%;  modify it under the terms of the GNU General Public License as
%;  published by the Free Software Foundation; either version 2 of
%;  the License, or (at your option) any later version.
%;
%;  This program is distributed in the hope that it will be useful,
%;  but WITHOUT ANY WARRANTY; without even the implied warranty of
%;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%;  GNU General Public License for more details.
%;
%;  You should have received a copy of the GNU General Public
%;  License along with this program; if not, write to the Free
%;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
%;  MA 02139, USA.
%;
%;  Correspondence concerning AIPS should be addressed as follows:
%;          Internet email: aipsmail@nrao.edu.
%;          Postal address: AIPS Project Office
%;                          National Radio Astronomy Observatory
%;                          520 Edgemont Road
%;                          Charlottesville, VA 22903-2475 USA
%-----------------------------------------------------------------------
\input al82.mac
\input al8pt.mac
\letterbegin {IX} {4} {October 15, 1989}

\subtitf{Definition of the \AIPS\ Project}

   In a memo to the NRAO Scientific Staff dated 20 September 1989 the
Director of the NRAO, Paul Vanden Bout, made the following statements
about the goals of the \AIPS\ project and on the rearrangement of the
management of the project.

{\narrower
     ``There has been considerable discussion within the Observatory
recently about the AIPS project, both its management structure and its
long--range future.  The discussion has been useful and I have written
this memorandum to reaffirm the goals of the AIPS project, describe
the management restructuring that has been put in place, and announce
the appointment of a long range planning group for NRAO analysis
software.

     ``AIPS is the only array telescope software reduction package for which
NRAO has long-term plans.  The Observatory is committed to continuing the
AIPS project with these goals:

\item 1 To develop and maintain algorithms and portable code for
calibration and imaging of radio astronomical data, with emphasis
(but not exclusive concentration) on data from the VLA and VLBA.
To develop and maintain portable code for general image analysis.

\item 2 To maintain AIPS in the NRAO computing environments in Socorro and
Charlottesville in support of VLA and VLBA data processing by
visiting users and by NRAO scientific staff.

\item 3 To help non-NRAO user sites install and maintain AIPS in their
computing environments when these environments are inside the
general envelope of AIPS portability; to help these sites to
assist one another if not.

\item 4 To benchmark AIPS code performance in new hardware and software
environments, both to assist NRAO and its user sites with hardware
procurements and to explore ways to enhance the performance of
AIPS.

\item 5 To educate and inform users about AIPS through tutorial manuals,
workshops, and consultation.

``All of these goals are important to the Observatory's mission and all have
my full support.

     ``The following management changes are effective immediately.  Bill
Cotton replaces Eric Greisen as Project Manager, making it possible
for Eric to take a long delayed and much deserved leave from the
Observatory.  We all appreciate the enormous contribution Eric has
made in over ten years of developing and guiding AIPS, and we look
forward to his eventual return to the AIPS project as a design
specialist.  Gareth Hunt is appointed Deputy Project Manager, with
responsibility for AIPS activities in Socorro.

     ``The long range directions AIPS could take, as well as long range
goals and strategies for all NRAO data analysis software, merit
serious attention and discussion.  Accordingly, I have asked Tim
Cornwell to head a long range planning group for analysis software.
The group membership will include all sites and areas of computing:
array telescope and single dish data analysis, hardware, software, and
computing management.  A few experts outside NRAO will also be added,
from our user community or elsewhere as appropriate.  The group will
discuss strategic issues dealing with both technical and management
aspects of analysis software projects, for both array and single dish
data, and advise me and software project managers about such issues.''

}


\subtitf{\AIPS\ Overhaul}


   The 15OCT89 release of \AIPS\ will be the first of the
``overhauled'' version of the software.  The overhaul involved the
translation of the Fortran source code from modified Fortran 66 to
strict Fortran 77 (after the use of a preprocessor).  This should make
the installation of \AIPS\ on newer computer architectures simpler and
remove the source (2-byte integers) of many compiler problems on all
systems.  In addition, the new language is friendlier to programmers.
More details of the \AIPS\ overhaul were reported in the 15APR89
edition of the \AIPS Letter.

   The overhauled version of \AIPS\ has been tested extensively inside
the NRAO and most of the problems associated with this change are
thought to have been fixed.  As an added precaution, we will maintain
a version of the software for 15OCT89 on disk on an NRAO computer in
which major bugs will be fixed.  These fixes may then be accessed from
user sites using the AIPSSERV facility described in a later article.

   Due to the extensive changes in the file formats used in \AIPS\
there will not be an UPDAT procedure to translate the file structures
from older versions of \AIPS\ as there has been in the past.  To move
data from an older version of \AIPS\ to the 15OCT89 or later versions,
data or images must be copied to FITS format (either disk or tape)
from the old version and read into the new version.  This also applies
to \AIPS\ files written using BAKTP or other system-dependent backup
procedures.  For this reason, we advise sites installing the 15OCT89
or later AIPS to keep the old version of the AIPS program, task FITTP,
and the \AIPS\ system files necessary to run these until all users at
their site have copied their data to FITS format files.

   Users having custom \AIPS\ software will need to translate this
software to the new language standards.  Most of this translation can
be done with software provided in the 15OCT89 and later \AIPS\
releases.  Detailed instructions on using this software and other
changes are distributed with the installation tapes.

   Not all sets of ``Y'' routines have been translated yet.  Those
that have not been overhauled have been removed from the 15OCT89
release but will be reinstated when they have been translated.  The
affected systems are Args, Lexidata and Comtal Vision 1/20.


\subtit{User Agreement}

   Starting with the 15OCT89 release of \AIPS\ all user sites will
need to provide the NRAO with a signed ``User Agreement'' form.  This
``User Agreement'' will be a no-cost item for sites engaged in basic
research in astronomy.  The need for an \AIPS\ ``user agreement'' has
arisen for several reasons.  The most important is that we want all
\AIPS\ sites to obtain their copies of \AIPS\ from the NRAO and
thereby to be made aware of the restrictions that apply to their use
of the code and to our support of it.  For sites doing astronomical
research, these restrictions are only to maintain the proprietary
nature of the code and to direct third parties who wish to receive the
code to the \hbox{NRAO}.  Once properly signed, a User Agreement
remains valid for 5 years and does not need to be renewed before this
time.  A copy of this agreement is printed in the back of this
\AIPS Letter.  The agreement should be signed by an individual in a
position to take responsibility that the user group will follow the
agreement.  This may be a department chairman or an administrative
officer.  Mail signed forms to Amy Shepherd, NRAO, Edgemont Road,
Charlottesville, VA 22903-2475

\subtit{\AIPS\ Calibration Package}


   Much of the recent work in \AIPS\ has been on the calibration
package for interferometer data; in recent months this package has
begun to stabilize.  The calibration package is complete (or nearly
so) for VLA data and most of the functions needed for VLBI data are
also available.

   A recent addition is the use of a ``compressed'' uv data format which
uses up to a factor of 3 less disk space for large line projects.
Not all \AIPS\  tasks yet honor compressed format data but it is
supported in all of the main calibration package routines and many
others.  This format represents data in a 16 bit form in most
implementations and may not be suitable for cases where high ($>$ few
1000:1) spectral dynamic range is needed in the raw data or for
calibrated continuum data to be used for high dynamic range images.

   Ionospheric Faraday rotation corrections can now be made using task
FARAD.  This task allows use of either a model of the ionosphere using
the mean Sunspot number as the free parameter, or, direct measurements of
the total electron content of the ionosphere.  In the latter case, a
file containing these measurements made in Boulder, Colorado is needed
to calibrate data from the \hbox{VLA}.  This file can be obtained from NRAO,
when available, as described in a later article.

   MKIII VLBI data is supported in a limited way beginning with the
15OCT89 release.  The new task, MK3IN, can read single polarization line
or continuum data from ``A'' tapes.  The \AIPS\ calibration system is
just beginning to be used for MKIII data for fringe fitting and other
calibration so problems may be uncovered for this type of data.  This
area is being actively developed so substantial improvements are
expected soon in the ability of \AIPS\  to process MKIII data.

\vfill\eject
\subtit{Personnel}

   As of 15 September 1989,  Bill Cotton has assumed the role
of project manager and Eric Greisen has returned to the \AIPS\
programming staff.  We wish to thank Eric for his many years of
inspired leadership of the \AIPS\  project.  Gareth Hunt has become
the Deputy AIPS Manager and will be responsible for \AIPS\ operations
in Socorro.

   We are happy to announce that the former Nancy Wiener was married
to Ron Maddalena of NRAO-Green Bank.   We are sad to announce that this
event involved a move to Green Bank so Nancy is no longer a
member of the AIPS group.  Due to a hiring freeze at NRAO we have been
unable to replace her position.

   In September, Glen Langston joined the \AIPS\  project as a programmer.
Glen has extensive experience in VLA, VLBI and single dish radio
observations and will assume much of the responsibility for the
interferometer imaging functions in \AIPS .  Glen is a valuable addition
to the \AIPS\  programming staff.

\subtit{Document and Software Distribution by AIPSSERV}

   The NRAO maintains a mail-based file server, AIPSSERV, which is
available for use by \AIPS\ users to fetch files from {\tt CVAX}.
Detailed instructions for using this facility may be obtained by
sending an E-mail message containing the single word ``{\tt help}'' to
one of the following addresses:\hfil\break
{\tt aipsserv@nrao.edu}, {\tt aipsserv@nrao.bitnet}, {\tt
...!uunet!nrao1!aipsserv} or {\tt 6654::aipsserv}.

   We intend to use this facility to distribute text files such as the
ionospheric measurements needed for task FARAD, bug fixes for software
in the 15OCT89 release, and the contents of the CHANGE.DOC files
(documentation of software changes).  A general guide to special files
for distribution by AIPSSERV will be kept in file DOC:README.  To
obtain a text file in plain text form send AIPSSERV a message of the
form ``{\tt sendplain logical:filename.ext}'' where {\tt logical} is the
logical name of the directory and {\tt filename.ext} is the name of the
desired file.  Multiple files may be obtained by multiple
``sendplain'' commands one per line.  Individual uses are discussed
separately below.

{\bf Ionospheric data:}

   The ionospheric total electron content files will be kept in a
directory with logical name AIPSIONS.  These files  will be named
TECB.yy where yy are the last two digits of the year.  The data will
be entered into the appropriate file as it becomes available to the
NRAO; this is typically several months after the fact.
As an example, to fetch the file containing information for
February 1989 send AIPSSERV the message ``{\tt sendplain
aipsions:tecb.89}''.


{\bf Bug fixes in 15OCT89:}

   For the 15OCT89 release only, we will maintain a version of the
source code in which any remaining serious bugs will be fixed.  These
corrected routines can be obtained via AIPSSERV.  In addition,
AIPSSERV will provide a docmentation file describing fixes that have
been made in this software.  This will allow user sites to determine
if the fix they need has been made, and if so, which files to fetch.
This file is named DOC:FIXED.DOC.  To fetch this file, send AIPSSERV
the message ``{\tt sendplain doc:fixed.doc}''.  After receiving this
file if you determine that you need {\tt aplxxx:subr.for} then send
AIPSSERV a two line message containing ``{\tt version old}'' to
specify the version of \AIPS\ desired and ``{\tt sendplain
aplxxx:subr.for}'' to obtain the desired file.

{\bf CHANGE.DOC:}

   The old versions of the software change documentation files,
CHANGE.DOC, will be kept in area AIPSPUBL with names of the form
``CHANGED.yyn'' where ``yy'' is the last two digits of the year number
and ``n'' is ``A'', ``B'', ``C'' or ``D'' for the 15JAN, 15APR, 15JUL
or 15OCT release.  For example, to obtain the CHANGE.DOC file for
15OCT89 send AIPSSERV the message ``{\tt sendplain
aipspubl:changed.89d}''.  Note that we are no longer distributing
CHANGE.DOC as part of the \AIPS Letter.


\subtit{\AIPS\ - ISIS Comparison}

   Recently \AIPS\  calibration and imaging software was compared with
the ISIS system.  ISIS is a VLA specific software system which was
designed to take advantage of the hardware of the Convex computers in
the NRAO-Socorro facility.  NRAO does not intent to distribute ISIS
outside of the observatory; but, a comparison of its speed with that
of \AIPS\  is instructive.

   It is frequently argued that a general purpose system such as \AIPS\
pays a high penalty in execution time for being both portable and
general.  A comparison of \AIPS\  with ISIS on a machine for which ISIS
was optimized shows, in this instance, what penalty, if any, \AIPS\
pays for its portability and generality.  The pair of reports below
indicate that the penalty is gratifingly small.

   These tests gave essentially the same results in both the
\AIPS\ and ISIS systems except for a bug that was found and later
corrected in the ISIS polarization calibration routine.  Several areas
for improving the speed of the \AIPS\ calibration routines were
identified on the basis of these tests; especially the least squares
routine used to solve for antenna gains.  The basic results of these
comparisons were that \AIPS\ was slower than ISIS for some of the
calibration functions and faster in others.  However, \AIPS\ was much
faster for imaging continuum data and marginally faster at imaging
spectral line data in ``typical'' computing environments.
\medskip
\centerline{\bf{Continuum Tests}}

   The following discussion of the relative performance of \AIPS\ and
ISIS for continuum data is from VLA Computer Memo 181: ``The Relative
Performance of AIPS and ISIS'' by Rick Perley, Phil Diamond and Chris
Flatters (28 Sept. 1989).

{\narrower
``Considerable discussion has occurred over the last summer concerning the
relative merits of ISIS and AIPS for data calibration and basic imaging.
Various reports, invariably anecdotal, have circulated which claimed enormous
advantages of ISIS, especially with respect to its speed in performing the
essential steps of data listing, gain solution, and image generation.
However, no quantitative study has yet been attempted.

``Recently, the AIPS programmers discovered that the CONVEX AIPS code had been
compiled without optimization, which had the especially damaging effect of
forcing all vector operations to be scalar.
[This condition lasted for a few weeks on the Convex C1s in the AOC
only - ed.]
It would thus appear likely that
many of the reports circulating about were based on impressions gained when
the AIPS code was at a distinct disadvantage.

``It is obviously important to compare the running speeds of ISIS with AIPS.
The results of such tests not only allow detection and correction of errors in
codes, but also will highlight programs which are especially efficient, and
which can then be transferred from one set to the other.  From the user's
point of view, it is important to know which calibration route, if any, holds
especial advantages.

``For these reasons, we have performed benchmarking tests.  We selected a short
but representative database for comparison of the basic tasks of database
filling, matrix listing, gain solution and application, polarization
calibration, and imaging.  No attempt was made to compare flagging and
editing speeds, as these are dominated by user skill levels, as well as being
intrinsically subjective.  Tests comparing spectral line functions were
performed separately and will be reported separately.

``The remaining sections of this report cover the following items:

(1) Discussion of the database and the operations which were performed. We
also discuss the methods of timing which were employed.

(2) Display of the results in tabular form.

(3) A short discussion of the differences and implications.
\medskip
``{\bf The Database}.  We selected a continuum database, taken from a 1.5
hour observing run on 14 September.  These observations were at 3.6cm, and
included 24 antennas.  (Twenty-seven antennas were present for the initial 5
minutes, but three were then transferred to a separate subarray for VLB
observing.)  The data were of excellent quality, and no editing was
required.  Despite the short duration, polarization calibration was feasible,
since the observer thoughtfully observed a strong calibrator throughout the
run, giving sufficient parallactic angle coverage for an excellent solution.

	``The tests were performed after ensuring all other users were off the
system.  We monitored both wall clock time (through use of the UNIX `date'
command), and computer CPU time.  This latter quantity seems a little slippery
to quantify, so we attempted numerous methods.  For AIPS tasks, we recorded
the CPU and elapsed times, as given by each task upon completion.
Note that the CPU
times reported by AIPS are the sum of system plus user time.  Some ISIS tasks
report the CPU usage (broken down into `user' and `system' times), and we
recorded these.  For all ISIS tasks, we monitored CPU usage through the
UNIX `ps' and `w' commands.  The results of these two always closely tracked
(within 5 seconds) the accounting information given by the program itself.
It proved not possible to monitor the AIPS tasks in the same way, since the
shed task disappears from the system upon completion.  However, by monitoring
the task performance as it executed, we are confident that the statistics
given upon completion by the AIPS task can be compared to those determined for
the comparable ISIS task within a few seconds.

	It is interesting to compare the database sizes.  On the Dec-10, the
two data bases (for the `AC', and `BD' IFs) together took 13,480 blocks, or
6.90 Mbytes.  On ISIS, after filling with the same PASSFLAG parameters, the
database took 7.80 Mbytes.  Comparing ISIS with AIPS required filling with
PASSFLAG = BOTH, after which the ISIS database took 8.34 Mbytes, the AIPS
database 8.28 Mbytes.  Thus, the use of data compression in AIPS has made the
databases in the two systems the same size to better than 1\%.

``The various tasks are described below, with the ISIS/AIPS names given as
indicated.

\item{1.} DBFIL/FILLM  The data were located approximately halfway down the
tape.  We monitored both the wall clock time and CPU time to space down the
tape to the data, and to actually read the data into the file.  We ran with
PASSFLAG = BOTH, to ensure the same data were filled, since AIPS has a more
liberal interpretation of what constitutes bad data.  It was not possible to
separate CPU times in AIPS for these two operations, so only the sum is given
in the table.

\item{2.} LISTER/LISTR We produced a matrix listing of the data, with
ampscalar averaging over the entire scan length for all sources and scans.
We printed both the averages
and the rms's.  The test was run twice, the first without applying the gains,
and the second with gains applied in order to assess the effects on
performance of gain application.  (Since the data were uncalibrated, the actual
listings were identical.)

\item{3.} ANTSOL/CALIB The next step was to produce the calibration
parameters.  The fluxes were first entered manually (using SETJY/SETJY), from
values determined earlier. (No accounting was attempted here for reasons the
authors hope are obvious.)  We used ampscalar averaging, solving for 24
antennas, with UVLIMITS appropriate for
3C48 and 3C138.  The same reference antenna (\#4) was chosen to allow detailed
comparison of solutions and polarizations.  Listings of the solutions were
produced, and these timings were separated from the solution times.

\item{4.} GTBCAL/CLCAL  We used 2-point interpolation.

\item{5.} POLCAL/PCAL  The database contained two calibrators which were
tracked throughout -- one for calibration of the source, and other
specifically intended for polarization calibration.  We used both
sources, with the same reference antenna in both programs.

\item{6.} MAKMAP/HORUS Finally, we produced an image.  Two tests were done,
the first being a $512^2$ image, the second a $128^2$ image.
\medskip
``{\bf The Timing Results}.  The table at the end of this report displays the
results of the tests.  All times are in seconds.
Both the CPU and elapsed times are given, as are comments pertinent
to the test performed.  Note that the AIPS CPU time for the data loading test
is not broken down into `move time' vs. `load time'.
\medskip
``{\bf Discussion}.  The table shows that AIPS is markedly faster than ISIS in
most of the areas tested.  The difference is most marked in imaging, where the
ratio exceeds a factor of two.  There is no great difference in filling data,
although we note that ISIS is slower in spacing down the tape, but is
considerably faster
in actually loading the data.  It might interest some to note the tape-to-disk
data transfer rates are approximately 65 and 30 Kbytes/sec for ISIS and AIPS,
respectively.  We do not know the reason for these differences in tape spacing
and loading, but an explanation has been suggested and is being tested.

   ``For matrix listings, AIPS has a clear advantage -- nearly a factor of
two.  The advantage reverses for generating the solution.  Apparently the
cause of AIPS' relative slowness is known and will be shortly addressed.  The
test should be repeated after any changes are made to the algorithm.  The
listing of the gain solutions is similar for both systems, and in any event,
takes very little time compared to the generation of solutions.  The same can
be said of applying the gains, which is considerably faster in ISIS, but takes
only a very short time in either system.
Calculation of polarization is similarly quicker in AIPS, although the
factor is not large.
The large difference in imaging is very significant, because in
general, one calibrates the data only once, but commonly makes images many
times.

   ``Overall, the results surprised the testers.  We had assumed, along
with everyone else, that code especially written for the CONVEX would have
definite advantages in speed over the much more general purpose AIPS.  But
rumour and assumption are here clearly deceiving, and the results of these
tests clearly show that AIPS is to be preferred for data calibration and
imaging, when speed and efficiency are important.

``Three additional, and very important issues relating to deciding which
software package should be employed by the users (both in-house and visitors)
are the questions of ease of use, reliability, and responsiveness.
With regard to the first, it is certainly true that ISIS has
the advantage of familiarity, while at the same time, the generality of the
AIPS package and its formidable list of adverbs for the basic calibration
tasks will deter use.
Two comments are
appropriate:   (1) The VLA-specific RUN file, provided by Bill Cotton, has
largely removed the tremendous burden of sorting through the adverbs in CALIB
and CLCAL to find the ones relevant to VLA data.  (2) The user-unfriendly
gain listings provided by LISTR will shortly be rewritten to provide the
relevant information without disturbing the eye.  We feel that AIPS
calibration will soon be as easy as ISIS calibration.

   ``The last point is related to the second - having quick response to
problems builds confidence.  The recent structural changes to the AIPS group
will, we expect, provide the needed level of support.''


}

{\vskip 0.25in
\centerline{\bf RESULTS OF CONTINUUM TIMING TESTS}
\medskip
{\smaller
$$\vbox{\offinterlineskip
\hrule
\halign{&\vrule#&\strut\quad\hfil#\quad\cr
height2pt&\omit&&\omit&\omit&\omit&&\omit&\omit&\omit&&\omit&\cr
&\omit&&ISIS\hfil&\omit&ISIS\hfil&&AIPS\hfil&\omit&AIPS\hfil&&\omit&\cr
&Task\hfil&&Elapsed Time\hfil&&CPU Time\hfil&&Elapsed Time\hfil&&
	CPU Time\hfil&&Comments\hfil&\cr
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr
\noalign{\hrule}
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr
&DBFIL/FILLM&&420 + 120&&200 + 65&&300 +270&&245&&Spacing + Filling\hfil&\cr
&LISTER/LISTR&&82&&64&&45&&40&&No calibration, 1 IF\hfil&\cr
&LISTER/LISTR&&150&&130&&80&&72&&With calibration, 1 IF\hfil&\cr
&ANTSOL/CALIB&&60&&42&&110&&97&&Solutions\hfil&\cr
&ditto\hfil&&14&&10&&19&&10&&Listings\hfil&\cr
&GTBCAL/CLCAL&&12&&5&&19&&14&&\omit&\cr
&POLCAL/PCAL&&100&&80&&75&&58&&\omit&\cr
&MAKMAP/HORUS&&260&&250&&128&&140&&$512^2$ Image\hfil&\cr
&ditto\hfil&&253&&245&&109&&105&&$128^2$ Image\hfil&\cr
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr}
\hrule}$$}}

\medskip
\centerline{\bf{Spectral Line Tests}}

   The following memo is a discussion of a comparison of \AIPS\ and ISIS
for spectral line data entitled ``The Relative Performance of AIPS and
ISIS for Line Data'' by  Phil Diamond, Arnold Rots and Bill Junor (15
November 1989).

{\narrower
``{\bf Summary}. We performed tests of the efficiencies of the calibration
and imaging routines in the AIPS and ISIS packages. Our results demonstrate
that under typical conditions the two systems are very similar in
speed and produce images which are essentially identical.

``{\bf Introduction}.  The purpose of this document is to compare
 the efficiencies of the
AIPS routines with those of the ISIS system for calibrating and
imaging spectral line interferometer data as run on a Convex C1
computer.  The ISIS system was optimized for the C1 and makes
especially heavy use of the large amount of memory available.  We will
present the results of a careful comparison of the relative
performance of the two systems.

   ``In these tests we will compare the speed of calibration and
imaging in these two systems.  In all cases the images produced were
equivalent so only the timing results are of interest.  The bulk of
the time, both CPU and wall clock (real) in these tests are devoted to
the imaging step.  For this step we used the AIPS task HORUS which
reads multi-source files, calibrates and edits data 8 channels at a
time and then images the data one channel at a time.  The equivalent
ISIS routine is MAKMAP which calibrates and images all channels in
parallel.

   ``The most important difference is in the use of memory.  The
AIPS task HORUS uses only enough memory to grid a single channel at a
time; this reduces paging problems at a cost of increased I/O (the
data must be read multiple times).  ISIS routine MAKMAP reduces the
I/O required by imaging all channels in parallel at the cost of
potential problems with paging.  Since the timing results will
obviously depend on the loading of the machine (and thus the amount of
memory available for a given process) these tests were carried out in
different machine loading environments.  The following sections
describe the data used, the detailed tests and a discussion of the
results.

``{\bf The Data}. The data consist of a test VLA observation taken for
the express purpose of exercising the on-line system and the data reduction
systems. Two sets of seven scans on 3C48 were taken, with the correlator
mode, number of channels and number of IF's varying from scan to scan.
This was followed by two
series of three 64-channel scans on 0023-263; the three scans
comprised a ``source'' scan (looking for HI absorption) and two
bracketing scans (in frequency) for bandpass calibration.

``{\bf The Tests}. The first set of tests were run in a relatively
unloaded, but not empty Convex C1 (YUCCA).  Only CPU times were
recorded for these tests. In the following, the ISIS program and
equivalent AIPS task are given as (ISIS program/AIPS task).

\item{1.} Read data (DBFILL/FILLM): fill all data from tape onto disk.
%\item{} SETJY/SETJY (not timed): set flux of 3C48 to 15.4 Jy,
%0023-263 to 8.1 Jy.
\item{2.} Gain solutions (ANTSOL/CALIB): solve for all antennae, all
IFs, all modes, all sources, scan ampscalar averaging; and list.
\item{3.}  Apply calibration (GTBCAL/CLCAL): calibrate IF A on
0023-263 by itself.
%\item{5.} BASBP: generate baseline-based bandpasses for IFpair AA on
%0023-263, qualifiers 1 and 2, scan average.
\item{4.} Bandpass calibration (BASBP$+$ANTBP/BPASS): generate a
single set of antenna-based bandpasses, three-point Hanning smoothing,
vector (real/imaginary)
solution.
\item{5.} Image data (MAKMAP/HORUS): make an image and a beam cube,
256x256x63, 1"
cellsize from  the 0023-263:0 scans, natural weighting, IF A, bandpass
correction, 27,000 visibilities.
%In ISIS, we did this twice: once
%with a box convolution, and once with the spheroidal functions.  The
%reason is, that the gridding  convolution in MAKMAP has not been
%optimized, yet.  A realistic number should therefore be somewhere
%between the two.
\item{6.} Write UV FITS to tape (UVFITS/FITTP): write 63 channels,
0023-263:0, with bandpass correction.
\item{7.} Write image to tape (FITS/FITTP): write two cubes (maps and
beams), 256x256x63, to FITS tape.
\item{8.} Form continuum channel (CHZERO/AVSPC): for 0023-263:0,
replace channel zero by the average of  channels 8 through 55.

   ``After these tests were completed it was realized that paging could
be a problem for MAKMAP under ``typical'' (i.e. heavy) loading
conditions on the same computer. Several tests were then run to
examine the results of loading on the relative performance of the AIPS
and ISIS routines.  For these tests, the equivalent programs in the two
systems were initiated at the same time during normal daytime
conditions to ensure that the loading was the same.  For these tests
the real (wall clock times) were recorded.

\item{9.}  Gain solutions (ANTSOL/CALIB): solve for all antennae, all
IFs, all modes, like 2 above.
\item{10.} Image data (HORUS/MAKMAP) make an image and a beam
$256\times 256\times 63$\ similar to test 5. above using the same
gridding convolution function in HORUS and MAKMAP.

\item{11.} A repeat of 10.

\item{12.} A repeat of 10.

%\vfill\eject
   The timing results are given in the following table.  All times are
in seconds.


}
{
\vskip 0.25in
\centerline{\bf RESULTS OF SPECTRAL LINE TIMING TESTS}
\medskip
{
$$\vbox{\offinterlineskip
\hrule
\halign{&\vrule#&\strut\quad\hfil#\quad\cr
height2pt&\omit&&\omit&\omit&\omit&&\omit&\omit&\omit&&\omit&\cr
&\omit&&ISIS\hfil&\omit&ISIS\hfil&&AIPS\hfil&\omit&AIPS\hfil&&\omit&\cr
&Task\hfil&&Real Time\hfil&&CPU Time\hfil&&Real Time\hfil&&
	CPU Time\hfil&&Comments\hfil&\cr
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr
\noalign{\hrule}
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr
&1. Read data\hfill&&\hfil&&338.0&&\hfil&&700&&\hfil&\cr
&2. Gain solution\hfill&&\hfil&&52.3&&\hfil&&140&&\hfil&\cr
&3. Apply cal.\hfill&&\hfil&&(1)&&\hfil&&8.8&&\omit&\cr
&4. Bandpass cal.\hfill&&\hfil&&19.6&&\hfil&&122.0&&\omit&\cr
&5. Image\hfill&&\hfil&&1088.3&&\hfil&&3193.9&&\omit&\cr
&6. Write UVFITS\hfill&&\hfil&&198.1&&\hfil&&135.0&&\omit&\cr
&7. Write image\hfill&&\hfil&&2.6&&\hfil&&78.0&&\omit&\cr
&8. Form continuum\hfill&&\hfil&&257.7&&\hfil&&98.3&&\omit&\cr
&9. Gain solution\hfill&&130.0&&12.6&&84.0&&9.81&&Load factor 8\hfil&\cr
&10. Image\hfill&&10200&&\hfil&&10560&&\hfil&& Load factor 4-8\hfil&\cr
&11. ditto\hfill&&24371&&\hfil&&20710&&\hfil&& Load factor 5-14\hfil&\cr
&12. ditto\hfill&&21434&&\hfil&&20300&&\hfil&& Load factor 5-12\hfil&\cr
height2pt&\omit&&\omit&&\omit&&\omit&&\omit&&\omit&\cr}
\hrule}$$}}

{\narrower
``{\bf Discussion}. In tests 1-8 ISIS appears to be faster than AIPS in
terms of CPU time especially in the imaging step in which the CPU time
reported by HORUS was 2.5 times that reported by MAKMAP.  This step dominates
both the CPU and real time needed for both systems.  The large
difference in writing FITS files containing the images to tape is due
to the use by ISIS of a disk-based FITS format for its internal
storage of images.

   ``Tests 9-12 test the relative performance of the more expensive
steps, especially imaging, under various typical daytime loadings.
The results were illuminating.  These tests demonstrate that there is
rough parity in the speeds of the tasks on typically loaded machines.
This is due to the fact that the ISIS tasks assume that a large amount
of memory is available and when many users are running they start
having problems as the paging rate increases. The AIPS tasks are
written in such a way that they don't often hit paging troubles.
Roughly the real/cpu ratio of an AIPS tasks increases linearly with
load factor wheres the same ratio for an ISIS task increases with some
power of the load factor.  During one of these tests MAKMAP required
$60\%\ $ of the memory whereas HORUS only needed $15\%\ $.

``According to Chris Flatters, the time spent in paging on a Convex does
not actually appear anywhere in the time statistics associated with
a task. So the algorithms we use for real time consumed should be
modified to take the difference between the start and stop times
provided by the system.

``The near equality of the timing results of MAKMAP and HORUS under
typical loading conditions indicate that there is no performance
advantage to using large amounts of virtual memory.  The cost of
paging in ISIS seems to roughly balance the cost of extra I/O in the
AIPS routines.  The huge difference in the apparent CPU times seems to
be due in part to inadequate accounting in the Convex system.''

}

\subtit{Documentation}

   Several documentation publications are currently being revised and
will soon be available for distribution.  These items may be ordered
on the form at the end of this \AIPS Letter.

   The first two of these are the \AIPS\  \Cookbook\ chapters on
calibration of VLA and VLBI data using the \AIPS\  calibration package.
These describe in some detail the use of the calibration package of
tasks and procedures.  These should soon be ready for distribution and
can be ordered using the form at the back of this issue.  Chapter 10
describes processing VLBI data in \AIPS\ and chapter 99 describes
calibrating VLA data.  These chapters will be sent automatically to
sites requesting the 15OCT89 release of \AIPS.

   The second document being revised is the programmers' manual
``Going \AIPS''.  This manual is being rewritten to reflect the many
changes made in the \AIPS\ software system during the recent overhaul.
Outstanding orders for these items will be held until the documents
are ready; these items may be ordered using the form at the end of
this issue.


\subtit{Summary of Changes:  15 July 1989 --- 15 October 1989}

   As of this edition of the \AIPS Letter, we are no longer
printing the contents of the software change documentation file,
CHANGE.DOC.  Anyone wishing to see the details previously given in
these files may obtain them as described in the article on AIPSSERV.

\smallhead{Changes of Interest to Users: 15OCT89}


   As part of the overhaul, there was a change made in the case of
messages coming from \AIPS\ .  Error messages as always are in upper case
but informative or warning messages are in mixed case.  Also the
normal maximum number of interactive \AIPS\  has been changed from 6 to 12 (the
actual value may be set to less than 12 by the \AIPS\  system manager).

   After the overhaul, the most widespread changes in \AIPS\  were in the
general area of calibration of interferometer data.  In particular,
the adoption of an optional ``compressed'' uv data format allows a
significant reduction (up to a factor of 3) in the amount of disk
space and I/O time at the cost of a minor increase in the CPU time
used.  Not all uv tasks can process the ``compressed'' format data but
those that can't will inform the user.  All of the calibration
package tasks process compressed data properly as do many of the other
critical uv data tasks.  New task {\tt UVCMP} will convert between
compressed and uncompressed format.
Another major improvement for spectral line or snapshot project is
the task {\tt HORUS}, which can image uv data directly from a multisource
data file, optionally calibrating and editing, and without
requiring the data to be sorted.  Multiple channels and/or sources can be
processed in a single run of the task.

   Task {\tt FARAD} allows the correction of uv data for the effects of
variable ionospheric Faraday rotation.  This task can either use a
model of the ionosphere based on the mean Sunspot number, or use
measurements of the total zenith electron content.  These latter
measurements, made at Boulder Colorado, are suitable for use with
VLA data and will now be distributed by AIPSSERV as described in
another article.
A set of \AIPS\  POPS procedures to simplify the calibration of VLA
data are included in a run file named {\tt VLAPROCS}.  These include
{\tt VLACALIB} to run {\tt CALIB}, {\tt VLACLCAL} to run {\tt CLCAL}
and {\tt VLARESET} to reset the calibration tables.
Task {\tt FILLM} which reads VLA archive format tapes was extensively
modified.  It will now read all data from a given project in a single
pass through the tape, automatically expand and contract files as
needed, and can append data to the end of a previously existing uv data
file.  A bug was fixed in the Convex tape handling routines which
caused tapes to advance file rather than go to the beginning of the
file and to advance file before rewinding.
Several serious bugs were fixed in {\tt TRANS}, {\tt PBCOR} and the
VMS Pseudo AP version of {\tt UVMAP}.

   The ability of \AIPS\ to handle frequency and/or bandwidth
switching in uv data was strengthened with the introduction of ``{\tt
FQ}'' tables.  These tables, with the optional addition of a new
random parameter, allow data with different sets of frequencies to be
kept in the same uv data file.  A uv data file is still required to
have a constant number of spectral channels, polarizations and IFs.
The calibration software allows selection by frequency setting or
bandwidth using adverbs {\tt SELBAN}, {\tt SELFREQ} and {\tt FREQID}.
As currently implemented, the calibration and editing routines will
process one set of frequencies/bandwidths at a time.  New task {\tt
SNPLT} allows plotting values from the calibration ({\tt SN} or {\tt
CL}) tables.  {\tt INDXR} will now create a dummy {\tt CL} table if
none previously exists.  New task {\tt QUACK} will flag fixed amounts
at the beginning of each scan; this is useful for systems which start
recording data prematurely.  New task {\tt UVHGM} provides plots of
statistics of a uv data set.

   New task {\tt MK3IN} will read MKIII VLBI correlator ``A'' tapes.  In this
release it only works for simple continuum and line data (a single
polarization).  An accurate description of the geometry used by the
correlator is preserved in the {\tt CL} table as are the measured values of
the ``phase-cal'' signal.  The full set of calibration routines have not
been tested on this kind of data and are likely to contain errors.

A new adverb, {\tt CHANSEL}, was introduced into a number of tasks to allow
a generalized specification of uv line channels to be averaged into a
continuum channel.  New task {\tt AVSPC} uses this to produce a continuum uv
data set by averaging selected line channels.  {\tt BPASS} now can average a
specified set of channels to form the ``channel 0'' (continuum) to
divide into the line data.
New task {\tt ISPEC} plots spectra from a specified pixel or region
of a line cube.
New task {\tt KNTR} will make multiple contour plots on the same
page and uses an algorithm suitable for pen plotters.  {\tt LISTR} has a new
option to list the source elevations at the times and antenna
locations corresponding to entries in an {\tt SN} or {\tt CL} table.
A bug was fixed in {\tt MX} which caused it not to process ``R'' or
``L'' Stokes' polarization data.

   New task {\tt FETCH} will read and catalog an image from an external text
file with a flexible format.  This is like {\tt CANDY} but does not need to
be modified, compiled and linkedited.  New task {\tt TBOUT} will write an
\AIPS\  table into a FITS-like external text file.  A task to read
these files ({\tt TBIN}) will be available in the {\tt 15JAN90}
release.  New task {\tt SETAN} will create an {\tt AN} table and fill
it with information read from an external text file.

\smallhead{Changes of Interest to Users: 15JAN90}

A number of new tasks are introduced in this release.
{\tt RSTOR} will convolve CLEAN components with a Gaussian and
add them to an image.
{\tt SOLCL} will apply system temperature measurements for solar
observations made with the VLA.
{\tt UVMTH} will time average one uv data file and will add,
subtract, multiply or divide the averaged values to/from/by/into the
visibility data in another uv data file.
{\tt BLFIT} will solve for source and/or antenna positions from
residual phases in an {\tt SN} or {\tt CL} table.
{\tt TBIN} can read external FITS-like tables of the form written
by {\tt TBOUT} and convert them into \AIPS\ tables.
{\tt ACFIT} will determine the amplitude of antenna gains for spectral line
uv data by fitting a ``template'' spectrum to the observed
autocorrelation spectra.
A new adverb, {\tt FQTOL}, was added to {\tt DBCON} to allow user
control of the definition of {\tt FQ} ids.

{\tt BPASS} can now divide line uv data by continuum data from another file.
The polarization calibration task {\tt PCAL} will now apply ionospheric
Faraday rotation corrections before determining the instrumental and
source polarizations.
Corrections for ionospheric Faraday rotation are now applied in any
routine that applied the polarization correction.
{\tt MK3IN} can now read data from polarization experiments done with the
MkIII VLBI system.
{\tt LISTR} now can have a fixed scaling for amplitude listing, separate
scaling for amplitude and RMS values and a Dec-10 like gain listing
option.
{\tt SNPLT} can now plot doppler offsets from a {\tt CL} table.
{\tt UVFIX} can now process compressed format uv data and a correction has
been made in it's computation of the correct orientation of the field
at the standard epoch.
{\tt UVCOP} can now select either auto- or cross- correlation data to copy.
A bug was fixed in {\tt VLBIN} in the lobe rotation of station ``B'' data in
MkII VLBI spectral line data.
Numerous bugs were fixed in {\tt CVEL} which corrects spectroscopic
interferometer data for the earth's rotation.
A bug was fixed in verb {\tt GET} which caused minimum match to fail if
there were more than 10 potential matches.
A bug in the gridded subtraction method used by {\tt MX} and {\tt
UVSUB} which introduced horizontal stripes into images was fixed.

The \AIPS\ table access routines have been modified to recatalog
``forgotten'' \AIPS\ tables; reading a ``forgotten'' table with {\tt PRTAB}
or other task will cause it to be recataloged.
The Unix file destruction routines have been streamlined to speed up
the destruction of files.

\smallhead{Changes of Interest to Programmers: 15OCT89}

     As the first release of the overhauled version of \AIPS\ there
are many more changes of interest to programmers than can be given
here.  The programmers' manual ``Going \AIPS '' is being revised, and
interested programmers should order a copy.  In addition, a more
detailed description of the changes is now included with \AIPS\
installation tapes.  Among the many changes are the way INCLUDEs are
processed.  There is now a source preprocessor on all systems to
include text files; among other things.  This preprocessor allows
defining a ``Local INCLUDE'' in the file in which it is to be
included.

     Another major change is the way in which characters are handled
inside \AIPS.  Most usages are now declared CHARACTER in Fortran;
in the few cases where the definition of Fortran disallows use of
CHARACTER variables (in any form of equivalencing) \AIPS\ uses type
HOLLERITH (4 characters per numeric element).  Since only ANSI Fortran
77 types are allowed, literals are now allowed in call sequences.
Another major improvement is the ability to store Keyword/Value pairs
in the catalog header file using routine {\tt CATKEY}.

   The introduction of ``Compressed'' uv data into \AIPS\ has an
impact on many tasks which process uv data.  Tasks that use UVGET to
read uv data are unaffected, as this routine always returns data in the
expanded form.  In the compressed format, the correlation coefficients
are each packed into a single real word in a machine dependent manner.
On current implementations this packing is into a pair of scaled 2 byte
integers with magic value blanking.  Compressed data thus can have only
a single weight per visibility but individual values may be flagged
bad.  New subroutines {\tt ZUVPAK} and {\tt ZUVXND} are used
to compress and uncompress data.  Two new random parameters ``{\tt
WEIGHT}'' and ``{\tt SCALE}'' are introduced for compressed data.

     Programmers interested in the calibration area should study the
wide-reaching changes brought about with the {\tt FQ} tables .  All
calibration tasks require three new adverbs, {\tt SELBAND}, {\tt
SELFREQ}, and {\tt FREQID}.  Several basic subroutines have had their
call sequences changed; these include {\tt CHNDAT}, {\tt FRQTAB}, {\tt
CHNCOP}, {\tt TABCAL}, {\tt TABSN}, {\tt TABBL}, {\tt TABBP}, and {\tt
TABNDX}.  Several table files have had an extra column added, namely
the {\tt CL}, {\tt SN}, {\tt BL}, {\tt BP} and {\tt NX} tables.  Other useful
tools in the uv-calibration area are {\tt SETSTK}, to translate the
user's {\tt STOKES} value, {\tt REQBAS}, to translate the user's {\tt
ANTENNAS} and {\tt BASELINE} adverb values, and {\tt WANTCH} and {\tt
AVGCHN}, to handle the new {\tt CHANSEL} adverb.  The polarization
calibration routines have been modified to process data from
orthogonal linearly polarized feeds although this is still largely
untested.

     Documentation of the new formats of a number of the \AIPS\ system
files are now available.  The descriptions of the {\tt AC}, {\tt BA},
{\tt BQ}, {\tt CA}, {\tt CB}, {\tt HI}, {\tt IC}, {\tt MS}, {\tt SG},
{\tt TC}, {\tt TD}, and {\tt TS} files are in {\tt DOCTXT:MV2C06xx.}
where xx is the 2 character file type.  This documentation will
probably be incorporated into Going \AIPS .

     Several useful utility routines have been introduced which are
related to the introduction of {\tt FQ} tables.  {\tt FQMATC} checks user
specified frequency/bandpass specification against the {\tt FQ} table
to determine the desired {\tt FQID}.  Routine {\tt SELINI} is useful
for initializing the selection and control parameters passed to {\tt
UVGET} in include {\tt DSEL.INC}.  If tables need to be reformatted
due to the additions of columns, then {\tt BLREFM}, {\tt BPREFM}, {\tt
CLREFM} and {\tt SNREFM} will reformat {\tt BL}, {\tt BP}, {\tt CL},
and {\tt SN} tables.  The number of columns in each type of table used
in the calibration system is now defined in the PARAMETER include {\tt
PUVD.INC}.

     The format of help files has also changed.  They begin with a
precursor section which gives a one-line description and keyword values
as well as separator lines.  Any line beginning with a semi-colon is now
treated as a comment line in help files and is ignored.  The
maximum number of interactive \AIPS\ allowed is now 15 if there is no
batch and 14 minus the number of batch queues if there is batch.
The system manager can set the limit lower than this.  All service
programs now have free-format inputs.  This makes a serious
change in the format of the main input line to {\tt POPSGN}, for example.
Gripes may be sent to Charlottesville on tape using {\tt GRITP} or
turned into text form for e-mail using {\tt GR2TEX}.  Both of these
were cleaned up and improved over the quarter.

     The Z-routine areas of the code were simplified by dropping
obsolete operating systems and distinctions.  The Unix area is
now divided into a Bell area, with Masscomp and Cray sub-areas, and
a Berkeley area, with Alliant, Convex, Sun, and Vax sub-areas.
The midnight job was generalized to have separate
``lastgood'' dates for the separate portions of the job.
The VMS system translation of logicals was extended and the
printing of a {\tt NOTICE.TXT} file was added to the {\tt AIPS}
procedure.  The TV is now a logical rather than a symbol for \hbox{VMS}.

\smallhead{Changes of Interest to Programmers: 15JAN90}

A package or J2000 precession routines is now available; {\tt JPRECS}
is the highest level routine.
New routines {\tt HIMERG} and {\tt HIADDN} simplify the concatenating
of history files; two copies of the same file will not be written to
the output file.
Utility routine {\tt GETFQ} will get the information for a given FQID from
the {\tt FQ} table.
The axis labels for plots are now allowed to be 20 characters rather
than 8.
New routine {\tt PUTCOL} stores a given value into an \AIPS\ table entry.
Parsing routine {\tt GETNUM} now returns a value of {\tt DBLANK} when
it attempts to read a bad value.
A number of improvements were made to the DDT tests.
\vfill\eject
%\pgskip
\input order.tex
\end
