%-----------------------------------------------------------------------
%;  Copyright (C) 1995
%;  Associated Universities, Inc. Washington DC, USA.
%;
%;  This program is free software; you can redistribute it and/or
%;  modify it under the terms of the GNU General Public License as
%;  published by the Free Software Foundation; either version 2 of
%;  the License, or (at your option) any later version.
%;
%;  This program is distributed in the hope that it will be useful,
%;  but WITHOUT ANY WARRANTY; without even the implied warranty of
%;  MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.  See the
%;  GNU General Public License for more details.
%;
%;  You should have received a copy of the GNU General Public
%;  License along with this program; if not, write to the Free
%;  Software Foundation, Inc., 675 Massachusetts Ave, Cambridge,
%;  MA 02139, USA.
%;
%;  Correspondence concerning AIPS should be addressed as follows:
%;          Internet email: aipsmail@nrao.edu.
%;          Postal address: AIPS Project Office
%;                          National Radio Astronomy Observatory
%;                          520 Edgemont Road
%;                          Charlottesville, VA 22903-2475 USA
%-----------------------------------------------------------------------
%Summary of bits of recent DDTs: AIPS MEMO 73
%last edited by glen langston on 91 August 21
\documentstyle{article}
\newcommand{\lastedit}{{\it 91 May 16}}
\newcommand{\uvdata}{{\it uv}-data~}
\input{latexmacro}
\large
\parskip 3mm
\baselineskip 7mm
\textwidth 6.5in
\linewidth 6.5in
\marginparsep 0in
\oddsidemargin 0in
\evensidemargin 0in
\topmargin 0.0in
\headheight 0in
\headsep 0.25in
\textheight 8.5in
\headheight 0.25in
\pretolerance=10000
\parindent 0in
\itemindent 2in
\newcommand{\AIPS}{{$\cal AIPS$}\ }
\newcommand{\APEIN}[1]{{\normalsize \sc {#1}}}
\newcommand{\DDTE}{{\normalsize \sc DDTEXEC}~}
\newcommand{\DDTL}{{\normalsize \sc DDTLOAD}~}

\begin{document}

\leftmargin 2.5in
\labelsep .25in
\labelwidth 1.in

\pagestyle{myheadings}

\newcommand{\HEADING}{{\it DDT History} \hfill \lastedit \hfill Page }
\markboth{\HEADING}{\HEADING}
\huge
\centerline{\AIPS DDT History}
\vskip .5in
\large
\centerline{Glen Langston, Pat Murphy and Dean Schlemmer}
\centerline{\lastedit}

\begin{quote}
This document is intended for \AIPS system managers who are installing
a new version of \AIPS.
After installation, the accuracy and speed of the primary \AIPS tasks
can be measured with a set of \AIPS DDTs.
The results of the DDTs should be compared with results tabulated
in this document.
\end{quote}

\section{Summary of \AIPS DDTs}
The \AIPS DDTs are a selection of tasks designed to find
{\it bugs} which are to be {\it exterminated} before software
is released to the public.
To ensure the quality of basic image processing software,
these DDT tasks measure quantitatively the reliability of a few key tasks.
This document records the results of \AIPS tasks and compares
them with those of earlier versions.
The tasks tested with the DDTs are
UVSRT, UVMAP, APCLN, ASCAL, MX and VTESS.
Additional \AIPS tasks run are UVDIF, COMB, and SUBIM.
These tasks start with a standard set of \uvdata and manipulate
it to produce images which are compared with reference images.

Three types of tests are run; SMALL, MEDIUM and LARGE,
producing 256, 512 and 1024 square pixel images from
\uvdata sets with 8000, 13000 and 77000 visibilities, respectively.
The quality of these tests is determined by subtraction
of the resulting images from previously determined standard
images.

Appendix A of this document is a listing of an execution of the
\AIPS RUNFILE \DDTL, which produces procedures necessary for
executing the DDT.  Appendix B is a listing of an execution of
the \AIPS RUNFILE \DDTE.

For a comparison of execution speeds
of the Convex C1 and the IBM 6000 series computers, see \AIPS memo 71.
This memo supersedes \AIPS memo 63.

\clearpage
\section{DDT History}
The \uvdata and resulting images have changed periodically,
the latest change occurred in January 1989 and the
previous change occurred in October 1987.
The changes in January 1989 have been termed the 15APR89
copy of the DDT.

The dates of major events in \AIPS coding which effect the
DDTs are described below.
\begin{description}
\item [\bf 84 Apr] Method for validating new CPU running \AIPS
needed for new computer procurement.
\item [\bf 84 Dec] First DDT set of routines defined by
Don Wells, Eric Greisen and Bill Cotton
\item [\bf 85 Nov] DDT modified to allow one of three
sizes of DDT, Small, Medium, and Large.
\item[\bf 87 Oct] A new DDT tape created
with high dynamic range LARGE DDT from Alan Bridles data.
Tape created by Eric Greisen and Kerry Hilldrup.
\item[\bf 89 Jan] Code overhaul begins, numerous (All)
routines are modified.
\item[\bf 89 Mar] Clean components are merged before
ASCAL to speed execution.
New maps created for MEDIUM and LARGE tests of MX tasks.
Modification by Chris Flatters.
\item[\bf 89 Sep] Overhaul bug found in AP routines
used by UVMAP.
\item[\bf 89 Oct] MX bug found by Patrick Leahy
which caused strips to be inserted into {\it some}
images at the 0.1 \% level.
Bug traced to ALGSUB, which had
incorrectly performed complex conjugation.
This bug was released in several versions of AIPS.
\item [\bf 90 Jan] Anomalously small number of bits found
on CONVEX and SUN for UVMAP and UVBEAM tests.
Fixed by replacing single precision with double precision
arithmetic.
\item[\bf 90 Jan] Minor bugs fixed in DDT procedure
and printout shortened by Glen Langston.
\item[\bf 90 Apr] DDTs detect error in MX task just before
distribution, showing DDTs are still required.
\item[\bf 90 May] DDTs show MEDIUM and LARGE reference images
for MX contain ALGSUB errors which produce stripes at 1000 to
1 dynamic range.  Current MX results are believed correct
and a new DDT tape release is planned.
\item[\bf 91 Apr] Additional changes to MX and its subroutines
improve accuracy of output images.  In the Pseudo-Array Processor
code, fourier transforms are performed with double precision
accuracy.  The Convex version of these routines are single
precision and the results have a small but measurable difference
in the output image.
\item[\bf 91 May] A new HUGE DDT is under development.
This DDT produces 4096$\times$4096 pixel images of the supernova
remnant Cassiopeia A.
\end{description}
\parindent 0in

\clearpage
\begin{figure}[t]
\vskip 2.in

{\it \hskip 1in a) \hfill b) \hskip 1in}

{\bf Figure 1:}
UV Plots of small DDT data. Radio source is 3C161.
{\it a)} Plot of Amplitude versus Baseline
length. {\it b)} Location of visibilities in the UV plane.
\end{figure}

\begin{figure}
\vskip 2.5in

{\bf Figure 2:}
Cleaned Image of 3C161 produced by DDT task MX.
\end{figure}

\begin{figure}
\vskip 2.5in

{\bf Figure 3:}
Image of 3C161 produced by DDT task VTESS using Maximum Entropy
Deconvolution.
\end{figure}

\clearpage
\section{How to Run the DDT}
Running the DDT consists of three major steps;
{\it a)} creating the DDT procedures,
{\it b)} loading the DDT master images, and
{\it c)} running the DDT procedures which start tasks
and compares output.
Parts of a sample \AIPS session is listed
below for reading the master images from tape.
The \AIPS RUNFILE \DDTL is used to create the required AIPS
procedures.  The \AIPS RUNFILE \DDTE runs the \AIPS tasks and
produces the test results.
Note that \DDTE must be run twice, once to read the master images
and once to test the \AIPS tasks.
Also note, after setting the DDT inputs, the DDT inputs MUST
be \APEIN{TPUT} for the inputs to have effect (ie. \APEIN{TPUT DDT}).
The text the user must type is in lower case.
\begin{verbatim}
>input mount
AIPS 1: MOUNT:    Verb to software mount a tape
AIPS 1: Adverbs         Values            Comments
AIPS 1: ----------------------------------------------------------------
AIPS 1: INTAPE        1                    Tape unit # (0=> 1)
AIPS 1: DENSITY    1600                    Density to set on mount.
>density=6250                      $ ddt data is on a 6250 bpi tape
>mount                             $ mount the master tape
>run ddtload                       $ create the DDT procedures
   ...  much printout ...
>input ddt
AIPS 3: DDT  :  Verification/timing test; see DDTLOAD.001, DDTEXEC.001
AIPS 3: Adverbs         Values            Comments
AIPS 3: ----------------------------------------------------------------
AIPS 3: TCODE      'TEST'                  INIT,TEST,READ, or WRIT
AIPS 3: TMODE      'M'                     T or M; Test or Master input
AIPS 3: TMASK        32                    test selection bit mask
AIPS 3: DDTSIZE    'SMALL   '              'SMALL','MEDIUM','LARGE' test
AIPS 3: DDISK         5                    Disk drive #: master UVDATA
AIPS 3: MDISK         5                    Disk drive #: other masters
AIPS 3: TDISK         1                    Disk drive #: test images
AIPS 3: IOTAPE        3                    Input/Output tape drive #.
AIPS 3: EDGSKP        4                    Pixels to skip at edges
AIPS 3: TERSE         1                    > 0 => reduced output
AIPS 3: OUTPRINT   '                                                '
AIPS 3:                                    Printer disk file to save
AIPS 3: BADDISK    *all 0                  Disks to avoid
AIPS 3: VERSION    '                                                '
>tcode='read'                      $ prepare to read master files
>tmask=127                         $ read in all ddt master files
>ddtsize='large'                   $ do the big test
>edgskp=16                         $ ignore a few pixels around edge
>iotape=1                          $ use the tape you mounted for read
>baddisk=2,3,4,5,6,0               $ only use disk 1
>input ddt                         $ check ddt inputs one more time
>tput ddt                          $ save ddt adverbs for DDTEXEC !!!
>run ddtexec                       $ load the master files
  ... much printout and many executions of IMLOD and UVLOD ...
>dismount                          $ go put away the tape
\end{verbatim}

Each DDT (small, medium, or large) requires 11 files from
the tape.  Three of these files contain \uvdata and the remaining
eight are master images, which will be used to test the DDT results.
The files loaded for the small test are listed below.
\begin{verbatim}
>inname='*S'; inty=''; ins=0; incl=''
>catalog
AIPS      Catalog on disk  5
AIPS       Cat Usid Mapname      Class   Seq  Pt     Last access      Stat
AIPS         1    1 DDDTS       .UVDATA.    1 UV 01-MAY-1991 00:18:23
AIPS         2    1 MDDTS       .UVSRT .    1 UV 01-MAY-1991 00:18:33
AIPS         3    1 MDDTS       .UVMAP .    1 MA 01-MAY-1991 00:18:44
AIPS         4    1 MDDTS       .UVBEAM.    1 MA 01-MAY-1991 00:18:51
AIPS         5    1 MDDTS       .APCLN .    1 MA 01-MAY-1991 00:18:59
AIPS         6    1 MDDTS       .APRES .    1 MA 01-MAY-1991 00:19:06
AIPS         7    1 MDDTS       .ASCAL .    1 UV 01-MAY-1991 00:19:14
AIPS         8    1 MDDTS       .MXMAP .    1 MA 01-MAY-1991 00:19:25
AIPS         9    1 MDDTS       .MXBEAM.    1 MA 01-MAY-1991 00:19:33
AIPS        10    1 MDDTS       .MXCLN .    1 MA 01-MAY-1991 00:19:40
AIPS        11    1 MDDTS       .VTESS .    1 MA 01-MAY-1991 00:19:48
\end{verbatim}
The original \uvdata file is named DDDTS.UVDATA.1, and all the other
files were created from this \uvdata.
The other two \uvdata sets are MDDTS.UVSRT.1, an XY sorted version
of DDDTS.UVDATA.1, and MDDTS.ASCAL.1, a self-calibrated version
of DDDTS.UVDATA.1.

Now that the \uvdata and master images are loaded, the test
can be started.  The tests will create 1 file for each of the
original master files.  These files will be named TDDTS.?????.1
(ie. the UVSRT test will create TDDTS.UVSRT.1).

\clearpage

The inputs are set and the test output is summarized below.
(After setting the inputs to DDT, do NOT forget to \APEIN{TPUT}
the DDT inputs or they will have no effect!)

\begin{verbatim}
>tget ddt                          $ get ddt adverbs
>tcode='test'                      $ indicate test should be run
>input ddt                         $ always check the inputs
>tput ddt                          $ save ddt adverbs for DDTEXEC !!!
>clrmsg                            $ clean up the message file
>run ddtexec                       $ actually run the DDTs
  ... much more printout and several tasks run ...
...
 Mapping tasks should have about 10 BITS of accuracy
 A section of the output is listed below
...
          task             Peak Bits   RMS Bits
AIPS      UVMAP            13.6202     19.2307
AIPS      UVBEAM           10.2618     16.4874
AIPS      APCLN            18.798      24.0944
AIPS      APRES            16.8828     22.2016
AIPS      MXMAP            12.5708     19.0115
AIPS      MXBEAM           14.125      19.4482
AIPS      MXCLN            14.4533     17.5482
AIPS      VTESS            18.6827     25.9653
AIPS      THAT"S ALL, FOLKS!
\end{verbatim}

\section{Record of the BITS}
The DDT results are presented in 4 groups:
\begin{description}
\item {\it a)} Sorting and Mapping
\item {\it b)} AP Clean and Self Calibration
\item {\it c)} MX Map and Cleaning
\item {\it d)} Maximum Entropy and DDT Notes
\end{description}
The results of the tests are dependent on the
software version, CPU, and the level of debugging
of the code.
The level of debugging is specified by whether
the version is OLD, NEW or TST.
The TST version of code is compiled with
the DEBUG option and is generally slower than
code in the OLD or the NEW versions.
Systems with Array Processors also have the option of
running with Pseudo-Array Processors by appending PSAP
to the version name. (i.e. to run TST with the pseudo array
processor code set VERSION='TSTPSAP')
Normally, the default VERSION is used by setting VERSION=''.

The results of the tests are quantified by the number of BITS
of accuracy.
The number of BITS are determined by subtracting the
{\it reference} image from {\it test result} image.
The BITS of accuracy are defined
in two ways; {\it 1)}
by the MAX difference of the Master image and the
Test image and
{\it 2)} by the RMS differences between the images.
The BITS calculated from the base 2 logarithm of the ratio
of PEAK of the reference image and DIFFERENCE of the images.

The maximum and the RMS differences as described by equation (1) below.
\beq
BITS = \log_2{\frac{DIFF}{PEAK}} =
       \frac{1}{log_{10}{2}}\log_{10}{\frac{DIFF}{PEAK}} =
       -3.3219 \times \log_{10}{\frac{DIFF}{PEAK}}
\eeq
($log_{10}$ is the Base 10 Logarithm)
The reference images are stored on tape in a 32 BIT floating point
format, but eight of the Bits are used for the sign and
exponent of the number.
The maximum accuracy possible is roughly 24 bits.
(When the two images are produced by the same machine, the
BITS result can be higher.
In this case there are few, if any
differences between the images, giving a deceptively
large number of bits of accuracy.)

The mapping routines perform many multiplications
and round off errors accumulate, generally giving
images with 14 to 16 bits of accuracy
($2^{14} = 16384, 2^{16} = 65336$).
A dynamic range of 100,000 to 1 is marginally repeatable
for the different CPUs listed here.  ($100,000 = 2^{16.6}$)
The RMS BITS of accuracy is
always greater than the PEAK BITS of accuracy.
The expected BITS of accuracy for the tasks is in the
range of 10 to 20 BITS.

Note that slight changes in algorithm change the
{\it measured} accuracy of the task significantly.
For instance, changing the gridded weighting
of \uvdata in a mapping task
produces a slightly different type of map.
The comparison of apples and oranges results
in a small number of BITS of accuracy.

A final BITS note; the eye can not see differences between
two images if there is less than 7 BITS difference.
If two images are subtracted, then small differences
become visible.  (i.e. view the difference map to determine
the type of problems occurring in the test image)
At the end of each mapping task, a difference file is
created with the task COMB.
For the small test, the file named TDDTS.DIFF.1, which contains
the test image minus the master image.
This file is re-written after each task.
If a DDT test has a small number of BITs, re-run the DDT
only on that part.
Then examine the difference file, which can provide a
clue to the type of errors occuring.

Several different types of CPUs are listed in the tables.
Below the CPUs are described.

\begin{tabbing}
\={\bf CVAX-PS}~~~\= A Vax 780 computer using pseudo array processor code. \\
\>{\bf CVAX} \>A Vax 780 computer with FPS 120B array processor. \\
\>{\bf OUTBAX} \>A Vax computer with FPS array processor.\\
\>{\bf Nrao1} \>A Convex C1 computer.\\
\>{\bf Gorilla} \> A Sun 3/60 Workstation with MC68881 co-processor. \\
\>{\bf Kong} \>A Sun 3/60 Workstation with MC68881 co-processor.\\
\> {\bf SAIPS} \>A Sun 3/110 Workstation with MC68881 co-processor.\\
\>{\bf Mandrill} \> A Sun 4/110 Workstation. \\
\> {\bf Trace}\> A Multi-Flow Trace model 1400/300
mini-super computer. \\
\> {\bf Lemur}\> An IBM 6000 (Power Station), model 530. \\
\> {\bf Polaris}\> An IBM 6000 (Power Station), model 540. \\
\end{tabbing}

\section{UV Sorting and Mapping}
The first group of DDTs tests is Sorting and Mapping of \uvdata.
The Small, Medium and Large test results are presented
in tables 1, 2, and 3.

The UVSRT test is evaluated by printing the differences between
the output file and the master file using the task UVDIF.
There are frequently a few differences because several of the
\uvdata points have almost identical U or V values.
The exact order of these points seems to depend
very strongly on the internal floating point format of the
CPU being tested.  Inspection of the UVDIF output quickly
shows whether the different points are just in a slightly
different sort order.

The UVMAP test quality is determined by subtracting
the MASTER image from the TEST images to create a DIFF image.
If the differences are positive, the TEST image is brighter
than the MASTER image.

The DDT mapping results are sensitive to the
number of pixels around the edge of the maps excluded from
image comparison.  For SMALL DDTs 4 pixels should be excluded,
for MEDIUM 8 pixels, and for LARGE exclude 16 pixels.
The number of pixels to exclude is set by the EDGSKP adverb.
The sensitivity of edge pixels is due to the gridding correction
for the Fourier Transform.  Near the edge of the maps,
very small numbers are divided and round off error
becomes important.

\section{APCLN and ASCAL}
The second group of tests check APCLN and the Self Calibration
routine ASCAL.  Very few changes have been made to these
tasks in recent years.  New features for self calibration have
been put in the task CALIB.
The Small, Medium and Large test results are presented in tables 4, 5,
and 6.  The ASCAL success is determined by the number of
difference between the reference data set and the test data set.
There difference listed are frequently due to small differences in
U, V and W values resulting in the \uvdata being sorted differently
in the two data sets.  If there were more than 99 differences in
the \uvdata, the number of differences was listed as ``!''.
The \AIPS task UVDIF lists three types of differences,
{\it a)} flux,
{\it b)} U-V-W coordinate, and
{\it c)} other differences.

\section{Image Deconvolution, MX}
The third group of tables display the results of the task MX.
This task performs three functions;
{\it a)} producing a Fourier transform of the \uvdata,
{\it b)} creating the BEAM or point response function, and
{\it c)} deconvolving the \uvdata to produce a cleaned image.
The task MX requires a considerable fraction of the total
time spent on image analysis
and the REAL time used in the MX Clean step is also listed.

\section{Maximum entropy and DDT notes}
The task VTESS is presented in the fourth group of tables.
The total CPU (not Real) time required to execute the
DDT is given with notes concerning the DDTs.
The most common note is the number of pixels around the
edge of a map which are ignored during comparisons.
Typically a band 4 pixels wide around the edge are ignored
for the Small DDTs.  For Medium DDTs, eight pixel wide
strips are ignored and for Large, sixteen pixels.
This band is set with the DDT adverb EDGSKP.

\section{Future Developments}
Further DDT tasks are needed to test the \AIPS calibration
package and VLBI fring-fitting routines.
Also the task ASCAL is being phased out, to replaced by the
task CALIB.
These functions did not exist in \AIPS when the first
DDTs were developed.

A calibration DDT has been created, which is loaded with the
runfile VLACLOAD, and executed with the runfile VLACEXEC.
This procedure functions similarly to the DDT tests, and
will be described in a future \AIPS memo.

A spectral line DDT is also near completion.
The spectral line calibration and mapping tasks are tested
with this set of procedures.

\clearpage
\newcommand{\hinch}{\hspace{1in}}
\small

\textwidth 7.2in
\linewidth 7.2in
\marginparsep 0in
\oddsidemargin -.5in
\evensidemargin -.5in
\topmargin .5in
\headheight 0in
\headsep 0in
\textheight 9in
\itemindent 1cm
\pretolerance=10000
\parindent 0mm
\baselineskip 7mm

\begin{table}
\begin{center}
\begin{tabular}{cccc|rr|rrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{2}{c|}{UVSRT}&   \multicolumn{5}{c}{UVMAP}  \\
  Date    & Version  &          & New  &   CPU    &    U-V    &    CPU    &    MAP    &    MAP    &   BEAM    &   BEAM   \\
          &          &          & Tst  &  Time    &   Diffs   &   Time    &   Peak    &    RMS    &   Peak    &    RMS   \\

  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)    &   (10)    &  (11)   \\
\hline
\input{ddtuvs}
\hline \hline
\end{tabular}
\end{center}
\caption{Small DDT results for UVSRT and UVMAP}
\end{table}

\begin{table}
\begin{center}
\begin{tabular}{cccc|rr|rrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{2}{c|}{UVSRT}&   \multicolumn{5}{c}{UVMAP}  \\
  Date    & Version  &          & New  &   CPU    &    U-V    &    CPU    &    MAP    &    MAP    &   BEAM    &   BEAM   \\
          &          &          & Tst  &  Time    &   Diffs   &   Time    &   Peak    &    RMS    &   Peak    &    RMS   \\

  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)    &   (10)    &  (11)   \\
\hline
\input{ddtuvm}
\hline \hline
\end{tabular}
\end{center}
\caption{Medium DDT results for UVSRT and UVMAP}
\end{table}

\begin{table}
\begin{center}
\begin{tabular}{cccc|rr|rrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{2}{c|}{UVSRT}&   \multicolumn{5}{c}{UVMAP}  \\
  Date    & Version  &          & New  &   CPU    &    U-V    &    CPU    &    MAP    &    MAP    &   BEAM    &   BEAM   \\
          &          &          & Tst  &  Time    &   Diffs   &   Time    &   Peak    &    RMS    &   Peak    &    RMS   \\

  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)    &   (10)    &  (11)   \\
\hline
\input{ddtuvl}
\hline \hline
\end{tabular}
\end{center}
\caption{Large DDT results for UVSRT and UVMAP}
\end{table}

\clearpage

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrr|rr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{6}{c|}{APCLN}&   \multicolumn{2}{c}{ASCAL}  \\
  Date    & Version  &          & New  &   CPU    &    MAP    &    MAP    &    CPU    &    RES&RES &CPU     &    U-V   \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &   Time    &   Peak&RMS &Time    &   Diffs  \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)&(10)&  (11)  & (12) \\
\hline
\input{ddtaps}
\hline \hline
\end{tabular}
\end{center}
\caption{Small DDT results for APCLN and ASCAL}
\end{table}

\clearpage

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrr|rr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{6}{c|}{APCLN}&   \multicolumn{2}{c}{ASCAL}  \\
  Date    & Version  &          & New  &   CPU    &    MAP    &    MAP    &    CPU    &    RES&RES &CPU     &    U-V   \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &   Time    &   Peak&RMS &Time    &   Diffs  \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)&(10)&  (11)  & (12) \\
\hline
\input{ddtapm}
\hline \hline
\end{tabular}
\end{center}
\caption{Medium DDT results for APCLN and ASCAL}
\end{table}

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrr|rr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{6}{c|}{APCLN}&   \multicolumn{2}{c}{ASCAL}  \\
  Date    & Version  &          & New  &   CPU    &    MAP    &    MAP    &    CPU    &    RES&RES &CPU     &    U-V   \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &   Time    &   Peak&RMS &Time    &   Diffs  \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)     &    (9)&(10)&  (11)  & (12) \\
\hline
\input{ddtapl}
\hline \hline
\end{tabular}
\end{center}
\caption{Large DDT results for APCLN and ASCAL}
\end{table}

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{9}{c}{MX}  \\
  Date    & Version  &          & New  &   CPU    &   MAP &
    MAP   &   BEAM   & BEAM &  CPU     &  Real    &   CLN & CLN \\
          &          &          & Tst  &  Time    &   Peak    &    RMS
&   Peak    &    RMS&Time & Time & Peak   & RMS \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)
&   (8)     &    (9)&(10)&  (11)  & (12)  & (13) \\
\hline
\input{ddtmxs}
\hline\hline
\end{tabular}
\end{center}
\caption{Small DDT results for MX}
\end{table}
\clearpage

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{9}{c}{MX}  \\
  Date    & Version  &          & New  &   CPU    &   MAP &
    MAP   &   BEAM   & BEAM &  CPU     &  Real    &   CLN & CLN \\
          &          &          & Tst  &  Time    &   Peak    &    RMS
&   Peak    &    RMS&Time & Time & Peak   & RMS \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)
&   (8)     &    (9)&(10)&  (11)  & (12)  & (13) \\
\hline
\input{ddtmxm}
\hline \hline
\end{tabular}
\end{center}
\caption{Medium DDT results for MX}
\end{table}

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrrrrrrrr}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{9}{c}{MX}  \\
  Date    & Version  &          & New  &   CPU    &   MAP &
    MAP   &   BEAM   & BEAM &  CPU     &  Real    &   CLN & CLN \\
          &          &          & Tst  &  Time    &   Peak    &    RMS
&   Peak    &    RMS&Time & Time & Peak   & RMS \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)
&   (8)     &    (9)&(10)&  (11)  & (12)  & (13) \\
\hline
\input{ddtmxl}
\hline \hline
\end{tabular}
\end{center}
\caption{Large DDT results for MX}
\end{table}

%\clearpage
\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrr|rl}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{3}{c|}{VTESS}& Total & \\
  Date    & Version  &          & New  &   CPU    &    CLN    &    CLN    &     CPU     &   Notes           \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &    Time     &                   \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)      &    (9)  \\
\hline \input{ddtvts} \hline \hline
\end{tabular}
\end{center}
\caption{Small DDT results for VTESS and Total DDT CPU time}
\end{table}

\clearpage

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrr|rl}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{3}{c|}{VTESS}& Total & \\
  Date    & Version  &          & New  &   CPU    &    CLN    &    CLN    &     CPU     &   Notes           \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &    Time     &                   \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)      &    (9)  \\
\hline
\input{ddtvtm} \hline \hline
\end{tabular}
\end{center}
\caption{Medium DDT results for VTESS and Total DDT CPU time}
\end{table}

\begin{table}[h]
\begin{center}
\begin{tabular}{cccc|rrr|rl}
\hline \hline
Test      &  AIPS    &~~~CPU~~~~& Old  &  \multicolumn{3}{c|}{VTESS}& Total & \\
  Date    & Version  &          & New  &   CPU    &    CLN    &    CLN    &     CPU     &   Notes           \\
          &          &          & Tst  &  Time    &   Peak    &    RMS    &    Time     &                   \\
  (1)     &   (2)    &  (3)     & (4)  & (5)      &   (6)     &   (7)     &   (8)      &    (9)  \\
\hline
\input{ddtvtl}
\hline \hline
\end{tabular}
\end{center}
\caption{Large DDT results for VTESS and Total DDT CPU time}
\end{table}
\end{document}
