A Breakthrough in Astronomical Software Development
Raymond Plante
Doug Roberts
% R.\ M.\ Crutcher\altaffilmark{1}
% and the start of your text. \begin{abstract} % Leave intact XAssist is a NASA AISR-funded project for the automation of X-ray astrophysics, with emphasis on galaxies. It is nearing completion of its initially funded effort, and is working well for {Chandra} and {ROSAT} data. Initial support for {XMM-Newton} data is present as well. It is capable of data reprocessing, source detection, and preliminary spatial, temporal and spectral analysis for each source with sufficient counts. The bulk of the system is written in Python, which in turn drives underlying software (CIAO for {Chandra} data, etc.). Future work will include a GUI (mainly for beginners and status monitoring) and the exposure of at least some functionality as web services. The latter will help XAssist to eventually become part of the VO, making advanced queries possible, such as determining the X-ray fluxes of counterparts to HST or SDSS sources (including the use of unpublished X-ray data), and add the ability of ``on-the-fly'' X-ray processing. Pipelines are running on {ROSAT}, {Chandra} and now {XMM-Newton} observations of galaxies to demonstrate XAssist's capabilities, and the results are available online (in real time) at \htmladdnormallink{http://www.xassist.org}{http://www.xassist.org}. XAssist itself as well as various associated projects are available for download.
P3.4.tex
%ADASS_PROCEEDINGS_FORM%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
%
% TEMPLATE.TEX -- ADASS VIII (98) ASP Conference Proceedings template
% Updated by N. Manset for ADASS IX (99), F. Primini for ADASS 2000,
% and D.Bohlender for ADASS 2001.
%
% Use this template to create your proceedings paper in LaTeX format
% by following the instructions given below.  Much of the input will
% be enclosed by braces (i.e., { }).  The percent sign, "%", denotes
% the start of a comment; text after it will be ignored by LaTeX.  
% You might also notice in some of the examples below the use of "\ "
% after a period; this prevents LaTeX from interpreting the period as
% the end of a sentence and putting extra space after it.  
% 
% You should check your paper by processing it with LaTeX.  For
% details about how to run LaTeX as well as how to print out the User
% Guide, consult the README file.  
% You should also consult the sample LaTeX papers,
% sample1.tex and sample2.tex, for examples of including figures, html
% links, special symbols, and other advanced features.
%
% If you do not have access to the LaTeX software or a laser printer
% at your site, you can still prepare your paper following the
% instructions in the User Guide.  In such cases, the editors will
% process the file and make any necessary editorial adjustments.
% 
%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%%
% 
\documentstyle[11pt,adassconf]{article}  % Leave intact

\begin{document}   % Leave intact

%-----------------------------------------------------------------------
%			    Paper ID Code
%-----------------------------------------------------------------------
% Enter the proper paper identification code.  The ID code for your
% paper is the session number associated with your presentation as
% published in the official ADASS 2000 conference proceedings.  You can
% find this number locating your abstract in the printed proceedings
% that you received at the meeting or on-line via 
% http://adass2002.stsci.edu; the ID code is the letter/number sequence 
% proceeding the title of your presentation.  
%
% This will not appear in your paper; however, it allows different
% papers in the proceedings to cross-reference each other.
%
% EXAMPLE: \paperID{O4.1}
% EXAMPLE: \paperID{P7.7}
%
% Note that you should only have one \paperID, and it should not
% include a trailing period.  

\paperID{P3.4}

%-----------------------------------------------------------------------
%		            Paper Title 
%-----------------------------------------------------------------------
% Enter the title of the paper.
%
% EXAMPLE: \title{A Breakthrough in Astronomical Software Development}
% 

\title{XAssist: A System for the Automation of X-ray Astrophysics Analysis}

%-----------------------------------------------------------------------
%		          Authors of Paper
%-----------------------------------------------------------------------
% Enter the authors followed by their affiliations.  The \author and
% \affil commands may appear multiple times as necessary (see example
% below).  List each author by giving the first name or initials first
% followed by the last name.  Authors with the same affiliations
% should grouped together. 
%
% EXAMPLE: \author{Raymond Plante, Doug Roberts, 
%                  R.\ M.\ Crutcher\altaffilmark{1}}
%          \affil{National Center for Supercomputing Applications, 
%                 University of Illinois Urbana-Champaign, Urbana, IL
%                 61801}
%          \author{Tom Troland}
%          \affil{University of Kentucky}
%
%          \altaffiltext{1}{Astronomy Department, UIUC}
%
% In this example, the first three authors, "Plante", "Roberts", and
% "Crutcher" are affiliated with "NCSA".  "Crutcher" has an alternate 
% affiliation with the "Astronomy Department".  The fourth author,
% "Troland", is affiliated with "University of Kentucky"

\author{A. Ptak}
\affil{Johns Hopkins University}

\author{R. Griffiths}
\affil{Carnegie Mellon University}

%-----------------------------------------------------------------------
%			 Contact Information
%-----------------------------------------------------------------------
% This information will not appear in the paper but will be used by
% the editors in case you need to be contacted concerning your
% submission.  Enter your name as the contact along with your email
% address.
% 
% EXAMPLE:  \contact{Dennis Crabtree}
%           \email{crabtree@cfht.hawaii.edu}
%

\contact{Andy Ptak}
\email{ptak@pha.jhu.edu}

%-----------------------------------------------------------------------
%		      Author Index Specification
%-----------------------------------------------------------------------
% Specify how each author name should appear in the author index.  The 
% \paindex{ } should be used to indicate the primary author, and the
% \aindex for all other co-authors.  You MUST use the following
% syntax: 
%
% SYNTAX:  \aindex{LASTNAME, F. M.}
% 
% where F is the first initial and M is the second initial (if
% used).  This guarantees that authors that appear in multiple papers
% will appear only once in the author index.  
%
% EXAMPLE: \paindex{Crabtree, D.}
%          \aindex{Manset, N.}        
%          \aindex{Veillet, C.}        

\paindex{Ptak, A.}
\aindex{Griffiths, R. }     % Remove this line if there is only one author

%-----------------------------------------------------------------------
%			Subject Index keywords
%-----------------------------------------------------------------------
% Enter up to 6 keywords describing your paper.  These will NOT be
% printed as part of your paper; however, they will be used to
% generate the subject index for the proceedings.  There is no
% standard list; however, you can consult the indices for past ADASS
% proceedings (http://iraf.noao.edu/ADASS/adass.html). 
%
% EXAMPLE:  \keywords{visualization, astronomy: radio, parallel
%                     computing, AIPS++, Galactic Center}
%
% In this example, the author noticed that "radio astronomy" appeared
% in the ADASS VII Index as "astronomy" being the major keyword and
% "radio" as the minor keyword.

\keywords{automation, astronomy: x-ray, astronomy:galaxies}

%-----------------------------------------------------------------------
%			       Abstract
%-----------------------------------------------------------------------
% Type abstract in the space below.  Consult the User Guide and Latex
% Information file for a list of supported macros (e.g. for typesetting 
% special symbols). Do not leave a blank line between \begin{abstract} 
% and the start of your text.

\begin{abstract}          % Leave intact
XAssist is a NASA AISR-funded project for the automation of X-ray
astrophysics, with emphasis on galaxies.  It is nearing completion of
its initially funded effort, and is working well for {\it Chandra} and
{\it ROSAT} 
data.  Initial support for {\it XMM-Newton} data is present as well.
It is capable of data reprocessing, 
source detection, and preliminary 
spatial, temporal and spectral analysis for each source with
sufficient counts.  The bulk of the system is written in Python, which
in turn drives underlying software (CIAO for {\it Chandra} data,
etc.).  Future work will include a GUI (mainly for beginners and
status monitoring) and the exposure of at least some functionality as
web services.  The latter will help XAssist to eventually become part
of the VO, making advanced queries possible, such as determining the
X-ray fluxes of counterparts to HST or SDSS sources (including the use
of unpublished X-ray data), and add the ability of ``on-the-fly'' X-ray
processing.  Pipelines are running on {\it ROSAT}, {\it Chandra} and now
{\it XMM-Newton} observations of galaxies to demonstrate XAssist's
capabilities, and the results are available online (in real time) at
\htmladdnormallink{http://www.xassist.org}{http://www.xassist.org}.
XAssist itself as well as various associated
projects are available for download.
\end{abstract}

%-----------------------------------------------------------------------
%			      Main Body
%-----------------------------------------------------------------------
% Place the text for the main body of the paper here.  You should use
% the \section command to label the various sections; use of
% \subsection is optional.  Significant words in section titles should
% be capitalized.  Sections and subsections will be numbered
% automatically. 
%
% EXAMPLE:  \section{Introduction}
%           ...
%           \subsection{Our View of the World}
%           ...
%           \section{A New Approach}
%
% It is recommended that you look at the sample papers, sample1.tex
% and sample2.tex, for examples for formatting references, footnotes,
% figures, equations, html links, lists, and other special features.  

\section{Introduction}
We are currently in a renaissance for X-ray astronomy, with two major
missions, {\it Chandra} and {\it XMM-Newton}, currently operating.
These missions are producing large amounts of archival data, which is
supplementing 
existing databases from missions such as {\it ROSAT} and {\it ASCA}.
Historically, only X-ray ``experts'' usually attempted the analysis of
X-ray data.  This is because there were fundamental differences in the
analysis of X-ray data compared with other bandpasses, most notably
the fact that individual photons are detected as opposed to the
accumulation of (only) spectra or images.  Most modern X-ray detectors are
imaging spectrometers so each observation results in a photon list
from which images, spectra and light curves can be extracted.  In
general the numbers of photons are small so Poissonian rather
than Gaussian statistical methods must be used.  The spectral and
spatial resolution of most detectors is moderate at best and
forward-fitting convolution methods are needed to properly fit the
data.  All of these factors limit the accessibility of the X-ray data
to non-experts.  In addition each mission tends to
have its own unique software package for the reduction and analysis of
the data, and X-ray data often require reprocessing as the calibration
improves.  These latter two factors also limit the ability of experts
to take advantage of all available data for a given project,
particularly large-scale surveys.

We have developed a software package to address these concerns.
XAssist is capable of
performing data reduction and preliminary 
analysis for {\it ROSAT}, {\it Chandra} and {\it XMM-Newton} data.  It is fully
automatic making it well-suited for surveys, as well as for the
reprocessing of existing data.  Below we will discuss its
capabilities and prospects for the future.

\section{Capabilities}
XAssist currently has the following capabilities:
\begin{itemize}
\item Downloads data
\item Reprocesses data
\item Creates exposure maps and detector masks (if possible)
\item Detects sources (using built-in routine for {\it ROSAT} and {\it ASCA}, CIAO
  wavdetect for {\it Chandra}, etc.) 
\item Fits each source with "simple" (i.e., not including point-spread
  function) model to establish source extent and (Poisson-correct)
  significance (using the stand-alone python program
  \htmladdnormallinkfoot{ximgfit}{http://www.xassist.org/xassist/Download.jsp}) 
\item Flags extended, confused and problematic sources
\item Computes median (or mean) background level
\item Excludes times of high background
\item Extracts spectra, ``postage stamp'' images, and light curves of
  each source for more detailed analysis (a simple power-law model is
  fit to sources with more than 100 sources)
\item Optionally performs chip-by-chip analysis (relevant just for
  {\it Chandra} right now) 
\item Analysis can be restricted to an energy band
\item Large emphasis on detailed reporting
\item Looks for correlations of X-ray sources with astronomy databases
  and provides links on the detailed source web reports to query
  Simbad (see below)
\end{itemize}

Note that in the case of {\it Chandra} analysis, most of the data reduction
steps are based on the ``threads'' reported in the
\htmladdnormallinkfoot{{\it Chandra} web site}{http://cxc.harvard.edu}.
Most parameters controlling XAssist are read from IRAF/FTOOLS-style
parameter file, and can be set on the command-line as well (allowing
for the automated setup of XAssist for surveys).  XAssist
can be run (and configured) interactively (with a text-based interface).

\section{Sample Output}
Figure \ref{n1569_chandra_dss_fig} shows images created as part of the
report for the processing of the {\it Chandra} observation NGC 1569.  Figure
\ref{n1569_chandra_src2_fig} shows the detailed report generated for a
source.  While there are admittedly ``warts'' that occur (as in any
automated system), this example demonstrates that even in moderately
crowded fields the system performs well and continues to the point of
fitting a power-law spectrum model to the source spectrum.  Obviously,
this opens a powerful possibility for virtual observatories, namely that
searches could be performed on high-level quantites such as spectral
slope.  Even though human inspection would of course still be
necessary for science-grade results, an automated system such as this
could cull samples and produce useable results for many of the
sources, both of which would be particularly useful prior to observing
proposal deadlines (especially if the data of interest had only
recently become publically available and had not been published yet).
Queries are also submitted to HEASARC to find correlations of X-ray sources
with 2$\mu$ass, USNO, FIRST, and other catalogs.
\begin{figure}[htbn]
\epsscale{0.75}
%\plottwo{center_acis_acisf00782_pi14-548_cl_sm2.00.eps}{center_acis_acisf00782_pi14-548_dss.eps}
\plottwo{P3.4_1a.eps}{P3.4_1b.eps}
\caption{{\it Chandra} ACIS (left) and DSS (right) images of NGC 1569.
  Detected X-ray sources are marked in both images.  The color code
  is:green = point source, red = problematic/questionable source, blue
  = extended source, magenta = asymmetric source (may be extended),
  cyan = estimated detector boundary.\label{n1569_chandra_dss_fig}}
\end{figure}

\begin{figure}[htbn]
\epsscale{0.7}
%\plotone{n1569_src2.ps}
\plotone{P3.4_2.eps}
\caption{Detailed web page report for a source in the {\it Chandra}
  observation of NGC 1569. \label{n1569_chandra_src2_fig}}
\end{figure}

\section{Pipelines}
To demonstrate XAssist's capabilities,
\htmladdnormallinkfoot{pipelines}{http://www.xassist.org/xassist/Pipelines.jsp}
have been set up to run XAssist on {\it ROSAT} HRI and {\it Chandra} observations
of RC3 galaxies.  The {\it ROSAT} analysis is nearing completion, and
crontab jobs are checking for the public release of {\it Chandra} observations
that may containing RC3 galaxies in the FOV of view, and these data sets are
downloaded (using the script cda.py also available for download).
{\it XMM-Newton} has recently been added as a supported mission, and we are
in the process of establishing a pipeline for public {\it XMM} data.  The
pipeline products are searchable and work is in progress to allow users to
request fields to be added to the processing lists.

\section{Current and Future Work}
\begin{itemize}
\item Finishing off details (especially for {\it ASCA} and {\it ROSAT} PSPC
  analysis)
\item Better installation and configuration support (including the
  creation of a GUI)
\item Web access to XAssist
\item Web services: limited status reporting is already available using
  SOAP and it will be possible to request full or partial processing
  of data via SOAP requests
\end{itemize}
%-----------------------------------------------------------------------
%			      References
%-----------------------------------------------------------------------
% List your references below within the reference environment
% (i.e. between the \begin{references} and \end{references} tags).
% Each new reference should begin with a \reference command which sets
% up the proper indentation.  Observe the following order when listing
% bibliographical information for each reference:  author name(s),
% publication year, journal name, volume, and page number for
% articles.  Note that many journal names are available as macros; see
% the User Guide listing "macro-ized" journals.   
%
% EXAMPLE:  \reference Hagiwara, K., \& Zeppenfeld, D.\  1986, 
%                Nucl.Phys., 274, 1
%           \reference H\'enon, M.\  1961, Ann.d'Ap., 24, 369
%           \reference King, I.\ R.\  1966, \aj, 71, 276
%           \reference King, I.\ R.\  1975, in Dynamics of Stellar 
%                Systems, ed.\ A.\ Hayli (Dordrecht: Reidel), 99
%           \reference Tody, D.\  1998, \adassvii, 146
%           \reference Zacharias, N.\ \& Zacharias, M.\ 2003,
%                \adassxii, \paperref{P7.6}
% 
% Note the following tricks used in the example above:
%
%   o  \& is used to format an ampersand symbol (&).
%   o  \'e puts an accent agu over the letter e.  See the User Guide
%      and the sample files for details on formatting special
%      characters.  
%   o  "\ " after a period prevents LaTeX from interpreting the period 
%      as an end of a sentence.
%   o  \aj is a macro that expands to "Astron. J."  See the User Guide
%      for a full list of journal macros
%   o  \adassvii is a macro that expands to the full title, editor,
%      and publishing information for the ADASS VII conference
%      proceedings.  Such macros are defined for ADASS conferences I
%      through XI.
%   o  When referencing a paper in the current volume, use the
%      \adassxii and \paperref macros.  The argument to \paperref is
%      the paper ID code for the paper you are referencing.  See the 
%      note in the "Paper ID Code" section above for details on how to 
%      determine the paper ID code for the paper you reference.  
%
%\begin{references}




%\end{references}

% Do not place any material after the references section

\end{document}  % Leave intact
