VISTA Data Flow System

Science Archive Deliverables

The CDR for the v1 Science Archive occurred in April 2003 where the design was generally approved.

1. VDFS-v1(WFCAM) Delivered end Q4 2003; ready for WFCAM commissioning
   Will allow access to the basic data products, will have a web-based interface, and documentation. It will include access to 2MASS, SSS, SDSS-DR1, and USNO-B catalogues as well as WFCAM data. It will allow the user to :
   · Browse the database of calibrated observation frames and associated catalogues, and download any one of these
   · Produce small images and associated multi-colour merged catalogues, from either the observation frame collection, or the accumulating UKIDSS surveys
   · Produce mosaic images on demand up a width of 0.8 degrees
   · Search UKIDSS surveys on a position rectangle or circle in Celestial or Galactic co-ordinates, and within a radius of a resolvable source name
   · Allow joint position searches as above, and produce merged catalogues, with the combination of UKIDSS plus any of 2MASS, SSS, SDSS-DR1
   · Make SQL-like queries in any sensible arithmetic combination of the catalogue parameters, for example colours
   · Allow general queries on the combination of UKIDSS, 2MASS, SSS, SDSS-DR1
   
2. VDFS-v2(WFCAM) released end Q4 2004
   Will be compliant with any relevant protocols and formats produced as part of the various Virtual Observatory initiatives, and will work especially closely with AstroGrid. It will allow the user to :
   · Search open-time data as well as UKIDSS data
   · Plot returned parameters in histograms
   · Plot pairs of returned parameters as (x,y) graphs
   · Generate on request images and catalogues across observation frame boundaries
   · Generate mosaic and stacked images and source catalogues from any area, from user specified observation frames
   · Choose a range of stacking and source extraction algorithms on request, with a menu of tunable parameters
   Probably other simple user tools will be available, such as the SED tool developed for the AVO first-light demo with the GOODS dataset. v2 will also :
   · Return catalogues and similar data in VOTable format, and images using binary data access protocols still under development
   · Recast all the V1 services as web services for publication in the AstroGrid Registry
   
3. VDFS-v3(WFCAM) released end Q4 2005
   · Develop (or install) a suite of advanced visualisation tools, e.g. pannable large area imaging, multi-dimensional catalogue plotting and rotation
   · Develop (or install) a suite of data analysis tools, e.g. cluster analysis, principal component analysis, etc
   · Implement both the above as server-based tools
   · Develop a system to allow uploadable user-specified analysis algorithms
   · Recast our web services as grid services, in order to allow queries and analysis using shared managed resources with other data centres
   
4. VDFS-v4 released Q4 2006 (ESO deliverables in Q3) to operate on VISTA data
   · as for v3 except handles the much greater VISTA data rates
   
5. VDFS-v5 released end Q2 2008 final release for VISTA after 18 months shakedown period
   · incorporates changes from experience with running actual VISTA data, and completion of interfaces to the VO.