New HAP data products; HLA footprint viewer retired (2021 January 13)

NEW DATA PRODUCTS

• The MAST Portal now has new Hubble image products for WFC3 and ACS. The Hubble Advanced Products (HAP) are generated by a new data processing pipeline that runs alongside the standard HST calibration pipeline. The HAP images are intended to replace the HLA images, but they have several advantages over the HLA products:
  1. They use the most recent calibrations.
  2. They are automatically regenerated whenever the HST data get reprocessed (e.g., due to improved calibrations).
  3. They are available with a much shorter delay after the observations.
  4. They are even available for proprietary (exclusive access) observations for use by principal investigators, and will usually be available immediately when datasets become public.
• The HAP products include newer data than was processed for the last HLA data release DR10. The products are still being generated at this time, but soon will replace nearly all the ACS and WFC3 DR10 products.
• In the initial release there are no HAP source lists. After the source list generation has been optimized, a future HAP release will include source lists that will be comparable in quality to (or better than) the HLA source lists.
• Some HLA data products will not be replaced by the HAP project. There is no plan to generate HAP products for WFPC2 or NICMOS images. Those HLA data products will continue to be available. The final release of superseded HLA data products will be preserved as static reference data and are planned to be available indefinitely as MAST High-Level Science Products.
• We welcome feedback on any differences between the HAP and HLA datasets to archive@stsci.edu.

INTERFACE CHANGES

• The HLA footprint view relies on Adobe Flash, which is now retired and is no longer supported. Consequently the footprints are no longer visible in the HLA user interface. Currently there are no plans to replace the footprint viewer. Users who need this view (which is certainly valuable) can use the MAST Portal, which has most of the capabilities of the HLA (plus many additional functions) and which has a similar footprint viewer. The HLA data products are available in the Portal.

Hubble Catalog of Variables (2019 September 24)

NEW DATA PRODUCTS

• The Hubble Catalog of Variables (HCV), a major new High-Level Science Product derived from the Hubble Source Catalog, was released. The HCV is the first homogeneous catalog of variable sources found in the HSC. It includes variable stars in our Galaxy and nearby galaxies, as well as transients and variable active galactic nuclei. The HCV contains 84,428 candidate variable sources (out of 3.7 million HSC sources that were searched for variability) with V <= 27 mag; for 11,115 of them the variability is detected in more than one filter. The number of data points in a light curve range from 5 to 120, and the time baseline ranges from under a day to over 15 years. The released data includes both variable objects and objects identified as "constant".
• The HCV is fully integrated with the MAST user interfaces for the HSC, including the HSC CasJobs and VO TAP database query interfaces and the MAST catalogs simple form interface and query API. There are Python Jupyter notebooks available that show how to access the data in scripts. The ESAC Science Data Centre has created the HCV Explorer, a new online web tool to access, visualize, and interactively explore the HCV.

Hubble Source Catalog Version 3.1 (2019 June 27)

NEW DATA PRODUCTS

• HSC version 3.1 was released on 2019 June 26. It adds to the current HSC version 3.0 by providing proper motions of over 400,000 objects in the augmented Sagittarius Window Eclipsing Extrasolar Planet Search (SWEEPS) HST field. This field is within a few degrees of the Galactic center, and most of the stars belong to the Galactic bulge. The field has been observed by ACS and WFC3 with a time baseline as long as 12 years. The proper motion information is available in the form of database tables within the HSCv3 context of the STScI CasJobs interface. The data can be readily queried by means of SQL through that interface.
• In addition, we provide a Python Jupyter notebook that runs on Python 3.6. The notebook queries the database through a Python interface to CasJobs. It provides statistical information about the photometry and proper motions and includes some science use cases. More details about HSC v3.1 are found here.

New HLA Web Server (2019 March 18)

CONFIGURATION CHANGES

• The HLA web server has been updated, which is expected to fix a relatively long-standing intermittent problem with downloading data via the HLA Cart. Please report problems with the cart or other aspects of the HLA interface to the archive help desk (archive@stsci.edu). .

INTERFACE ENHANCEMENTS

• For scripted access to the HLA SIAP web service, there are some new (optional) parameters that allow restricting the search by the name of the dataset (Visit) or by the date when the dataset was processed (MinDate and MaxDate). See the format=metadata query for details of these and other search parameters.

HLA DR10.1 and Hubble Source Catalog Version 3 (2018 July 5)

NEW DATA PRODUCTS

• Source lists from HLA DR10 have been combined to create version 3 of the Hubble Source Catalog. See the HSC Home Page for details and data access links.
• There are approximately 25% more ACS source lists and almost twice as many WFC3 source lists compared with HSC v2 (using data public as of October 2017).
• The source list photometric quality is significantly improved. These improvements are mainly the result of an improved alignment algorithm used to match exposures and filters in the HLA image processing. There were also improvements and bug fixes for the sky matching algorithm and the Source Extractor background computation that significantly improved the photometry and the incidence of spurious detections near the edges of images.
• Improved astrometric calibration is based on the Gaia DR1 catalog. Gaia has allowed the confident identification of much larger errors in the HST pointing (with shifts up to 100 arcsec correctly identified). In this version of the catalog, 94% of the fields have matches to external astrometric reference catalogs.
• The scatter in magnitudes is measured using the median absolute deviation (MAD) of the differences from the median magnitude. This is more robust and less sensitive to outliers than the RMS used in previous versions.

USER INTERFACE ENHANCEMENTS

• HSCv3 astrometric corrections are used for image cutouts and in the interactive display.
• The HSCv3 catalog is available for overlay in the interactive display. The older HSCv2 and HSCv1 catalogs are also accessible using the HSC Controls in the interactive display.

DR10 Release (2018 January 8)

NEW DATA PRODUCTS

• All ACS and WFC3 data that were public as of 2017 October 1 have been processed. This release includes 25% more ACS images and more than twice as many WFC3 images compared with the previous release. All old ACS and WFC3 images have been replaced in this release. These new source lists are being used for version 3 of the Hubble Source Catalog, which will be released in Spring 2018.
• New deep, wide-field ACS and WFC3 multi-visit mosaic data products have been generated for 1348 fields. The new mosaic pipeline is based on the processing used for the Hubble Frontier Fields images. These images are astrometrically corrected and aligned using HSC version 2. ACS and WFC3 products are drizzled onto a common pixel grid, which makes them easy to use. See the mosaics FAQ for more information.
• Combined (level 2) data products have been added for the ACS/SBC detector. These will usually be the preferred HLA products for the SBC.
• Moving target images have better quality drizzled products (but still do not have level 2 products that combine exposures.)
• The new image processing and source list pipelines fix numerous data problems. A partial list of the improvements:
  • The data processing starts with the HST online cache data, so it uses the best available calibration at the time of HLA pipeline processing.
  • Exposures that have pointing positions that are shifted are aligned using a robust histogram matching method. The alignment corrects both shifts and (small) rotations. The results are dramatically better for the rare images that have large shifts, but they are also systematically better for the much more common case of small (sub-pixel) offsets between exposures. The resulting images are sharper and have more consistent PSFs, which results in better photometry in the HLA source lists.
  • Images that have a combination of short and long exposures previously had poor data products filled with dark speckles in the sky ("grot"). This problem has been largely eliminated by tuning the astrodrizzle image combination parameters.
  • Pixels flagged by the WFC3/IR "blob" flag, which indicates regions of less accurate flat-fielding, are now included in the combined images. Recent experience indicates that the current flat-fields are sufficiently accurate to utilize these pixels. The result is WFC3/IR images with better cosmetic qualities (fewer holes) that are easier to use for analysis.
  • The pipeline can now process subarray observations for all instruments. It also handles other rare observing modes, so that it now is capable of processing all ACS and WFC3 data.
  • The detection image (the "total" image), which is used for source identifcation, is improved in visits that have variable sky.
  • A problem that led to increased photometric errors near image edges in the SExtractor catalogs has been corrected. (This problem and some others were discovered by the Hubble Catalog of Variables project at the National Observatory of Athens.)

USER INTERFACE ENHANCEMENTS

• Previews and the interactive display handle photon-counting detectors such as the ACS/SBC better.
• Color images for mosaic data products can combine ACS/WFC, WFC3/UVIS and WFC3/IR images.
• The PanSTARRS DR1 catalog is included for overlay in the interactive display. See the display help for more information on the PS1 catalog overlay.

DR9 image for NGC 6388 showing misalignment	DR10 image with corrected alignment

Hubble Source Catalog Version 2.1 Release (2017 January 25)

• A new version of the Hubble Source Catalog includes additional spectroscopic matches, new tables that provide median rather than mean magnitudes for sources with multiple measurements, and a table that provides cross-match information between HSCv2 sources and HSCv1 sources.
• See the HSC documentation for more details.

DR9.1 Release (2016 September 29)

NEW DATA PRODUCTS

• Newly processed ACS and WFC3 source lists. Almost all old source lists have been replaced in this catalog. The new source lists have higher quality astrometry and photometry, especially for SExtractor catalogs in crowded fields where the parameters that were used previously led to errors in the segmentation map. Note that these new source lists were used in the construction of version 2 of the Hubble Source Catalog, which is being released simultaneously with HLA DR9.1.

USER INTERFACE ENHANCEMENTS

• Changes have been made to support the release of version 2 of the Hubble Source Catalog (HSC).
• There are several changes in the interactive display:
  • The advanced HSC controls include an option to show either the current HSCv2 catalog or the previous HSCv1 catalog.
  • The HSC-derived astrometric correction (in the advanced HSC controls) is applied by default.
  • The Gaia DR1 catalog is available for overlay.
  • There are new controls to change the catalog overlay region so that sources from a small part of the image can be shown. This greatly improves performance in very large images (e.g., the COSMOS mosaic) or in regions where the HSC is extremely dense on the sky.
  • See the display help for more information on the new display features.

DR9 Release (2015 December 11)

NEW DATA PRODUCTS

• Newly processed ACS image data. The new pipeline products use Drizzlepac and Astrodrizzle and have a data format that matches the WFC3 HLA products.
• Improved ACS source lists. Source lists are available for more observations and go significantly deeper than the previous versions. Note that these new source lists will be used in the construction of version 2 of the Hubble Source Catalog.
• Almost all ACS data that were public as of 2015 June 9 have been processed. The number of ACS visits processed is larger by a factor of 1.4 compared with DR8, and the number of catalogs has increased even more than than (1.6 times as many SExtractor catalogs, and 2.1 times as many DAOphot catalogs.)
• One important area where the DR9 ACS images are degraded compared with the older images is in astrometry. Astrometric corrections have not been made for the DR9 ACS data; the astrometric accuracy of the DR9 images is the same as for the standard HST archive images. The astrometry for both ACS and WFC (as well as WFPC2) will be updated as part of the incremental DR9.2 data release planned for early 2017. That will greatly improve the accuracy of the astrometry for all the HLA catalogs and images by incorporating the much improved astrometry that is determined during the construction of version 2 of the Hubble Source Catalog.

NEW HIGH-LEVEL SCIENCE PRODUCTS

• LEGUS (Legacy Extragalactic UV Survey)
• M83MOS (M83 Mosaics)
• WISP (WFC3 Infrared Spectroscopic Parallel Survey)
• Also added additional data for other HLSP projects including the Frontier Fields, Hubble Heritage, and UVUDF.

DR8.1 Release (2015 February 25)

USER INTERFACE ENHANCEMENTS

• Changes have been made to support the release of version 1 of the Hubble Source Catalog (HSC).
• The advanced HSC controls in the interactive display now include an option to choose whether MAGAUTO or MAGAPER2 magnitudes are shown. See the display help for more information.
• A new higher performance approach is being used for the HSC overlay query in the interactive display. This supports catalogs with up to 50,000 sources with reasonably good performance.
• The HSC overlay catalog (accessed by clicking on HSC sources in the display) now includes all filters that have data and has some additional columns. See the HSC help for more information.

DR8 Release (2014 December 3)

NEW DATA PRODUCTS

• Improved WFC3 image data. Use of Drizzlepac and Astrodrizzle greatly improve alignnment and final image quality. Many images that were previously rejected due to alignment problems now have HLA combined images. Most WFC3 images have been reprocessed using the new pipeline.
• Improved WFC3 source lists. Source lists are available for more observations and go deeper than the previous versions.
• All WFC3 non-GRISM data that were public as of June 1, 2014 have been processed. The number of WFC3 images and catalogs is larger by a factor of 2.6 compared with DR7.2.

USER INTERFACE ENHANCEMENTS

• For visits that have more than 3 filters available, the color image entries have been modified so that the interactive display has access to all those images. You can change the red/green/blue images using the pop-up menus in the interactive display.
• When overlaying objects from the Hubble Source Catalog in the interactive display, the new HSC option to remove null columns is used. That makes the pop-up information (accessible by clicking on HSC sources) much more useful.
• The interactive display automatically resamples PHAT images when displaying color images because the PHAT images from different filters are not aligned. This slows down the display slightly but makes the color display useful for PHAT.
• The handling of matches from source lists when the same dataset includes more than one source has changed. Now the same image can show up multiple times in the list, once for each source in that image. This is particularly helpful in the Images view with image cutouts, where it will show cutouts for all sources in in image rather than just one.

NEW HIGH-LEVEL SCIENCE PRODUCTS

• Frontier Fields
• HLASTARCLUSTERS
• ISON (Comet ISON)
• M82
• UVUDF
• Also added additional data for many other HLSP projects.

DR7.2 Release (2013 October 23)

NEW DATA PRODUCTS

• Improved WFC3 source lists. Now 85% of WFC3 combined images have either a SExtractor or a DAOphot source list, and the quality of the lists has been substantially improved by flagging spurious sources. See the source list FAQ for more information.

DR7.1 Release (2013 May 8)

USER INTERFACE ENHANCEMENTS

• The interactive display includes additional options related to the beta release of the Hubble Source Catalog (HSC). The HSC can be overlaid on the image, and the image astrometry can be shifted using corrections derived during the catalog construction.
• The display also now allows the filters used for each color to be changed. See the interactive display help for more information.

NEW Hubble Source Catalog - Beta 0.2(HSC)

• The Beta 0.2 version of the Hubble Source Catalog (HSC) is now available. The primary goal of the HSC is to combine the tens of thousands of individual visit-based, general-purpose source lists from the Hubble Legacy Archive (HLA - data release DR7.1) into a single master catalog, using algorithms described in Budavari and Lubow (2012)
• The HSC currently contains data from ACS/WFC and WFPC2 Source Extractor source lists. The primary enhancements for Beta 0.2 are improvements in the matching algorithms, development of a HSC Summary Search form, and access to a version of the HLA interactive display that allows the user to overlay the catalog on HLA images.

NEW HLA source list modification

• The magnitude zero points used in the source list generation have been adjusted for about 1100 WFPC2 images and 250 ACS images.

NEW HLSP

• HUDF12 (The Hubble Ultra Deep Field 2012)
• XDF (Hubble eXtreme Deep Field)
• Also, the most current PHAT, CANDELS, CLASH and Heritage programs.

DR7 Release (2012 November 2)

NEW DATA PRODUCTS

• HLSP Spectra: 586 spectra from the StarCAT project
• Additional HLSP from the Hubble Treasury programs CANDELS, CLASH, PHAT
• New HLSP imaging products:ORION , BORG,GHOSTS

USER INTERFACE ENHANCEMENTS

• Scatter plotting tool that allows users to plot the properties HLA source lists. See the Scatter plotting tool help for more information.
• The ability to view spectral HLSP through the Interactive Display
• Much faster overlay of catalogs in interactive display
• Spectrum/line plot tool rewritten in HTML5
• Line plots now available for HLSP images
• Footprint view automatically adapts for large or all-sky searches to show filtered sky area

WFC3 daophot source list magnitude error (2012 September 12) - Fixed (2012 October 2)

DR6.1 Release (2012 August 7)

• WFC3 source lists: includes 5201 daophot and 7019 sextractor source lists from observations that have a release date up to 11-30-2011

DR6 Release (2012 January 5)

NEW DATA PRODUCTS

• WFC3 image data: includes ~62% of data from observations that have a release date up to 11-30-2011
• New High Level Science Products
• Additional ACS (updated to release date 2-16-2011), WPFC2 and NICMOS data

USER INTERFACE ENHANCEMENTS

• Interactive Display enhancements, featuring:
  • Contrast setting (darker/lighter) is consistent when the image is zoomed
  • Save button that creates a JPEG image showing the same region of the image and contrast as the current view. This can be useful for generating publication-quality images from the HLA display. See thedisplay help for more information.
• The inventory and images interfaces now include a "Select All" button that can be used to select all the datasets in the current list.
• The HLA Simple Image Access Protocol server is now fully compliant with the Virtual Observatory standards. The HLA SIAP server will be added to the VO registry for easy access to HLA data from scripts and other VO tools.
• Searches using lists of positions now show the number of lines in the list.

Cart Download Issue (2011 May 11) - Bug Fixed, except in IE (2011 August 10)

• A bug has been discovered in the HLA cart download procedures that can lead a truncation of FITS files when they are downloaded using the "Download Sequentially" cart option. The default download mode as a zipped file always works correctly. The sequential download method was introduced in DR5. We have disabled the option to download files separately rather than as a zipped file until this problem has been corrected.
• Users who have downloaded data using the sequential download method are advised that their files could be missing part of the final FITS extension. The files can be retrieved again using the Zip option to ensure that they are correct. We are currently unsure how widespread this problem is and are working to identify the cause. When the bug has been fixed, we will restore the sequential download option to the HLA cart.

DR5 Release (2011 March 8)

USER INTERFACE ENHANCEMENTS

• A completely new footprint interface, featuring:
  • Zooming in/out capability
  • Panning
  • Fully integrated table, so that selections in one screen remain valid in the others
  • Selection across instruments possible
  • Option to search through proprietary and/or non-proprietary data
• Improved download flow from shopping cart
  • Single or sequential download
  • Partial downloads can be restarted
• Sources clickable in interactive display
• When filtering RA or Dec columns, the values may be entered either in decimal degrees or in "sexagesimal" format (hh:mm:ss, dd:mm:ss)
• Multiband source list (ACS and WFPC2)

NEW DATA PRODUCTS

• Completed WFPC2 processing (over 300 new visits with source lists)
  • lien - there are some source lists missing for total images
  • lien - 1070 wfpc2 visits to be processed
• New High Level Science Products
• ECF-generated ACS and NICMOS extracted GRISM data hosted at STScI

ST-ECF ACS/WFC Grism Final Release (2010 July 6)

• ST-ECF has released a full set (47,919) of extracted ACS grism spectra obtained from 153 archival fields.
• These data and their footprints are available through the HLA.

DR4 Data Repair (21 May 2010)

• Fixed a data problem for a level=best search (the default) that returns both level 1 and level 2 WFPC2 data. For a level=best search, level 2 data should supersede the corresponding level 1 data.

DR4 Incremental Update (May 2010)

• Reprocessed WFPC2 data (produced at CADC) for ~1000 images not included in initial DR4 release.
• Completed WFPC2 source lists (now available for 86% of images).
• Added WFPC2 images collected using area mode (on-chip binning).

DR4 (March 2010)

NEW DATA PRODUCTS

• Fully reprocessed NICMOS data
• Fully reprocessed WFPC2 data (produced at CADC) with new source lists
• Access to WFC3 early release science data
• New high-level science products
• Additional ACS multi-visit mosaics
• Updated ACS combined images

USER INTERFACE ENHANCEMENTS

• Plotting tool allows for more dynamic user interaction, including zooming and panning
• Spectra from FOS and GHRS are displayed in the plotting tool using the "Interactive Display" links in the inventory and images views.
• COS and WFC3 observations are included in the search results and footprints
• Proprietary data are included in the search results plus footprints and can be retrieved from DADS (by privileged users) via the HLA cart
• Search by IPPPSOOT exposures and HST visit names (propid_visit)
• Inventory table updates:
  • New column - ReleaseDate, gives the date when proprietary data become public
  • New column - NReleaseDate is a seconds-since-1970 numerical version of ReleaseDate
  • New columns - DOACat/SEXcat give processing status for catalogs for ACS and WFPC2 images
  • New column - regionSTCS, gives the STC/S footprint region description string
  • Links for proprietary data are flagged in yellow to alert users
  • Custom column ordering is saved for later use (when the same computer is used)
• Consolidated Help Center holds HLA resources such as FAQ, Help, Movies etc., and includes a new HLA documentation search tool.
• Source lists are dynamically generated from our databases, which improves consistency between the various catalog sources and makes it possible to download them from the inventory
• Search results get automatically gzip-compressed when the browser permits it (which is usually the case). This will make HLA searches noticeably faster on slower network connections.
• The interactive display shows the original image pixel value in the coordinate box when the mouse cursor stops moving.
• The interactive display "S" keystroke command (which inserts the current mouse position into the search box) now works on high-level science product images as well as HLA images.
• A problem where STIS data downloaded through the cart had the wrong name has been corrected. STIS data are now downloaded as tar files that include all the products associated with an observation (extracted spectra, trailer files, etc.)
• The JPEG images from the HLA image cutout server now include FITS world coordinate keywords in a format compatible with the Aladin image display, so JPEG images can be accessed directly by Aladin.

DR3 (May 2009)

NEW DATA PRODUCTS

• HLA drizzled combined images for ~30% of NICMOS data
• Sample of 1235 ACS extracted grism spectra (provided by ST-ECF)
• Preliminary DAOphot and SExtractor source lists for ~80% of WFPC2 images
• Sample prototypes of ACS multi-visit mosaics

USER INTERFACE ENHANCEMENTS

• User search list positions are accepted in a variety of input formats
• Selected datasets (e.g., from clicking on footprints) can be added to cart
• Selection of datasets can be used for sorting and filtering (e.g., to show selected data at the top of the table)
• Cart can be used to retrieve both DADS and HLA data products
• Cart includes estimated download data volume
• GALEX catalog is available for overlay in interactive display
• Advanced contrast controls in interactive display can be used to change color balance

DR2.5 (December 2008)

NEW DATA PRODUCTS

• Contributed high-level science products from MAST fully incorporated into HLA (e.g., UDF, COSMOS, GOODS...)

DR2 (September 2008)

NEW DATA PRODUCTS

• Combined HLA images for ~95% of WFPC2 data (produced at CADC)
• Combined HLA images for ~95% of ACS observations
• Both DAOphot and SExtractor source lists for 70% of ACS observations

USER INTERFACE ENHANCEMENTS

• (Shopping) Cart to allow users to download multiple files at once
• Enhanced plotting capabilities
• Additional catalog overlays in interactive display (SDSS, 2MASS, GSC2, FIRST)
• User defined search lists
• Much faster footprint interface
• Additional advanced search options (Proposal ID, Moving Targets, Filter)

DR1 (February 2008)

NEW DATA PRODUCTS

• HLA combined images for ~90% of ACS data
• NICMOS grism extractions for ~80% of available data (produced at ST-ECF)
• DAOphot source lists for ~50% of ACS images
• Prototype SExtractor source lists for ~10% of ACS images
• A few prototype combined WFPC2 images (produced at CADC)

USER INTERFACE ENHANCEMENTS

• Footprints for ACS, WFPC2, STIS, NICMOS, FOS, and GHRS
• STIS images accessible in HLA interactive display
• Non-HLA data products included in inventory and images, accessible through DADS
• Ability to filter inventory table to select observations of interest
• Readout of mouse RA/Dec position in interactive display
• Plot cuts of rows or columns through images (or 2-D spectra) in interactive display
• Introductory training movies on how to use the HLA

EDR (August 2007)

• Data online for immediate access
• Footprint service to make it easier to browse and download
• Extensive "composite images" (e.g., stacked, color)
• Improved absolute astrometry (i.e., from ~ 1-2" to ~ 0.3") when there is sufficient overlap with the Guide Star Catalog 2 (GSC2)
• Interactive image display with ability to overlay source lists
• HLA combined images for ~25% of ACS data
• Footprints only for WFPC2 and STIS data
• Prototype ACS source lists
• NICMOS grism extractions for ~10% of G141 data (produced at ST-ECF)