Hubble Advanced Products API¶
These modules provide the basic functionality used to process automatically data using this package to apply the distortion models to the WCS of HST observations and to verify the alignment of the observations.
haputils.astrometric_utils¶
Utilities to support creation of astrometrically accurate reference catalogs
The function, create_astrometric_catalog, allows the user to query an astrometric catalog online to generate a catalog of astrometric sources that should fall within the field-of-view of all the input images.
This module relies on the definition of an environment variable to specify the URL of the astrometric catalog to use for generating this reference catalog.
ASTROMETRIC_CATALOG_URL -- URL of web service that can be queried to
obtain listing of astrometric sources,
sky coordinates, and magnitudes.
-
drizzlepac.haputils.astrometric_utils.
create_astrometric_catalog
(inputs, catalog='GAIADR2', output='ref_cat.ecsv', gaia_only=False, table_format='ascii.ecsv', existing_wcs=None, num_sources=None, use_footprint=False, full_catalog=False)[source]¶ Create an astrometric catalog that covers the inputs’ field-of-view.
Parameters: - input : str, list
Filenames of images to be aligned to astrometric catalog
- catalog : str, optional
Name of catalog to extract astrometric positions for sources in the input images’ field-of-view. Default: GAIADR2. Options available are documented on the catalog web page.
- output : str, optional
Filename to give to the astrometric catalog read in from the master catalog web service. If None, no file will be written out.
- gaia_only : bool, optional
Specify whether or not to only use sources from GAIA in output catalog
- existing_wcs :
HSTWCS
existing WCS object specified by the user
- num_sources : int
Maximum number of brightest/faintest sources to return in catalog. If
num_sources
is negative, return that number of the faintest sources. By default, all sources are returned.- use_footprint : bool, optional
Use the image footprint as the source identification bounds insted of using a circle centered on a given RA and Dec? By default, use_footprint = False.
- full_catalog : bool, optional
Return the full set of columns provided by the web service.
Returns: - ref_table :
Table
Astropy Table object of the catalog
Notes
This function will point to astrometric catalog web service defined through the use of the ASTROMETRIC_CATALOG_URL environment variable.
-
drizzlepac.haputils.astrometric_utils.
get_catalog
(ra, dec, sr=0.1, epoch=None, catalog='GSC241')[source]¶ Extract catalog from VO web service.
Parameters: - ra : float
Right Ascension (RA) of center of field-of-view (in decimal degrees)
- dec : float
Declination (Dec) of center of field-of-view (in decimal degrees)
- sr : float, optional
Search radius (in decimal degrees) from field-of-view center to use for sources from catalog. Default: 0.1 degrees
- epoch : float, optional
Catalog positions returned for this field-of-view will have their proper motions applied to represent their positions at this date, if a value is specified at all, for catalogs with proper motions.
- catalog : str, optional
Name of catalog to query, as defined by web-service. Default: ‘GSC241’
Returns: - csv : CSV object
CSV object of returned sources with all columns as provided by catalog
-
drizzlepac.haputils.astrometric_utils.
find_gsc_offset
(image, input_catalog='GSC1', output_catalog='GAIA')[source]¶ Find the GSC to GAIA offset based on guide star coordinates
Parameters: - image : str
Filename of image to be processed.
Returns: - delta_ra, delta_dec : tuple of floats
Offset in decimal degrees of image based on correction to guide star coordinates relative to GAIA.
-
drizzlepac.haputils.astrometric_utils.
extract_sources
(img, dqmask=None, fwhm=3.0, kernel=None, photmode=None, segment_threshold=None, dao_threshold=None, dao_nsigma=3.0, source_box=7, classify=True, centering_mode='starfind', nlargest=None, outroot=None, plot=False, vmax=None, deblend=False)[source]¶ Use photutils to find sources in image based on segmentation.
Parameters: - img : ndarray
Numpy array of the science extension from the observations FITS file.
- dqmask : ndarray
Bitmask which identifies whether a pixel should be used (1) in source identification or not(0). If provided, this mask will be applied to the input array prior to source identification.
- fwhm : float
Full-width half-maximum (fwhm) of the PSF in pixels.
- threshold : float or None
Value from the image which serves as the limit for determining sources. If None, compute a default value of (background+5*rms(background)). If threshold < 0.0, use absolute value as scaling factor for default value.
- source_box : int
Size of box (in pixels) which defines the minimum size of a valid source.
- classify : bool
Specify whether or not to apply classification based on invarient moments of each source to determine whether or not a source is likely to be a cosmic-ray, and not include those sources in the final catalog.
- centering_mode : str
“segmentaton” or “starfind” Algorithm to use when computing the positions of the detected sources. Centering will only take place after
threshold
has been determined, and sources are identified using segmentation. Centering usingsegmentation
will rely onphotutils.segmentation.source_properties
to generate the properties for the source catalog. Centering usingstarfind
will usephotutils.IRAFStarFinder
to characterize each source in the catalog.- nlargest : int, None
Number of largest (brightest) sources in each chip/array to measure when using ‘starfind’ mode.
- outroot : str, optional
If specified, write out the catalog of sources to the file with this name rootname.
- plot : bool, optional
Specify whether or not to create a plot of the sources on a view of the image.
- vmax : float, optional
If plotting the sources, scale the image to this maximum value.
- deblend : bool, optional
Specify whether or not to apply photutils deblending algorithm when evaluating each of the identified segments (sources) from the chip.
-
drizzlepac.haputils.astrometric_utils.
classify_sources
(catalog, fwhm, sources=None)[source]¶ Convert moments_central attribute for source catalog into star/cr flag.
This algorithm interprets the central_moments from the source_properties generated for the sources as more-likely a star or a cosmic-ray. It is not intended or expected to be precise, merely a means of making a first cut at removing likely cosmic-rays or other artifacts.
Parameters: - catalog :
SourceCatalog
The photutils catalog for the image/chip.
- sources : tuple
Range of objects from catalog to process as a tuple of (min, max). If None (default) all sources are processed.
Returns: - srctype : ndarray
An ndarray where a value of 1 indicates a likely valid, non-cosmic-ray source, and a value of 0 indicates a likely cosmic-ray.
- catalog :
-
drizzlepac.haputils.astrometric_utils.
generate_source_catalog
(image, dqname='DQ', output=False, fwhm=3.0, **detector_pars)[source]¶ Build source catalogs for each chip using photutils.
The catalog returned by this function includes sources found in all chips of the input image with the positions translated to the coordinate frame defined by the reference WCS
refwcs
. The sources will be - identified using photutils segmentation-based source finding code - ignore any input pixel which has been flagged as ‘bad’ in the DQ array, should a DQ array be found in the input HDUList. - classified as probable cosmic-rays (if enabled) using central_moments properties of each source, with these sources being removed from the catalog.Parameters: - image :
HDUList
Input image as an astropy.io.fits HDUList.
- dqname : str
EXTNAME for the DQ array, if present, in the input image HDUList.
- output : bool
Specify whether or not to write out a separate catalog file for all the sources found in each chip.
- fwhm : float
Full-width half-maximum (fwhm) of the PSF in pixels.
Returns: - source_cats : dict
Dict of astropy Tables identified by chip number with each table containing sources from image extension
('sci', chip)
.
- image :
-
drizzlepac.haputils.astrometric_utils.
generate_sky_catalog
(image, refwcs, dqname='DQ', output=False)[source]¶ Build source catalog from input image using photutils.
This script borrows heavily from build_source_catalog.
The catalog returned by this function includes sources found in all chips of the input image with the positions translated to the coordinate frame defined by the reference WCS
refwcs
. The sources will be - identified using photutils segmentation-based source finding code - ignore any input pixel which has been flagged as ‘bad’ in the DQ array, should a DQ array be found in the input HDUList. - classified as probable cosmic-rays (if enabled) using central_moments properties of each source, with these sources being removed from the catalog.Parameters: - image :
HDUList
Input image.
- refwcs :
HSTWCS
Definition of the reference frame WCS.
- dqname : str, optional
EXTNAME for the DQ array, if present, in the input image.
- output : bool, optional
Specify whether or not to write out a separate catalog file for all the sources found in each chip.
Returns: - master_cat :
Table
Source catalog for all ‘valid’ sources identified from all chips of the input image with positions translated to the reference WCS coordinate frame.
- image :
-
drizzlepac.haputils.astrometric_utils.
compute_photometry
(catalog, photvals)[source]¶ Compute magnitudes for sources from catalog based on observations photmode.
Magnitudes will be AB mag values.
Parameters: - catalog :
Table
Astropy Table with ‘source_sum’ column for the measured flux for each source.
- photmode : str
Specification of the observation filter configuration used for the exposure as reported by the ‘PHOTMODE’ keyword from the PRIMARY header.
Returns: - phot_cat :
Table
Astropy Table object of input source catalog with added column for ABMAG photometry (in magnitudes).
- catalog :
-
drizzlepac.haputils.astrometric_utils.
filter_catalog
(catalog, bright_limit=1.0, max_bright=None, min_bright=20, colname='vegamag')[source]¶ Create a new catalog selected from input based on photometry.
Parameters: - catalog :
Table
Table containing the full set of identified sources.
- bright_limit : float
Fraction of catalog based on brightness that should be retained. Value of 1.00 means full catalog.
- max_bright : int
Maximum number of sources to keep regardless of
bright_limit
.- min_bright : int
Minimum number of sources to keep regardless of
bright_limit
.- colname : str
Name of column to use for selection/sorting.
Returns: - new_catalog :
Table
New table which only has the sources that meet the selection criteria.
- catalog :
-
drizzlepac.haputils.astrometric_utils.
build_self_reference
(filename, clean_wcs=False)[source]¶ This function creates a reference, undistorted WCS that can be used to apply a correction to the WCS of the input file.
Parameters: - filename : str
Filename of image which will be corrected, and which will form the basis of the undistorted WCS.
- clean_wcs : bool
Specify whether or not to return the WCS object without any distortion information, or any history of the original input image. This converts the output from
utils.output_wcs()
into a pristineHSTWCS
object.
Returns: - customwcs :
HSTWCS
HSTWCS object which contains the undistorted WCS representing the entire field-of-view for the input image.
Examples
This function can be used with the following syntax to apply a shift/rot/scale change to the same image:
>>> import buildref >>> from drizzlepac import updatehdr >>> filename = "jce501erq_flc.fits" >>> wcslin = buildref.build_self_reference(filename) >>> updatehdr.updatewcs_with_shift(filename, wcslin, xsh=49.5694, ... ysh=19.2203, rot = 359.998, scale = 0.9999964)
-
drizzlepac.haputils.astrometric_utils.
within_footprint
(img, wcsobj, x, y)[source]¶ Determine whether input x, y fall in the science area of the image.
Parameters: - img : ndarray
ndarray of image where non-science areas are marked with value of NaN.
- wcsobj :
HSTWCS
HSTWCS or WCS object with naxis terms defined.
- x, y : ndarray
arrays of x, y positions for sources to be checked.
Returns: - mask : ndarray
Boolean array of same length as x,y arrays where sources that fall within the footprint are True.
-
drizzlepac.haputils.astrometric_utils.
find_hist2d_offset
(filename, reference, refwcs=None, refnames=['ra', 'dec'], match_tolerance=5.0, chip_catalog=True, search_radius=15.0, min_match=10, classify=True)[source]¶ Iteratively look for the best cross-match between the catalog and ref.
Parameters: - filename :
HDUList
or str Single image to extract sources for matching to the external astrometric catalog.
- reference : str or
Table
Reference catalog, either as a filename or
astropy.Table
containing astrometrically accurate sky coordinates for astrometric standard sources.- refwcs :
HSTWCS
This WCS will define the coordinate frame which will be used to determine the offset. If None is specified, use the WCS from the input image
filename
to build this WCS usingbuild_self_reference()
.- refnames : list
List of table column names for sky coordinates of astrometric standard sources from reference catalog.
- match_tolerance : float
Tolerance (in pixels) for recognizing that a source position matches an astrometric catalog position. Larger values allow for lower accuracy source positions to be compared to astrometric catalog
- chip_catalog : bool
Specify whether or not to write out individual source catalog for each chip in the image.
- search_radius : float
Maximum separation (in arcseconds) from source positions to look for valid cross-matches with reference source positions.
- min_match : int
Minimum number of cross-matches for an acceptable determination of the offset.
- classify : bool
Specify whether or not to use central_moments classification to ignore likely cosmic-rays/bad-pixels when generating the source catalog.
Returns: - best_offset : tuple
Offset in input image pixels between image source positions and astrometric catalog positions that results in largest number of matches of astrometric sources with image sources
- seg_xy, ref_xy :
Table
Source catalog and reference catalog, respectively, used for determining the offset. Each catalog includes sources for the entire field-of-view, not just a single chip.
- seg_xy, ref_xy :
- filename :
-
drizzlepac.haputils.astrometric_utils.
build_wcscat
(image, group_id, source_catalog)[source]¶ Return a list of
FITSWCS
objects for all chips in an image.Parameters: - image : str,
HDUList
Either filename or HDUList of a single HST observation.
- group_id : int
Integer ID for group this image should be associated with; primarily used when separate chips are in separate files to treat them all as one exposure.
- source_catalog : dict
If provided, these catalogs will be attached as
catalog
entries in each chip’sFITSWCS
object. It should be provided as a dict of astropy Tables identified by chip number with each table containing sources from image extension('sci', chip)
as generated bygenerate_source_catalog()
.
Returns: - wcs_catalogs : list of
FITSWCS
List of
FITSWCS
objects defined for all chips in input image.
- image : str,
-
drizzlepac.haputils.astrometric_utils.
compute_similarity
(image, reference)[source]¶ Compute a similarity index for an image compared to a reference image.
Similarity index is based on a the general algorithm used in the AmphiIndex algorithm.
- identify slice of image that is a factor of 256 in size
- rebin image slice down to a (256,256) image
- rebin same slice from reference down to a (256,256) image
- sum the differences of the rebinned slices
- divide absolute value of difference scaled by reference slice sum
This index will typically return values < 0.1 for similar images, and values > 1 for dis-similar images.
Parameters: - image : ndarray
Image (as ndarray) to measure
- reference : ndarray
Image which serves as the ‘truth’ or comparison image.
Returns: - similarity_index : float
Value of similarity index for
image
-
drizzlepac.haputils.astrometric_utils.
determine_focus_index
(img, sigma=1.5)[source]¶ Determine blurriness indicator for an image
This returns a single value that serves as an indication of the sharpness of the image based on the max pixel value from the image after applying a Laplacian-of-Gaussian filter with sigma.
Implementation based on discussion from: https://stackoverflow.com/questions/7765810/is-there-a-way-to-detect-if-an-image-is-blurry
as supported by: Pertuz, S., et.al., 2013, Pattern Recognition, 46:1415–1432
This index needs to be based on ‘max’ value in order to avoid field-dependent biases, since the ‘max’ value will correspond to point-source-like sources regardless of the field (nebula, galaxy, …). Care must be taken, though, to ignore cosmic-rays as much as possible as they will mimic real sources without providing information on the actual focus through the optics. Similarly, saturation regions must also be ignored as they also only indicate a detector feature, not the focus through the optics or alignment of actual sources.
-
drizzlepac.haputils.astrometric_utils.
max_overlap_diff
(total_mask, singlefiles, prodfile, sigma=2.0, scale=1, lsigma=3.0)[source]¶ Determines the difference in the region of max overlap for all drizzled products
Parameters: - total_mask : ndarray
Mask (array) showing where each input exposure contributes to the final drizzle product
prodfile
. This could be created usingcell_utils.SkyFootprint
.- singlefiles : list
List of filenames for each single input exposure drizzled onto the same WCS as the final drizzle product
prodfile
- prodfile : str
Filename for the final drizzle product
- scale : int, optional
Factor to use in downsizing (resizing smaller) the images to be evaluated. The larger the value, the less sensitive this measurement becomes.
- sigma : float, optional
Size of default kernel (in pixels) to use for determining the focus.
Returns: - diff_dict : dictionary
Dictionary of difference scores for each input exposure drizzle product (from
singlefiles
) calculated for the region of maximum overlap with the final drizzle productprodfile
. Entries for each singlefile includes:- distance : Hamming distance of singlefile from prodfile
- focus : focus index of singlefile
- focus_pos : position for best focus in singlefile
- product_focus : focus index for prodfile
- product_focus_pos : position for best focus in prodfile
haputils.analyze¶
Utility to analyze an input dataset and determine whether the dataset can be aligned
The function analyze_data opens an input list containing FLT and/or FLC FITS filenames in order to access the primary header data. Based upon the values of specific FITS keywords, the function determines whether or not each file within this dataset can or should be reconciled against an astrometric catalog and, for multiple images, used to create a mosaic.
-
drizzlepac.haputils.analyze.
analyze_data
(input_file_list, log_level=0)[source]¶ Determine if images within the dataset can be aligned
Parameters: - input_file_list : list
List containing FLT and/or FLC filenames for all input images which comprise an associated dataset where ‘associated dataset’ may be a single image, multiple images, an HST association, or a number of HST associations
- log_level : int, optional
The desired level of verboseness in the log statements displayed on the screen and written to the .log file. Default value is 20, or ‘info’.
Returns: - output_table : object
Astropy Table object containing data pertaining to the associated dataset, including the do_process bool. It is intended this table is updated by subsequent functions for bookkeeping purposes.
Notes
The keyword/value pairs below define the “cannot process categories”. OBSTYPE : is not IMAGING MTFLAG : T SCAN-TYP : C or D (or !N) FILTER : G*, PR*, where G=Grism and PR=Prism FILTER1 : G*, PR*, where G=Grism and PR=Prism FILTER2 : G*, PR*, where G=Grism and PR=Prism TARGNAME : DARK, TUNGSTEN, BIAS, FLAT, EARTH-CALIB, DEUTERIUM EXPTIME : 0 CHINJECT : is not NONE
The keyword/value pairs below define the category which the data can be processed, but the results may be compromised FGSLOCK : FINE/GYRO, FINE/GY, COARSE, GYROS
FITS Keywords only for WFC3 data: SCAN_TYP, FILTER, and CHINJECT (UVIS) FITS Keywords only for ACS data: FILTER1 and FILTER2
Please be aware of the FITS keyword value NONE vs the Python None.
haputils.align_utils¶
-
class
drizzlepac.haputils.align_utils.
AlignmentTable
(input_list, clobber=False, dqname='DQ', log_level=0, **alignment_pars)[source]¶ - **alignment_pars needs to contain the following entries:
- # kernel defining, source finding par fwhmpsf=0.12, # background computing pars box_size=BKG_BOX_SIZE, win_size=BKG_FILTER_SIZE, bkg_estimator=SExtractorBackground, rms_estimator=StdBackgroundRMS, nsigma=5., threshold_flag=None, # object finding pars source_box=7, classify=True, centering_mode=”starfind”, nlargest=None, plot=False, vmax=None, deblend=False
-
drizzlepac.haputils.align_utils.
match_relative_fit
(imglist, reference_catalog, **fit_pars)[source]¶ Perform cross-matching and final fit using relative matching algorithm
Parameters: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs- reference_catalog : Table
Astropy Table of reference sources for this field
Returns: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs
-
drizzlepac.haputils.align_utils.
match_default_fit
(imglist, reference_catalog, **fit_pars)[source]¶ Perform cross-matching and final fit using default tolerance matching
Parameters: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs- reference_catalog : Table
Astropy Table of reference sources for this field
Returns: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs
-
drizzlepac.haputils.align_utils.
match_2dhist_fit
(imglist, reference_catalog, **fit_pars)[source]¶ Perform cross-matching and final fit using 2dHistogram matching
Parameters: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs- reference_catalog : Table
Astropy Table of reference sources for this field
Returns: - imglist : list
List of input image
FITSWCS
objects with metadata and source catalogs