Hubble Advanced Products API

These modules provide the basic functionality used to process automatically data using this package to apply the distortion models to the WCS of HST observations and to verify the alignment of the observations.

haputils.astrometric_utils

Utilities to support creation of astrometrically accurate reference catalogs

The function, create_astrometric_catalog, allows the user to query an astrometric catalog online to generate a catalog of astrometric sources that should fall within the field-of-view of all the input images.

This module relies on the definition of an environment variable to specify the URL of the astrometric catalog to use for generating this reference catalog.

ASTROMETRIC_CATALOG_URL  -- URL of web service that can be queried to
                            obtain listing of astrometric sources,
                            sky coordinates, and magnitudes.

drizzlepac.haputils.astrometric_utils.create_astrometric_catalog(inputs, catalog='GAIADR2', output='ref_cat.ecsv', gaia_only=False, table_format='ascii.ecsv', existing_wcs=None, num_sources=None, use_footprint=False, full_catalog=False)

Create an astrometric catalog that covers the inputs’ field-of-view.

Parameters:

input : str, list

Filenames of images to be aligned to astrometric catalog

catalog : str, optional

Name of catalog to extract astrometric positions for sources in the input images’ field-of-view. Default: GAIADR2. Options available are documented on the catalog web page.

output : str, optional

Filename to give to the astrometric catalog read in from the master catalog web service. If None, no file will be written out.

gaia_only : bool, optional

Specify whether or not to only use sources from GAIA in output catalog

existing_wcs : HSTWCS

existing WCS object specified by the user

num_sources : int

Maximum number of brightest/faintest sources to return in catalog. If num_sources is negative, return that number of the faintest sources. By default, all sources are returned.

use_footprint : bool, optional

Use the image footprint as the source identification bounds insted of using a circle centered on a given RA and Dec? By default, use_footprint = False.

full_catalog : bool, optional

Return the full set of columns provided by the web service.

Returns:

ref_table : Table

Astropy Table object of the catalog

Notes

This function will point to astrometric catalog web service defined through the use of the ASTROMETRIC_CATALOG_URL environment variable.

drizzlepac.haputils.astrometric_utils.get_catalog(ra, dec, sr=0.1, epoch=None, catalog='GSC241')

Extract catalog from VO web service.

Parameters:

ra : float

Right Ascension (RA) of center of field-of-view (in decimal degrees)

dec : float

Declination (Dec) of center of field-of-view (in decimal degrees)

sr : float, optional

Search radius (in decimal degrees) from field-of-view center to use for sources from catalog. Default: 0.1 degrees

epoch : float, optional

Catalog positions returned for this field-of-view will have their proper motions applied to represent their positions at this date, if a value is specified at all, for catalogs with proper motions.

catalog : str, optional

Name of catalog to query, as defined by web-service. Default: ‘GSC241’

Returns:

csv : CSV object

CSV object of returned sources with all columns as provided by catalog

drizzlepac.haputils.astrometric_utils.find_gsc_offset(image, input_catalog='GSC1', output_catalog='GAIA')

Find the GSC to GAIA offset based on guide star coordinates

Parameters:

image : str

Filename of image to be processed.

Returns:

delta_ra, delta_dec : tuple of floats

Offset in decimal degrees of image based on correction to guide star coordinates relative to GAIA.

drizzlepac.haputils.astrometric_utils.extract_sources(img, dqmask=None, fwhm=3.0, kernel=None, photmode=None, segment_threshold=None, dao_threshold=None, dao_nsigma=3.0, source_box=7, classify=True, centering_mode='starfind', nlargest=None, outroot=None, plot=False, vmax=None, deblend=False)

Use photutils to find sources in image based on segmentation.

Parameters:

img : ndarray

Numpy array of the science extension from the observations FITS file.

dqmask : ndarray

Bitmask which identifies whether a pixel should be used (1) in source identification or not(0). If provided, this mask will be applied to the input array prior to source identification.

fwhm : float

Full-width half-maximum (fwhm) of the PSF in pixels.

threshold : float or None

Value from the image which serves as the limit for determining sources. If None, compute a default value of (background+5*rms(background)). If threshold < 0.0, use absolute value as scaling factor for default value.

source_box : int

Size of box (in pixels) which defines the minimum size of a valid source.

classify : bool

Specify whether or not to apply classification based on invarient moments of each source to determine whether or not a source is likely to be a cosmic-ray, and not include those sources in the final catalog.

centering_mode : str

“segmentaton” or “starfind” Algorithm to use when computing the positions of the detected sources. Centering will only take place after threshold has been determined, and sources are identified using segmentation. Centering using segmentation will rely on photutils.segmentation.source_properties to generate the properties for the source catalog. Centering using starfind will use photutils.IRAFStarFinder to characterize each source in the catalog.

nlargest : int, None

Number of largest (brightest) sources in each chip/array to measure when using ‘starfind’ mode.

outroot : str, optional

If specified, write out the catalog of sources to the file with this name rootname.

plot : bool, optional

Specify whether or not to create a plot of the sources on a view of the image.

vmax : float, optional

If plotting the sources, scale the image to this maximum value.

deblend : bool, optional

Specify whether or not to apply photutils deblending algorithm when evaluating each of the identified segments (sources) from the chip.

drizzlepac.haputils.astrometric_utils.classify_sources(catalog, fwhm, sources=None)

Convert moments_central attribute for source catalog into star/cr flag.

This algorithm interprets the central_moments from the source_properties generated for the sources as more-likely a star or a cosmic-ray. It is not intended or expected to be precise, merely a means of making a first cut at removing likely cosmic-rays or other artifacts.

Parameters:

catalog : SourceCatalog

The photutils catalog for the image/chip.

sources : tuple

Range of objects from catalog to process as a tuple of (min, max). If None (default) all sources are processed.

Returns:

srctype : ndarray

An ndarray where a value of 1 indicates a likely valid, non-cosmic-ray source, and a value of 0 indicates a likely cosmic-ray.

drizzlepac.haputils.astrometric_utils.generate_source_catalog(image, dqname='DQ', output=False, fwhm=3.0, **detector_pars)

Build source catalogs for each chip using photutils.

The catalog returned by this function includes sources found in all chips of the input image with the positions translated to the coordinate frame defined by the reference WCS refwcs. The sources will be - identified using photutils segmentation-based source finding code - ignore any input pixel which has been flagged as ‘bad’ in the DQ array, should a DQ array be found in the input HDUList. - classified as probable cosmic-rays (if enabled) using central_moments properties of each source, with these sources being removed from the catalog.

Parameters:

image : HDUList

Input image as an astropy.io.fits HDUList.

dqname : str

EXTNAME for the DQ array, if present, in the input image HDUList.

output : bool

Specify whether or not to write out a separate catalog file for all the sources found in each chip.

fwhm : float

Full-width half-maximum (fwhm) of the PSF in pixels.

Returns:

source_cats : dict

Dict of astropy Tables identified by chip number with each table containing sources from image extension ('sci', chip).

drizzlepac.haputils.astrometric_utils.generate_sky_catalog(image, refwcs, dqname='DQ', output=False)

Build source catalog from input image using photutils.

This script borrows heavily from build_source_catalog.

The catalog returned by this function includes sources found in all chips of the input image with the positions translated to the coordinate frame defined by the reference WCS refwcs. The sources will be - identified using photutils segmentation-based source finding code - ignore any input pixel which has been flagged as ‘bad’ in the DQ array, should a DQ array be found in the input HDUList. - classified as probable cosmic-rays (if enabled) using central_moments properties of each source, with these sources being removed from the catalog.

Parameters:

image : HDUList

Input image.

refwcs : HSTWCS

Definition of the reference frame WCS.

dqname : str, optional

EXTNAME for the DQ array, if present, in the input image.

output : bool, optional

Specify whether or not to write out a separate catalog file for all the sources found in each chip.

Returns:

master_cat : Table

Source catalog for all ‘valid’ sources identified from all chips of the input image with positions translated to the reference WCS coordinate frame.

drizzlepac.haputils.astrometric_utils.compute_photometry(catalog, photvals)

Compute magnitudes for sources from catalog based on observations photmode.

Magnitudes will be AB mag values.

Parameters:

catalog : Table

Astropy Table with ‘source_sum’ column for the measured flux for each source.

photmode : str

Specification of the observation filter configuration used for the exposure as reported by the ‘PHOTMODE’ keyword from the PRIMARY header.

Returns:

phot_cat : Table

Astropy Table object of input source catalog with added column for ABMAG photometry (in magnitudes).

drizzlepac.haputils.astrometric_utils.filter_catalog(catalog, bright_limit=1.0, max_bright=None, min_bright=20, colname='vegamag')

Create a new catalog selected from input based on photometry.

Parameters:

catalog : Table

Table containing the full set of identified sources.

bright_limit : float

Fraction of catalog based on brightness that should be retained. Value of 1.00 means full catalog.

max_bright : int

Maximum number of sources to keep regardless of bright_limit.

min_bright : int

Minimum number of sources to keep regardless of bright_limit.

colname : str

Name of column to use for selection/sorting.

Returns:

new_catalog : Table

New table which only has the sources that meet the selection criteria.

drizzlepac.haputils.astrometric_utils.build_self_reference(filename, clean_wcs=False)

This function creates a reference, undistorted WCS that can be used to apply a correction to the WCS of the input file.

Parameters:

filename : str

Filename of image which will be corrected, and which will form the basis of the undistorted WCS.

clean_wcs : bool

Specify whether or not to return the WCS object without any distortion information, or any history of the original input image. This converts the output from utils.output_wcs() into a pristine HSTWCS object.

Returns:

customwcs : HSTWCS

HSTWCS object which contains the undistorted WCS representing the entire field-of-view for the input image.

Examples

This function can be used with the following syntax to apply a shift/rot/scale change to the same image:

>>> import buildref
>>> from drizzlepac import updatehdr
>>> filename = "jce501erq_flc.fits"
>>> wcslin = buildref.build_self_reference(filename)
>>> updatehdr.updatewcs_with_shift(filename, wcslin, xsh=49.5694,
... ysh=19.2203, rot = 359.998, scale = 0.9999964)

drizzlepac.haputils.astrometric_utils.within_footprint(img, wcsobj, x, y)

Determine whether input x, y fall in the science area of the image.

Parameters:

img : ndarray

ndarray of image where non-science areas are marked with value of NaN.

wcsobj : HSTWCS

HSTWCS or WCS object with naxis terms defined.

x, y : ndarray

arrays of x, y positions for sources to be checked.

Returns:

mask : ndarray

Boolean array of same length as x,y arrays where sources that fall within the footprint are True.

drizzlepac.haputils.astrometric_utils.find_hist2d_offset(filename, reference, refwcs=None, refnames=['ra', 'dec'], match_tolerance=5.0, chip_catalog=True, search_radius=15.0, min_match=10, classify=True)

Iteratively look for the best cross-match between the catalog and ref.

Parameters:

filename : HDUList or str

Single image to extract sources for matching to the external astrometric catalog.

reference : str or Table

Reference catalog, either as a filename or astropy.Table containing astrometrically accurate sky coordinates for astrometric standard sources.

refwcs : HSTWCS

This WCS will define the coordinate frame which will be used to determine the offset. If None is specified, use the WCS from the input image filename to build this WCS using build_self_reference().

refnames : list

List of table column names for sky coordinates of astrometric standard sources from reference catalog.

match_tolerance : float

Tolerance (in pixels) for recognizing that a source position matches an astrometric catalog position. Larger values allow for lower accuracy source positions to be compared to astrometric catalog

chip_catalog : bool

Specify whether or not to write out individual source catalog for each chip in the image.

search_radius : float

Maximum separation (in arcseconds) from source positions to look for valid cross-matches with reference source positions.

min_match : int

Minimum number of cross-matches for an acceptable determination of the offset.

classify : bool

Specify whether or not to use central_moments classification to ignore likely cosmic-rays/bad-pixels when generating the source catalog.

Returns:

best_offset : tuple

Offset in input image pixels between image source positions and astrometric catalog positions that results in largest number of matches of astrometric sources with image sources

seg_xy, ref_xy : Table

Source catalog and reference catalog, respectively, used for determining the offset. Each catalog includes sources for the entire field-of-view, not just a single chip.

drizzlepac.haputils.astrometric_utils.build_wcscat(image, group_id, source_catalog)

Return a list of FITSWCS objects for all chips in an image.

Parameters:

image : str, HDUList

Either filename or HDUList of a single HST observation.

group_id : int

Integer ID for group this image should be associated with; primarily used when separate chips are in separate files to treat them all as one exposure.

source_catalog : dict

If provided, these catalogs will be attached as catalog entries in each chip’s FITSWCS object. It should be provided as a dict of astropy Tables identified by chip number with each table containing sources from image extension ('sci', chip) as generated by generate_source_catalog().

Returns:

wcs_catalogs : list of FITSWCS

List of FITSWCS objects defined for all chips in input image.

drizzlepac.haputils.astrometric_utils.compute_similarity(image, reference)

Compute a similarity index for an image compared to a reference image.

Similarity index is based on a the general algorithm used in the AmphiIndex algorithm.

• identify slice of image that is a factor of 256 in size
• rebin image slice down to a (256,256) image
• rebin same slice from reference down to a (256,256) image
• sum the differences of the rebinned slices
• divide absolute value of difference scaled by reference slice sum

This index will typically return values < 0.1 for similar images, and values > 1 for dis-similar images.

Parameters:

image : ndarray

Image (as ndarray) to measure

reference : ndarray

Image which serves as the ‘truth’ or comparison image.

Returns:

similarity_index : float

Value of similarity index for image

drizzlepac.haputils.astrometric_utils.determine_focus_index(img, sigma=1.5)

Determine blurriness indicator for an image

This returns a single value that serves as an indication of the sharpness of the image based on the max pixel value from the image after applying a Laplacian-of-Gaussian filter with sigma.

Implementation based on discussion from: https://stackoverflow.com/questions/7765810/is-there-a-way-to-detect-if-an-image-is-blurry

as supported by: Pertuz, S., et.al., 2013, Pattern Recognition, 46:1415–1432

This index needs to be based on ‘max’ value in order to avoid field-dependent biases, since the ‘max’ value will correspond to point-source-like sources regardless of the field (nebula, galaxy, …). Care must be taken, though, to ignore cosmic-rays as much as possible as they will mimic real sources without providing information on the actual focus through the optics. Similarly, saturation regions must also be ignored as they also only indicate a detector feature, not the focus through the optics or alignment of actual sources.

drizzlepac.haputils.astrometric_utils.max_overlap_diff(total_mask, singlefiles, prodfile, sigma=2.0, scale=1, lsigma=3.0)

Determines the difference in the region of max overlap for all drizzled products

Parameters:

total_mask : ndarray

Mask (array) showing where each input exposure contributes to the final drizzle product prodfile. This could be created using cell_utils.SkyFootprint.

singlefiles : list

List of filenames for each single input exposure drizzled onto the same WCS as the final drizzle product prodfile

prodfile : str

Filename for the final drizzle product

scale : int, optional

Factor to use in downsizing (resizing smaller) the images to be evaluated. The larger the value, the less sensitive this measurement becomes.

sigma : float, optional

Size of default kernel (in pixels) to use for determining the focus.

Returns:

diff_dict : dictionary

Dictionary of difference scores for each input exposure drizzle product (from singlefiles) calculated for the region of maximum overlap with the final drizzle product prodfile. Entries for each singlefile includes:

• distance : Hamming distance of singlefile from prodfile
• focus : focus index of singlefile
• focus_pos : position for best focus in singlefile
• product_focus : focus index for prodfile
• product_focus_pos : position for best focus in prodfile

drizzlepac.haputils.astrometric_utils.detect_point_sources(arr, background=None, nsigma=4, log_sigma=3.0, scale=1, sigma=3.0, exp_weight=None)

haputils.analyze

Utility to analyze an input dataset and determine whether the dataset can be aligned

The function analyze_data opens an input list containing FLT and/or FLC FITS filenames in order to access the primary header data. Based upon the values of specific FITS keywords, the function determines whether or not each file within this dataset can or should be reconciled against an astrometric catalog and, for multiple images, used to create a mosaic.

drizzlepac.haputils.analyze.analyze_data(input_file_list, log_level=0)

Determine if images within the dataset can be aligned

Parameters:

input_file_list : list

List containing FLT and/or FLC filenames for all input images which comprise an associated dataset where ‘associated dataset’ may be a single image, multiple images, an HST association, or a number of HST associations

log_level : int, optional

The desired level of verboseness in the log statements displayed on the screen and written to the .log file. Default value is 20, or ‘info’.

Returns:

output_table : object

Astropy Table object containing data pertaining to the associated dataset, including the do_process bool. It is intended this table is updated by subsequent functions for bookkeeping purposes.

Notes

The keyword/value pairs below define the “cannot process categories”. OBSTYPE : is not IMAGING MTFLAG : T SCAN-TYP : C or D (or !N) FILTER : G*, PR*, where G=Grism and PR=Prism FILTER1 : G*, PR*, where G=Grism and PR=Prism FILTER2 : G*, PR*, where G=Grism and PR=Prism TARGNAME : DARK, TUNGSTEN, BIAS, FLAT, EARTH-CALIB, DEUTERIUM EXPTIME : 0 CHINJECT : is not NONE

The keyword/value pairs below define the category which the data can be processed, but the results may be compromised FGSLOCK : FINE/GYRO, FINE/GY, COARSE, GYROS

FITS Keywords only for WFC3 data: SCAN_TYP, FILTER, and CHINJECT (UVIS) FITS Keywords only for ACS data: FILTER1 and FILTER2

Please be aware of the FITS keyword value NONE vs the Python None.

haputils.align_utils

class drizzlepac.haputils.align_utils.AlignmentTable(input_list, clobber=False, dqname='DQ', log_level=0, **alignment_pars)

**alignment_pars needs to contain the following entries:

# kernel defining, source finding par fwhmpsf=0.12, # background computing pars box_size=BKG_BOX_SIZE, win_size=BKG_FILTER_SIZE, bkg_estimator=SExtractorBackground, rms_estimator=StdBackgroundRMS, nsigma=5., threshold_flag=None, # object finding pars source_box=7, classify=True, centering_mode=”starfind”, nlargest=None, plot=False, vmax=None, deblend=False

drizzlepac.haputils.align_utils.match_relative_fit(imglist, reference_catalog, **fit_pars)

Perform cross-matching and final fit using relative matching algorithm

Parameters:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs

reference_catalog : Table

Astropy Table of reference sources for this field

Returns:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs

drizzlepac.haputils.align_utils.match_default_fit(imglist, reference_catalog, **fit_pars)

Perform cross-matching and final fit using default tolerance matching

Parameters:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs

reference_catalog : Table

Astropy Table of reference sources for this field

Returns:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs

drizzlepac.haputils.align_utils.match_2dhist_fit(imglist, reference_catalog, **fit_pars)

Perform cross-matching and final fit using 2dHistogram matching

Parameters:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs

reference_catalog : Table

Astropy Table of reference sources for this field

Returns:

imglist : list

List of input image FITSWCS objects with metadata and source catalogs