DrizzlePac Release Notes¶
The version of DrizzlePac can be identified using
> python >>> import drizzlepac >>> drizzlepac.__version__
The following notes provide some details on what has been revised for each version in reverse chronological order (most recent version at the top of the list).
- Update calibrated (FLC/FLT) files with RMS and NMATCH keywords when it successfully aligns the data to GAIA using the a posteriori fit. Headerlet files for this fit which already have these keywords are now retained and provided as the final output headerlets as well. [#554]
- Fixed a bug in the
updatehdr.update_from_shiftfile()function that would crash while reading shift files. [#448]
- Migration of the HAP portion of the package to an object-oriented implemenation. [#427]
- Added support for providing HSTWCS object as input to ‘final_refimage’ or ‘single_refimage’ parameter. [#426]
- Implementation of grid definition interface to support returning SkyCell objects that overlap a mosaic footprint. [#425]
- Complete rewrite of
runastrodrizfor pipeline processing to include multi-level verification of alignment. [#440]
- Removed deprecated parameter
coordsfrom the parameter list of
- Modified the behavior of the
pixtopix.tran()to not print coordinates when not run as a script and when
- Fixed a compatibility issue in
tweakutilsthat would result in crash in
skytopixwhen converting coordinates in
- Fixed a bug in the
astrodrizzle.skymodule due to which sky matching fails with “Keyword ‘MDRIZSKY’ not found” error when some of the input images do not overlap at all with the other images. [#380]
- Fixed a bug in the
util.WithLoggingdecorator due to which incorrect log file was reported when user-supplied log file name does not have
- Fixed a bug introduced in #364 returning in
util.WithLoggingdecorator to handle functions that return values. [#364]
- Fixed a bug in the automatic computation of the IVM weights when IVM was not provided by the user. [#320]
- Fixed a bug in the 2D histogram code used for estimating shifts for catalog pre-matching. This may result in better matching. [#286]
tweakreg) is no longer ignored when
use2dhistis enabled. [#286]
- Fixed VS compiler errors with pointer artithmetic on void pointers. [#273]
- Fix logic so that code no longer tries to update headers when no valid fit could be determined. [#241]
- Fixed a bug in the computation of interpolated large scale flat field
for STIS data. The bug was inconsequential in practice.
Removed the dependency on
- Removed the dependency on
scipyroutines instead). [#225]
'Advanced Pipeline Products'alignment code to
runastrodrizto compute and apply absolute astrometric corrections to GAIA (or related) frame to images where possible. [#200, #213, #216, #223, #234, #235, #244, #248, #249, #250, #251, #259, #260, #268, #271, #283, #294, #302]
- Add computation and reporting of the fit’s Root-Mean-Square Error (RMSE) and Mean Absolute Error (MAE). [#210]
- Replaced the use of
- Removed support for Python 2. Only versions >= 3.5 are supported. [#207]
- Use a more numerically stable
numpy.linalg.invinstead of own matrix inversion. [#205]
- The intermediate fit match catalog, with the name
tweakregnow has correct RA and DEC values for the sources after applying the fit. [#200, #202]
- Simplify logic for determining the chip ID for each source. [#200]
- Fix a bug that results in
tweakregcrashing when no sources are found with user-specified source-finding parameters and when
tweakregthen attempts to find sources using default parameters. [#181]
- Updated unit_tests to use original inputs, rather than updated inputs used by nightly regression tests.
numpy“floating” deprecation warnings. [#175]
- Fix incorrect units in CR-cleaned images created by
astrodrizzle. Now CR-cleaned images should have the same units as input images. [#190]
- Changed the color scheme of the
- Refactored test suite
sdistnow packages C extension source code
- Updated links in the documentation to point to latest
drizzlepacwebsite and online API documentation.
- Code cleanup.
- Updated C code to be more compatible with latest numpy releases in order to reduce numerous compile warnings.
- Updated documentation to eliminate (at this moment) all sphinx documentation generation warnings.
'CHANGELOG.rst'in the top-level directory.
- Improved setup to allow documentation build. See drizzlepac PR #142 and Issue #129 for more details.
- Fixed a bug in a print statement in the create median step due to which background values for input images used in this step were not printed.
- Fixed a bug due to which
TweakRegmay have effectively ignored
- Fixed a bug in
drizzlepac.util.WithLoggingdue to which
astrodrizzlewould throw an error trying when to raise another error. See Issue #157 for more details.
- Fixed a bug in
v2.2.0due to which, when
TweakRegis run from the interpreter, the code may crash when trying to interpret input files.
- Fixed problems with processing WFPC2 data provided by the archive. User will
need to make sure they run
updatewcson all input WFPC2 data before combining them with
- Implemented a major refactor of the project directory structure. Building no
stsci.distutils. Drizzlepac’s release information (i.e. version, build date, etc) is now handled by
relic. See https://github.com/spacetelescope/relic
- Added basic support for compiling Drizzlepac’s C extensions under Windows.
- Documentation is now generated during the build process. This ensures the
end-user always has access to documentation that applies to the version of
- Swapped the effect of setting
TweakReg. When calling one of these tasks with
configobjparameter set to
None, values for the not-explicitly-specified parameters should be set to the default values for the task. When
configobjis set to
'defaults'not-explicitly-specified parameters will be loaded from the
~/.teal/tweakreg.cfgfiles that store latest used settings (or from matching configuration files in the current directory). See https://github.com/spacetelescope/drizzlepac/pull/115 for more details.
- Changed the definition of Megabyte used to describe the size of the buffer
for create median step (
combine_bufsize). Previously a mixed (base-2 and base-10) definition was used with 1MB = 1000x1024B = 1024000B. Now 1MB is defined in base-2 (MiB) as 1MB = 1024x1024B = 1048576B.
- Redesigned the logic in
createMedianstep used to split large
single_sciimages into smaller chunks: new logic is more straightforward and fixes errors in the old algorithm that resulted in crashes or unnecessarily small chunk sizes that slowed down
- Due to the above mentioned redesign in the logic for splitting large images
into smaller chunks, now
overlapcan be set to 0 if so desired in the
minmedcombine type. Also, it is automatically ignored (set to 0) for all non-
minmedcombine types. This will result in additional speed-up in the Create Median step.
TweakReg()now can be called with
configobjparameter set to
'defaults'in order to indicate that values for the not-explicitly-specified parameters should be set to the default values for the task instead of being loaded from the
~/.teal/tweakreg.cfgfiles that store latest used settings.
- Updated documentation.
- Restore recording of correct
EXPTIMEvalue in the headers of single drizzled (“single_sci”) images. See https://github.com/spacetelescope/drizzlepac/issues/93 for more details.
- Fixed a bug in
drizzlepacdue to which user provided
CREATE MEDIAN IMAGEstep were not converted correctly to electrons (processing unit). This bug affected processing of WFPC2, STIS, NICMOS, and WFC3 data. See https://github.com/spacetelescope/drizzlepac/issues/94 for more details.
- Modified print format so that scales, skew and rotations are printed with 10 significant digits while shifts are printed with 4 digits after the decimal point.
- Fixed a bug in expanding reference catalog in
TweakRegthat would result in the code crashing. See https://github.com/spacetelescope/drizzlepac/pull/87 for more details.
- Fixed a bug due to which user catalog fluxes would be interpreted as
fluxunitswas set to
'cps'. See https://github.com/spacetelescope/drizzlepac/pull/88 for more details.
- Fixed a bug due to which user-supplied flux limits were ignored for the reference catalog. See https://github.com/spacetelescope/drizzlepac/pull/89 for more details.
- Fixed a bug in computing optimal order of expanding reference catalog that resulted in code crashes. See https://github.com/spacetelescope/drizzlepac/pull/86 for more details.
astrodrizzlelowers the case of the path of output images issue. See https://github.com/spacetelescope/drizzlepac/issues/79 for more details.
tweakregignores user-specified units of image catalogs (provided through the
refcatparameter) issue. See https://github.com/spacetelescope/drizzlepac/issues/81 for more details.
- Corrected a message printed by tweakreg about used WCS for alignment. Also
improved documentation for the
drizzlepac.adrizzleupdated to work with numpy >=1.12 when they implemented more strict array conversion rules for math. Any input which still has INT format will be converted to a float before any operations are performed, explicitly implementing what was an automatic operation prior to numpy 1.12.
- Fixed a bug introduced in release v2.1.15 in the logic for merging WCS due to which custom WCS scale was being ignored.
fits.iooperations will no longer use memory mapping in order to reduce the number of file handles used when running either
tweakreg. See issue #39 for more details.
- Fixed bugs and improved the logic for merging WCS that is used to define
astrodrizzle’s output WCS.
crpix2parameters to custom WCS.
- Fixed a bug due to which sky background was subtracted by
astrodrizzlefrom the images even though
skysubwas set to
MDRIZSKYwas already present in input images’ headers.
astrodrizzlenow will run
updatewcs()on newly created images when necessary, e.g., after converting WAVERED FITS to MEF format (
*_c0h.fits) or after unpacking multi-imset STIS
_fltfiles. See PR #56 for more details.
- Fixed a bug that was preventing processing STIS image data.
- Fixed a bug in reading user input (see issue #51).
Bug fix release (a bug was introduced in v2.1.10).
Some of the changes introduced in release v2.1.9 were not backward compatible. This release makes those changes backward compatible.
Compatibility improvements with Python 3 and other STScI software packages.
- Drizzlepac code will no longer attempt to delete “original” (WCS key ‘O’) resulting in a decreased number of warnings (see issue #35 ).
- Negative values are now zeroed in the ‘minmed’ step before attempting to estimate Poisson errors (see issue #22).
- Fixed a bug in
tweakregdue to incorrect matrix inversion.
- Improved compatibility with
astropy.io.fits(‘clobber’ parameter) and
numpywhich has reduced the number of deprecation warnings).
- Existing static masks in the working directory are now overwritten and not simply re-used (see issue #23).
- Corrected formula for \(\sigma\) computation in the “create median” step to convert background to electrons before computations. This bug was producing incorrect \(\sigma\) for instruments whose gain was different from one.
combine_typeparameter which now also documents the formula for \(\sigma\) computation when
combine_typeparameter is set to
2.1.6 and 2.1.7rc (15-Aug-2016)¶
Package maintenance release.
Technical re-release of
The following bug fixes have been implemented:
tweakregcrashes when run with a single input image and a reference catalog.
- Fixes an issue due to which
tweakreg, when updating image headers, would not add ‘-SIP’ suffix to CTYPE
- Improved ASN input file handling.
astrodrizzledoes not delete
d2imfileanylonger allowing multiple runs of
updatewcson the same WFPC2 image, see Ticket 1244 for more details.
- Allow exclusion regions in
tweakregto be in a different directory and allow relative path in exclusion region file name.
- Improved handling of empty input image lists.
tweakregbug fix: use absolute value of polygon area.
- Improved logic for duplicate input detection.
- Improved logic for handling custom WCS parameters in
- Compatibility improvements with Python 3.
Available under SSBX/IRAFX starting: Nov 17, 2015
This release includes the following bug fixes:
- Resolved order of operation problems when processing WFPC2 data with DGEOFILEs.
- The conversion of the WFPC2
D2IMFILEis now incorporated into
STWCSv1.2.3 (r47112, r47113, r47114) rather than a part of
astrodrizzle. This requires users to run updatewcs first, then
tweakregwill work with that WFPC2 data seamlessly (as if they were ACS or WFC3 data).
- Compatibility improvements with Python 3.
Available under SSBX/IRAFX starting: Nov 2, 2015
This version builds upon the major set of changes implemented in v2.0.0 by not only fixing some bugs, but also cleaning up/changing/revising some APIs and docstrings. The complete list of changes includes:
- [API Change] The ‘updatewcs’ parameter was removed from both the
tweakreginteractive TEAL interfaces. The ‘updatewcs’ parameter can still be used with the Python interface for both the
astrodrizzle``() and ``tweakreg. Call the
stwcs.updatewcs.updatewcs()function separately before running
- [API Change] The stand-alone interface for the blot routine
ablot.blot()) has been revised to work seamlessly with astrodrizzle-generated products while being more obvious how to call it correctly. The help file for this task was also heavily revised to document all the input parameters and to provide an example of how to use the task.
- [API Change] Coordinate transformation task
skytopix) interfaces changed to be more consistent, yet remain backward-compatible for now.
tweakregnow return an output CD matrix which has identical cross-terms indicating the same scale and orientation in each axis (an orthogonal CD matrix). This relies on a revision to the
- The user interfaces to all 3 coordinate transformation tasks now use ‘coordfile’ as the input file of coordinates to transform. The use of ‘coords’ has been deprecated, but still can be used if needed. However, use of ‘coordfile’ will always override any input provided simultaneously with ‘coords’ parameter. Help files have been updated to document this as clearly as possible for users.
- User-provided list of input catalogs no longer needs to be matched exactly
with input files. As long as all input images are included in input catalog
list in any order,
tweakregwill apply the correct catalog to the correct file.
tweakreghas been updated to correctly and fully apply source selection criteria for both input source catalogs and reference source catalogs based on
- All use of keyword deletion has been updated in
fitsblender) to avoid warnings from astropy.
- All 3 coordinate transformation tasks rely on the input of valid WCS information for the calculations. These tasks now warn the user when it could not find a valid WCS and instead defaulted to using a unity WCS, so that the user can understand what input needs to be checked/revised to get the correct results.
- Exclusion/inclusion region files that can be used with
tweakregcan now be specified in image coordinates and sky coordinates and will only support files written out using DS9-compatible format.
- The filename for ‘final_refimage’ in
astrodrizzleand ‘refimage’ in
tweakregcan now be specified with OR without an extension, such as ‘[sci,1]’ or ‘’. If no extension is specified, it will automatically look for the first extension with a valid HSTWCS and use that. This makes the use of this parameter in both place consistent and more general than before.
- The reported fit as written out to a file has been slightly modified to report more appropriate numbers of significant digits for the results.
- Use of astrolib.coords was removed from
drizzlepacand replaced by use of astropy functions instead. This eliminated one more obsolete dependency in our software.
- Code was revised to rely entirely on
astropy.wcsinstead of stand-alone pywcs.
- Code was revised to rely entirely on
astropy.io.fitsinstead of stand-alone pyfits.
photeqtask to account for inverse sensitivity variations across detector chips and/or epochs.
- WFPC2 data from the archive with
DGEOFILEreference files will now need to be processed using
stwcs.updatewcsbefore running them through
tweakreg. This update converts the obsolete, unsupported
DGEOFILEcorrection for the WFPC2 data into a
D2IMFILEspecific for each WFPC2 observation, then uses that to convert the WCS based on the new conventions used for ACS and WFC3.
This set of changes represents the last major development effort for
DrizzlePac in support of HST. Support of this code will continue
throughout the lifetime of HST, but will be limited primarily to bug fixes
to keep the code viable as Python libraries used by
to develop and evolve with the language.
** Available under SSBX/IRAFX starting:** Aug 4, 2014
This version encompasses a large number of updates and revisions to the
DrizzlePac code, including the addition of new tasks and several parameter
name changes. The scope of these changes indicates the level of effort that
went into improving the
DrizzlePac code to make it easier and more
productive for users. The most significant updates to the
- The Python code has been updated to work identically (without change) under both Python 2.7 and Python 3.x.
- Implementing sky matching, a new algorithm for matching the sky across a set
of images being combined by
tweakregto now align full mosaics where some images may not overlap others in the mosaic.
- Added the option to write out single drizzle step images as compressed images (to save disk space for large mosaics, and I/O time for single drizzle step).
tweakregresidual plots visually while allowing them to be written out automatically when
tweakreggets run in non-interactive mode.
- Renamed parameters in
tweakregand imagefind to eliminate name clashes.
- Added option to select sources based on sharpness/roundness when
tweakregsearches for sources.
- Added support for exclusion and inclusion regions arbitrary shape/size when
tweakregsearches for sources.
- Added a full set of source detection parameters for reference image to
support multi-instrument alignment in
- Added support for new (simpler, more robust) ACS calibration of time-dependent distortion.
- A full 6-parameter general linear fit can now be performed using
tweakreg, in addition to shift and rscale.
- Cleaned up logic for sky-subtraction: user can now turn off sky-subtraction
with skysub=no, and still specify a user-defined sky value as the skyuser
keyword. This will reduce(eliminate?) the need to manually set
In addition to these major updates/changes, numerous smaller bugs were fixed and other revisions were implemented which affected a small portion of the use cases, such as:
- headerlet code now accepts lists of files to be updated.
- source sky positions (RA and Dec) now included in match file.
- DQ flags can now be taken into account when performing source finding in
- all intermediate files generated by
astrodrizzlewill now be removed when using ‘clean’=’yes’.
- a problem was fixed that caused
createMedianto crash where there were no good pixels in one of the images (when they did not overlap).
- interpretation of shiftfile now improved to handle arbitrarily-long filenames, rather than being limited to 24 character filenames.
- documentation has been updated, sometimes with a lot more extensive descriptions.
This version of
DrizzlePac also requires use of the latest release version
of astropy primarily for WCS and FITS I/O support.
Publicly Released through PyPI: Mar 27, 2014
Available under SSBX/IRAFX starting: Mar 13, 2014
- Support for WFPC2 GEIS input images improved to correctly find the associated DQ images.
- Static mask files created for all chips in an image now get deleted when using the ‘group’ parameter to only drizzle a single chip or subset of chips.
- Fixed problem caused by changes to
stsci.toolscode so that
drizzlepacwill reference the correct extensions in input images.
Publicly Released through PyPI: Jan 14, 2014
Available under SSBX/IRAFX starting: Jan 6, 2014
- Files created or updated by
apply_headerlet, will now ensure that the
NEXTENDkeyword value correctly reflects the number of extensions in the FITS file upon completion.
Installed in OPUS: Dec 11, 2013
Available starting: Oct 28, 2013
- DQ arrays in input images now get updated with cosmic-ray masks
astrodrizzlewhen run with the parameter
in_memory=True. This restored the cosmic-ray masks detected during pipeline processing.
available starting: Oct 21, 2013
tweakregcan now be run in ‘batch’ mode. This allows the user to generate plots and have them saved to disk automatically without stopping processing and requiring any user input.
available starting: Sept 9, 2013
This version fixed a couple of bugs in
- Logic was updated to support pixfrac = 0.0 without crashing. Ths code will now automatically reset the kernel to ‘point’ in that case.
astrodrizzlenow forcibly removes all OPUS WCS keywords from drizzle product headers.
- Default rules for generating drizzle product headers (as used in the archive) were modified to add definitions for ‘float_one’, ‘int_one’, ‘zero’ that generate output values of 1.0, 1, and 0 (zero) respectively for use as keyword values. This allows the LTM* rules to replace ‘first’ with ‘float_one’ so that the physical and image coordinates for drizzle products are consistent.
Additionally, changes were made to
STWCS for reprocessing use:
- Problems with using
STWCSpackage on WFPC2 data have been corrected in this revision.
Available starting: July 15, 2013
- AstroDrizzle now can process all STIS data without crashing.
available starting: May 6, 2013
- The output drizzle image header no longer contains references to D2IM arrays.
tweakregto work with drizzled images as input where 2-D D2IM corrections were needed.
- Deprecated references to PyFITS .has_key() methods were also removed from the entire package, making it compatible with PyFITS 3.2.x and later.
available starting: Feb 11, 2013
- Fixed a bug in
astrodrizzlewhich caused blot to raise an exception when using ‘sinc’ interpolation.
- Cleaned up the logic for writing out the results from the pixtopix, pixtosky, and skytopix tasks to avoid an Exception when a list of inputs are provided and no output file is specified.
- A new parameter was added to the tweakback task to allow a user to specify
the value of
WCSNAMEwhen updating the FLT images with a new solution from a DRZ image header.
- Code in tweakback for updating the header with a new WCS will now
automatically generate a unique
WCSNAMEif the there is a WCS solution in the FLT headers with the default or user-defined value of
available starting: Feb 4, 2013
- Updated astrodrizzle to work with input images which do not have
WCSNAMEdefined. This should make it easier to support non-HST input images in the future.
- cleared up confusion between flux parameters in imagefindpars and catalog
- turned of use of fluxes for trimming input source catalogs when no flux column can be found in input source catalogs.
available starting: Dec 10, 2012
tweakreg2d histogram building mode to correctly find the peak when all the inputs match with the same offset (no spurious sources in either source catalog).
- Fixed a bug so that Ctrl-C does not cause an exception when used while
- revised the source finding logic to ignore sources near the image edge, a change from how daofind works (daofind expands the image with blanks then fits anyway).
- created a new function to apply the nsigma separation criteria to (try to) eliminate duplicate entries for the same source from the source list. It turns out daofind does have problems with reporting some duplicate sources as well. This function does not work perfectly, but works to remove nearly all (if not all) duplicates in most cases.
available starting: Jan 14, 2013
- Bug fixed in updatehdr module to allow shiftfiles without RMS columns to work as inputs to manually apply shifts to headers of input images.
astrodrizzleto update WCS of all input images BEFORE checking whether or not they are valid. This ensures that all files provided as input to
astrodrizzlein the pipeline have the headers updated with the distortion model and new WCS.
- Images with NGOODPIX=0 now identified for WFC3 and WFPC2 inputs, so they
can be ignored during
- Replaced 2d histogram building code originally written in Python with a C function that run about 4x faster.
available starting: Dec 10, 2012
tweakregv1.1.0 source finding algorithm now runs many times faster (no algorithmic changes). No changes have been made yet to speed up the 2d histogram source matching code.
- The ‘pixtopix’ task was updated to make the ‘outimage’ parameter optional by using the input image as the default. This required no API changes, but the help files were updated.
- Very minor update to guard against MDRIZTAB being specified without any explicit path.
astrodrizzleto correctly report the exposure time, exposure start, and exposure end for the single drizzle products, in addition to insuring the final drizzle values remain correct.
astrodrizzlealso includes initial changes to safeguard the C code from getting improperly cast values from the configObj(TEAL) input.
available starting: Oct 29, 2012
- Scaling of sky array for WFC3/IR IVM generation now correct.
- template mask files for WFPC2 no longer generated so that WFPC2 data can now be processed using num_cores > 1 (parallel processing).
- interpretation of the ‘group’ parameter fixed to support a single integer, a comma-separated list of integers or a single ‘sci,<n>’ value. The values correspond to the FITS extension number of the extensions that should be combined. This fix may also speed up the initialization step as more direct use of pyfits was implemented for the interpretation of the ‘group’ parameter.
available starting: Sept 26, 2012
The HST Archive and operational calibration pipeline started using this version on Sept 26, 2012.
available starting: Sept 24, 2012
- Bug fixed to allow use of final_wht_type=IVM for processing WFPC2 data.
- Revised Initialization processing to speed it up by using more up-to-date, direct pyfits calls.
available starting: Sept 17, 2012
- Fixed the logic so that crclean images always get created regardless of the value of the ‘clean’ parameter.
available starting: Sept 10, 2012
- Remove the restriction of only being able to process images which have
WCSNAMEkeyword as imposed by r15631. The removal of this restriction will now allow for processing of non-updated input files with
updatewcs=Falsefor cases where no distortion model exists for the data (as required by CADC).
- Added log statements reporting what sky value was actually used in the drizzle and blot steps
available starting: Sept 3, 2012
- Major revision to
astrodrizzleallowing the option to process without writing out any intermediate products to disk. The intermediate products remain in memory requiring significantly more memory than usual. This improves the overall processing time by eliminating as much disk activity as possible as long as the OS does not start disk swapping due to lack of RAM.
- revised to turn off ‘updatewcs’ when coeffs=False(no) so that exposures with filter combinations not found in the IDCTAB will not cause an error.
available starting: Aug 27, 2012
- Fixes problems with missing single_sci images.
- Static mask step revised to skip updates to static mask if all pixel data
falls within a single histogram bin. This avoids problems with masking out
entire images, which happens if low S/N SBC data is processed with
available starting: Aug 20, 2012
Use of IVM for final_wht now correct, as previous code used wrong inputs when
IVM weighting was automatically generated by
available starting: Aug 13, 2012
- Completely removed the use of the TIME arrays for weighting IR drizzle products so that the photometry for saturated sources in drizzled products now comes out correct.
- Corrected a problem with
astrodrizzlewhich affected processing of WFPC2 data where CRPIX2 was not found when creating the output single sci image.
available starting: Aug 3, 2012
The complete version of stsci_python can be downloaded from our download page
Used in archive/pipeline starting: July 10, 2012
Pipeline and archive started processing ACS data with this version.
Used in archive/pipeline starting: June 6, 2012
Pipeline and archive first started using
astrodrizzle by processing WFC3