spacetelescope / acs-notebook

Python Jupyter Notebooks that support users of the ACS instrument on board the Hubble Space Telescope

acs-notebook

This repository contains Jupyter notebooks to demonstrate how to calibrate and analyze data from the Hubble Space Telescope (HST) Advanced Camera for Surveys (ACS). Users are advised to visit the ACS website, Instrument Handbook, and Data Handbook for more information about the current status of ACS, instrument specifications, and data analysis.

Users who need help transitioning from IRAF/PyRAF to Python should see the stak-notebooks repository.

If you have questions about HST data analysis, calibration software, instrument capabilities, and/or the methods discussed in this repository, please visit the HST Help Desk. Through the help desk portal, you can explore the HST Knowledge Base and request additional help from experts.

Getting Started

To download the notebooks in this repository, simply open a terminal, go to the directory where you would like the notebooks to be, and type

git clone https://github.com/spacetelescope/acs-notebook.git

which will create a new acs-notebook/ directory containing the contents of this repository.

Warning: Before running these examples, you must install or update to the latest version of stenv.

Jupyter Notebooks allow code to be packaged with formatted text to create illustrative examples. Users who are unfamiliar with Jupyter Notebooks should also see the short guide for how to use these tools.

Contents

Simple Example Notebooks

• acs_subarrays: Instructions for how to use the acstools.acs_destripe_plus module to calibrate, de-stripe, and correct ACS subarray observations for charge transfer efficiency (CTE). Also included are instructions for updating the OSCNTAB reference file to subtract the bias level from the prescan columns in user-defined subarray observations.

• acs_pixel_area_maps: This notebook shows how to use Python to create pixel area maps (PAMs) to account for the effects of geometric distortion in distorted images. This enables users to perform photometry on distorted images.

• acs_polarization_tools: ACS/WFC offers two sets of polarizers, UV and visible, for imaging polarimetry and imaging spectropolarimetry. This notebook provides guidance on using the acstools.polarization_tools module to retrieve polarization calibration coefficients and calculate polarization properties of a source.

Complex Workflow Notebooks

• acs_reduction: This worked example demonstrates how to use the CALACS pipeline to re-process raw ACS data retrieved from MAST. Users are shown how to update reference files in the image headers, retrieve reference files from the Calibration Reference Data System (CRDS), and how to toggle steps in the calibration pipeline (e.g., CTE correction).

• acs_cte_forward_model The capability to simulate the readout of an ACS image at a given epoch is a feature of CALACS that does not run during standard calibration. This example demonstrates how to take a pristine image, or one corrected for time-dependent CTE loss, and simulate CTE losses for an arbitrary observation date using acstools.acscteforwardmodel.

• acs_saturation_trails The ACS/WFC CCDs remain linear beyond the full-well saturation, which allows users the opportunity to perform photometry on saturated stars. This notebook demonstrates the methods and caveats when performing saturated star photometry.

• acs_sbc_dark_analysis: The ACS Solar Blind Channel (SBC) has nominally negligible dark current. However, the temperature of the SBC detector increases steadily over time while it is in use, and the dark current is proportional to temperature. At temperatures above 25 ºC, the dark current is no longer negligible and must be subracted. This workflow demonstrates how to account for dark current in ACS/SBC observations.

• acs_findsat_mrt_example.ipynb: ACS/WFC imaging data is often affected by contamination by artificial satellites, compromising science data. The acstools.findsat_mrt module can be used to identify satellite trails and create masks to reject affected pixels from further analysis.

Contributing

If you have feedback concerning our documentation or examples presented here on GitHub, please open an issue. For feedback concerning content exclusively from the ACS website, Instrument Handbook, or Data Handbook, or for questions about ACS in general, please contact the ACS Branch at STScI via the Hubble Space Telescope Help Desk portal.

acs-notebook's Issues

DOC: Have README link to notebook file directly

Under "contents", notebook names are highlighted but then I have to dig through the different directories to get to them. It would be easier if I could click on a link directly.

Subarray CTE notebook

Some of the formatting in the subarray CTE notebook needs to be altered. In particular, div alert boxes should be used to warnings and important notes. Also emphasize more strongly to users that they will need to set up JREF to run the example. Add in a table of contents.

MNT: Remove __pycache__ from version control

I see __pycache__ directory in this repo. It is not supposed to be version controlled as it is generated by Python when you run the stuff locally in your source checkout.

Also have git ignore it permanently in .gitignore file in the root directory.

Add new throughput curves page

Replace http://www.stsci.edu/hst/acs/analysis/throughputs with a notebook.
Add new page with throughput curve example shown. Make sure it re-creates all of the plots on the page dynamically, so they are the most up to date possible.

If we do this, we may want to move the downloadable throughput files to another page similar to the ACS aperture corrections page and then just link to it. Unless we want the notebook to make these files on the fly or replace them in some other manner. @nmiles2718 any thoughts?

Update the help

Tyler Desjardins mentions that we should consider moving emails from help[at]stsci.edu to point to the web portal where possible and appropriate. For HST (or any non-JWST), it is https://hsthelp.stsci.edu . For JWST, it is https://jwsthelp.stsci.edu . Please update info in setup.py, setup.cfg, documentation, etc as appropriate.

Please close this issue if it is irrelevant to your repository. This is an automated issue. If this is opened in error, please let pllim know!

xref spacetelescope/hstcal#317

Make a conda environment

Provide a conda environment file that has all of the stuff necessary for running the examples (astroquery, nodejs, etc.). I'm not sure if the jupyterlab extensions can also be included this way (I hope so!).

Notebook crash when rerun with existing FITS files

I was re-running acs_destripe_plus notebook to test the changes, but it crashed when it attempted to overwrite existing FITS file. Either clarify in the notebook that user need to remove existing files or gracefully handle the situation.

Zeropoints Notebook Updates

The zeropoints notebook needs a few updates:

• The pivot wavelength is incorrect. It should be band.pivot(), not obs.pivot().
• The link to the zeropoints calculator webpage should be included since we are keeping it. Users should be told that both the notebook and the webpage tool use the same method and give the same answers.
• In the photometric systems section, the equation that defines vegamag says F = ... instead of M = ...
• All links that are not anchors to other parts of the page should open in a new tab

Change shutil.move to os.rename

According to Erik, os.rename will get around the error raised by shutil.move if the file already exists. We can then simplify all of the cells using astroquery that currently have try except blocks in them.

Comparison with 2004 PAM

I suggest you give more details about the 0.2% difference.
(Also I think it would be better to quote the ratio rather than the difference... making sure your PAM is generated using a dataset taken in 2004 in the F606W filter. That would tell you how much impact the improved distortion solution makes).

In 2004, a single PAM was created using the F606W distortion solution, based on polynomial fits applied by the IDCTAB reference file. The PAM did not account for filter-dependent polynomial solutions, higher order corrections applied now via the D2IMFILE and the NPOLFILE, a time-dependent skew corrections, or the effect of velocity aberration at the time of observation.

I'm not sure which of these would be the biggest effect, but you could determine it empirically by making different PAMs... one with the NPOLFILE set to N/A, one with the VAFACTOR set to 1.0, etc.

This is what I sent to WFC3 team, which puts the responsibility on the user for determining how accurate the PAM needs to be.
"These static files have not been updated to reflect improvements to the distortion model, and as the model has become more complex, a ‘one-size-fits-all’ PAM may no longer provide the accuracy required for some science goals."

Update links to new Jahia and HDox versions

Many of the links in the notebooks are now broken due to Jahia and HDox launches.

MNT: Delete stale calib_example branch

There is a stale calib_example branch that one inherits when forking this repo. Deleting the branch would cause less confusion.

CALACS Notebook

Create a new notebook detailing the behavior of calacs. This does not need to have comprehensive examples as there will be a CSI notebook with that, but it can have the examples presented in the Data Handbook. It should also mention acs_destripe_plus and link to the subarray CTE correction page.

Change description kernel

“This tutorial will show you how to…” in the intro of this notebook seems to be talking about the dark subtraction of SBC data.