diff --git a/Basics/Calexp_guided_tour.ipynb b/Basics/Calexp_guided_tour.ipynb new file mode 100644 index 00000000..dc7f37d5 --- /dev/null +++ b/Basics/Calexp_guided_tour.ipynb @@ -0,0 +1,754 @@ +{ + "cells": [ + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "# A Guided Tour of LSST Calexps\n", + "
Owner(s): **David Shupe** ([@stargaser](https://github.com/LSSTScienceCollaborations/StackClub/issues/new?body=@stargaser))\n", + "
Last Verified to Run: **2018-08-07**\n", + "
Verified Stack Release: **v16.0** (also lsst_w_2018_31, with `getName` modification)\n", + "\n", + "We'll inspect a visit image ``calexp`` object, and then show how a coadd image differs.\n", + "\n", + "### Learning Objectives:\n", + "\n", + "After working through this tutorial you should be able to follow some best practices when working with LSST ``calexp`` (image) objects.\n", + "\n", + "### Logistics\n", + "This notebook is intended to be runnable on `lsst-lspdev.ncsa.illinois.edu` from a local git clone of https://github.com/LSSTScienceCollaborations/StackClub.\n", + "\n", + "## Set-up" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from lsst.daf.persistence import Butler" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import lsst.afw.display as afw_display" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Retrieving and inspecting a calexp" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the first part of this tutorial we will use simulated LSST data from Twinkles, see https://github.com/LSSTDESC/Twinkles/blob/master/README.md" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Define a data directory and create a Butler" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "datadir = '/project/shared/data/Twinkles_subset/output_data_v2'\n", + "butler = Butler(datadir)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Other notebooks show how to view what data are available in a Butler object. Here we get a specific one." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "dataId = {'filter': 'r', 'raft': '2,2', 'sensor': '1,1', 'visit': 235}\n", + "calexp = butler.get('calexp', **dataId)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "In terms of pixel data, a calexp contains an image, a mask, and a variance.\n", + "\n", + "Let's see how to access the image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.image" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To show the pixel data, we will make use of the matplotlib backend to `lsst.afw.display`.\n", + "\n", + "Due to current limitations of this backend, the display must be defined and used in the same code cell, much as matplotlib commands in a notebook must all be in one cell to produce a plot." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "%matplotlib inline" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If the entire calexp is displayed, masks will be overlaid. Here we will eschew the mask display by showing only the image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(calexp.image)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "To access the pixel values as an array, use the `.array` attribute." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "data = calexp.image.array\n", + "data" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "data.__class__" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's list all the methods for our calexp." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp_methods = [m for m in dir(calexp) if not m.startswith('_')]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp_methods" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Access the masked Image" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.maskedImage" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Access the variance object and the underlying Numpy array" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.variance" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.variance.array" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Access the mask and its underlying array" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.mask" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.mask.array" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Get the dimensions of the image, mask and variance" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.getDimensions()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The image, maskedImage and Exposure objects in `lsst.afw.display` include information on **LSST pixels**, which are 0-based with an optional offset.\n", + "\n", + "For a calexp these are usually zero." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.getXY0()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.getX0(), calexp.getY0()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Access the wcs object" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "wcs = calexp.getWcs()\n", + "wcs" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The WCS object can be used e.g. to convert pixel coordinates into sky coordinates" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "wcs.pixelToSky(100.0, 100.0)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Let's try accessing the metadata, and see what (header) keywords we have." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "metadata = calexp.getMetadata()\n", + "# help(metadata)\n", + "metadata.getOrderedNames()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "metadata.get('CCDTEMP')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "> Post release 16.0, the `getName` method will be available." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Check if our calexp has a PSF" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.hasPsf()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "psf = calexp.getPsf()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The PSF object can be used to get a realization of a PSF at a specific point" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "from lsst.geom.coordinates import Point2D\n", + "psfimage = psf.computeImage(Point2D(100.,100.))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Access the calibration object which can be used to convert instrumental magnitudes to AB magnitudes" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calib = calexp.getCalib()\n", + "calib" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "### Image cutouts" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We can make a cutout from the calexp in our session." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "import lsst.afw.geom as afwGeom\n", + "import lsst.afw.image as afwImage" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "bbox = afwGeom.Box2I()\n", + "bbox.include(afwGeom.Point2I(2200,3200))\n", + "bbox.include(afwGeom.Point2I(2800,3800))\n", + "cutout = calexp.Factory(calexp, bbox, afwImage.LOCAL)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Notice that when the image is displayed, the pixel values relate to the parent image." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(cutout.image)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The coordinate of the lower-left-hand pixel is XY0." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "cutout.getXY0()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "If a cutout was all that was desired from the start, we could have used our BoundingBox together with our Butler to have read in only the cutout." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "cutout_calexp = butler.get('calexp_sub', bbox=bbox, immediate=True, dataId=dataId)\n", + "cutout_calexp.getDimensions()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(cutout_calexp.image)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The `clone` method makes a deep copy. The result can be sliced with a BoundingBox" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "clone_cutout = calexp.clone()[bbox]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(clone_cutout.image)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "## Repeat for a coadd" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For this last section, we will use Hyper Suprime-Cam (HSC) data that has been modified for tutorial purposes." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd_butler = Butler('/project/shared/data/with-globular/')" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd_butler.getKeys('deepCoadd_calexp')" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "We cannot use queryMetadata to look up what is available for coadds. This will be fixed in Butler Gen3." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "For the time being, open a terminal and list files in `/project/shared/data/with-globular/` to see what's available. Or just carry on using the following example:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "dataId = {'filter':'HSC-I', 'tract':9813, 'patch':'4,4'}" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Retrieve a coadd `calexp`, and see what methods it provides." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd = coadd_butler.get('deepCoadd_calexp', dataId)" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd_methods = [m for m in dir(coadd) if not m.startswith('_')]" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd_methods" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "set(coadd_methods).symmetric_difference(set(calexp_methods))" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "The result of the `set` command above shows that a calexp and a coadd have the same methods." + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "A `deepCoadd_calexp` and a visit `calexp` differ mainly in the masks and the xy0 value." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "calexp.mask.getMaskPlaneDict()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd.mask.getMaskPlaneDict()" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "coadd.getXY0()" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Display the coadd with all masks visible." + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(coadd)" + ] + }, + { + "cell_type": "markdown", + "metadata": {}, + "source": [ + "Display the image data only with a zoom and pan to some nice-looking galaxies, to show off our hyperbolic arcsine stretch:" + ] + }, + { + "cell_type": "code", + "execution_count": null, + "metadata": {}, + "outputs": [], + "source": [ + "display1 = afw_display.Display(frame=1, backend='matplotlib')\n", + "display1.scale(\"asinh\", \"zscale\")\n", + "display1.mtv(coadd.image)\n", + "display1.zoom(16)\n", + "display1.pan(18700, 17000)" + ] + } + ], + "metadata": { + "kernelspec": { + "display_name": "LSST", + "language": "python", + "name": "lsst" + }, + "language_info": { + "codemirror_mode": { + "name": "ipython", + "version": 3 + }, + "file_extension": ".py", + "mimetype": "text/x-python", + "name": "python", + "nbconvert_exporter": "python", + "pygments_lexer": "ipython3", + "version": "3.6.2" + } + }, + "nbformat": 4, + "nbformat_minor": 2 +} diff --git a/Basics/README.rst b/Basics/README.rst index e16e174c..7ab4161c 100644 --- a/Basics/README.rst +++ b/Basics/README.rst @@ -14,6 +14,17 @@ This folder contains a set of tutorial notebooks exploring the basic properties - Owner + * - **Calexp_guided_tour.ipynb** + - Shows how to read an exposure object from a data repository, and how to access and display various parts. + - `ipynb `_, + `rendered `_ + + .. image:: https://github.com/LSSTScienceCollaborations/StackClub/blob/rendered/Basics/log/Calexp_guided_tour.svg + :target: https://github.com/LSSTScienceCollaborations/StackClub/blob/rendered/Basics/log/Calexp_guided_tour.log + + - `David Shupe `_ + + * - **Data Inventory** - Explore the available datasets in the LSST Science Platform shared folders. - `ipynb `_,