<jupyter_start><jupyter_text>Working with FITS-cubes  Authors[<NAME> (DK)](http://www.astronomy.dk), [<NAME>](http://www.astro.wisc.edu/our-people/post-doctoral-students/shetty-shravan/), [<NAME>](http://www.astro.wisc.edu/our-people/graduate-students/gonzalez-casanova-diego/), [<NAME>](http://www.astro.wisc.edu/our-people/scientists/hernandez-audra/), <NAME>, <NAME>, <NAME>glas Learning Goals* Find and download data using `astroquery`* Read and plot slices across different dimensions of a data cube* Compare different data sets (2D and 3D) by overploting contours* Transform coordinate projections and match data resolutions with `reproject`* Create intensity moment maps / velocity maps with `spectral_cube` KeywordsFITS, image manipulation, data cubes, radio astronomy, WCS, astroquery, reproject, spectral cube, matplotlib, contour plots, colorbar SummaryIn this tutorial we will visualize 2D and 3D data sets in Galactic and equatorial coordinates. The tutorial will walk you though a visual analysis of the Small Magellanic Cloud (SMC) using HI 21cm emission and a Herschel 250 micron map. We will learn how to read in data from a file, query and download matching data from Herschel using astroquery, and plot the resulting images in a multitude of ways. The primary libraries we'll be using are: [astroquery](http://www.astropy.org/astroquery/), [spectral_cube](https://spectral-cube.readthedocs.io/en/latest/), [reproject](https://reproject.readthedocs.io/en/stable/), [matplotlib](https://matplotlib.org/)) They can be installed using conda: ```conda install -c astropy astroqueryconda install -c astropy spectral-cubeconda install -c astropy reproject``` Alternatively, if you don't use conda, you can use pip.<jupyter_code>import numpy as np
import matplotlib.pyplot as plt
from matplotlib.colors import LogNorm

import astropy.units as u
from astropy.utils.data import download_file
from astropy.io import fits  # We use fits to open the actual data file

from astropy.utils import data
data.conf.remote_timeout = 60

from spectral_cube import SpectralCube

from astroquery.esasky import ESASky
from astroquery.utils import TableList
from astropy.wcs import WCS
from reproject import reproject_interp<jupyter_output><empty_output><jupyter_text>Download the HI DataWe'll be using HI 21 cm emission data from the [HI4Pi survey](http://adsabs.harvard.edu/cgi-bin/bib_query?arXiv:1610.06175).  We want to look at neutral gas emission from the Magellanic Clouds and learn about the kinematics of the system and column densities. Using the VizieR catalog, we've found a relevant data cube to use that covers this region of the sky. You can also download an allsky data cube, but this is a very large file, so picking out sub-sections can be useful!For us, the [relevant file is available via ftp from CDS Strasbourg](http://cdsarc.u-strasbg.fr/vizier/ftp/cats/J/A+A/594/A116/CUBES/GAL/TAN/TAN_C14.fits). We have a reduced version of it which will be a FITS data cube in Galactic coordinates using the tangential sky projection.Sure, we could download this file directly, but why do that when we can load it up via one line of code and have it ready to use in our cache? Download the HI Fits Cube<jupyter_code># Downloads the HI data in a fits file format
hi_datafile = download_file(
    'http://data.astropy.org/tutorials/FITS-cubes/reduced_TAN_C14.fits',
    cache=True, show_progress=True)<jupyter_output><empty_output><jupyter_text>Awesome, so now we have a copy of the data file (a FITS file).  So how do we do anything with it?Luckily for us, the [spectral_cube](https://spectral-cube.readthedocs.io/en/latest/) package does a lot of the nitty gritty work for us to manipulate this data and even quickly look through it. So let's open up our data file and read in the data as a SpectralCube!The variable `cube` has the data using SpectralCube and `hi_data` is the data cube from the FITS file without the special formating from SpectralCube.<jupyter_code>hi_data = fits.open(hi_datafile)  # Open the FITS file for reading
cube = SpectralCube.read(hi_data)  # Initiate a SpectralCube
hi_data.close()  # Close the FITS file - we already read it in and don't need it anymore!<jupyter_output><empty_output><jupyter_text>If you happen to already have the FITS file on your system, you can also skip the fits.open step and just directly read a FITS file with SpectralCube like this:`cube = SpectralCube.read('path_to_data_file/TAN_C14.fits') `So what does this SpectralCube object actually look like? Let's find out!  The first check is to print out the cube.<jupyter_code>print(cube)<jupyter_output><empty_output><jupyter_text>Some things to pay attention to here:A data cube has three axes. In this case, there is Galactic Longitude (x), Galactic Latitude (y), and a spectral axis in terms of a LSR Velocity (z - listed as s with `spectral_cube`).The data hidden in the cube lives as an ndarray with shape (n_s, n_y, n_x) so that axis 0 corresponds with the Spectral Axis, axis 1 corresponds with the Galactic Latitude Axis, and axis 2 corresponds with the Galactic Longitude Axis. When we `print(cube)` we can see the shape, size, and units of all axes as well as the data stored in the cube. With this cube, the units of the data in the cube are temperatures (K). The spatial axes are in degrees and the Spectral Axis is in (meters / second).The cube also contains information about the coordinates corresponding to the data in the form of a WCS (World Coordinate System) object. SpectralCube is clever and keeps all the data masked until you really need it so that you can work with large sets of data. So let's see what our data actually looks like!SpectralCube has a `quicklook()` method which can give a handy sneak-peek preview of the data.  It's useful when you just need to glance at a slice or spectrum without knowing any other information (say, to make sure the data isn't corrupted or is looking at the right region.)  To do this, we index our cube along one axis (for a slice) or two axes (for a spectrum):<jupyter_code>cube[300, :, :].quicklook()  # Slice the cube along the spectral axis, and display a quick image
cube[:, 75, 75].quicklook()  # Extract a single spectrum through the data cube<jupyter_output><empty_output><jupyter_text>Try messing around with slicing the cube along different axes, or picking out different spectra  Make a smaller cube, focusing on the Magellanic CloudsThe HI data cube we downloaded is bigger than we actually need it to be.  Let's try zooming in on just the part we need and make a new `sub_cube`. The easiest way to do this is to cut out part of the cube with indices or coordinates.We can extract the world coordinates from the cube using the `.world()` method. Warning: using .world() will extract coordinates from every position you ask for. This can be a TON of data if you don't slice through the cube.  One work around is to slice along two axes and extract coordinates just along a single dimension. The output of `.world()` is an Astropy `Quantity` representing the pixel coordinates, which includes units. You can extract these Astropy `Quantity` objects by slicing the data.<jupyter_code>_, b, _ = cube.world[0, :, 0]  #extract latitude world coordinates from cube
_, _, l = cube.world[0, 0, :]  #extract longitude world coordinates from cube<jupyter_output><empty_output><jupyter_text>You can then extract a `sub_cube` in the spatial coordinates of the cube<jupyter_code># Define desired latitude and longitude range
lat_range = [-46, -40] * u.deg 
lon_range = [306, 295] * u.deg

# Create a sub_cube cut to these coordinates
sub_cube = cube.subcube(xlo=lon_range[0], xhi=lon_range[1], ylo=lat_range[0], yhi=lat_range[1])

print(sub_cube)<jupyter_output><empty_output><jupyter_text>Cut along the Spectral Axis:We don't really need data from such a large velocity range so let's just extract a little slab. We can do this in any units that we want using the `.spectral_slab()` method.<jupyter_code>sub_cube_slab = sub_cube.spectral_slab(-300. *u.km / u.s, 300. *u.km / u.s)

print(sub_cube_slab)<jupyter_output><empty_output><jupyter_text>Moment MapsMoment maps are a useful analysis tool to study data cubes.  In short, a moment is a weighted integral along an axis (typically the Spectral Axis) that can give information about the total Intensity (or column density), mean velocity, or velocity dispersion along lines of sight. SpectralCube makes this very simple with the `.moment()` method. We can convert to friendlier spectral units of km/s and these new 2D projections can be saved as new FITS files, complete with modified WCS information as well.<jupyter_code>moment_0 = sub_cube_slab.with_spectral_unit(u.km/u.s).moment(order=0)  # Zero-th moment 
moment_1 = sub_cube_slab.with_spectral_unit(u.km/u.s).moment(order=1)  # First moment

# Write the moments as a FITS image
# moment_0.write('hi_moment_0.fits') 
# moment_1.write('hi_moment_1.fits')

print('Moment_0 has units of: ', moment_0.unit)
print('Moment_1 has units of: ', moment_1.unit)

# Convert Moment_0 to a Column Density assuming optically thin media
hi_column_density = moment_0 * 1.82 * 10**18 / (u.cm * u.cm) * u.s / u.K / u.km<jupyter_output><empty_output><jupyter_text>Display the Moment MapsThe [WCSAxes](http://docs.astropy.org/en/stable/visualization/wcsaxes/) framework in Astropy allows us to display images with different coordinate axes and projections.As long as we have a WCS object associated with the data, we can transfer that projection to a matplotlib axis. SpectralCube makes it possible to access just the WCS object associated with a cube object.<jupyter_code>print(moment_1.wcs)  # Examine the WCS object associated with the moment map<jupyter_output><empty_output><jupyter_text>As expected, the first moment image we created only has two axes (Galactic Longitude and Galactic Latitude).  We can pass in this WCS object directly into a matplotlib axis instance.<jupyter_code># Initiate a figure and axis object with WCS projection information
fig = plt.figure(figsize=(18, 12))
ax = fig.add_subplot(111, projection=moment_1.wcs)

# Display the moment map image
im = ax.imshow(moment_1.hdu.data, cmap='RdBu_r', vmin=0, vmax=200)
ax.invert_yaxis()  # Flips the Y axis 

# Add axes labels
ax.set_xlabel("Galactic Longitude (degrees)", fontsize=16)
ax.set_ylabel("Galactic Latitude (degrees)", fontsize=16)

# Add a colorbar
cbar = plt.colorbar(im, pad=.07)
cbar.set_label('Velocity (km/s)', size=16)

# Overlay set of RA/Dec Axes
overlay = ax.get_coords_overlay('fk5')
overlay.grid(color='white', ls='dotted', lw=2)
overlay[0].set_axislabel('Right Ascension (J2000)', fontsize=16)
overlay[1].set_axislabel('Declination (J2000)', fontsize=16)

# Overplot column density contours 
levels = (1e20, 5e20, 1e21, 3e21, 5e21, 7e21, 1e22)  # Define contour levels to use
ax.contour(hi_column_density.hdu.data, cmap='Greys_r', alpha=0.5, levels=levels)<jupyter_output><empty_output><jupyter_text>As you can see, the WCSAxes framework is very powerful and similiar to making any matplotlib style plot.   Display a Longitude-Velocity SliceThe [WCSAxes](http://docs.astropy.org/en/stable/visualization/wcsaxes/) framework in Astropy also lets us slice the data accross different dimensions. It is often useful to slice along a single latitude and display an image showing longtitude and velocity information only (position-velocity or longitude-velocity diagram).This can be done by specifying the `slices` keyword and selecting the appropriate slice through the data. `slices` requires a 3D tuple containing the index to be sliced along and where we want the two axes to be displayed. This should be specified in the same order as the WCS object (longitude, latitude, velocity) as opposed to the order of numpy array holding the data  (velocity, latitude, longitude). We then select the appropriate data by indexing along the numpy array.<jupyter_code>lat_slice = 18  # Index of latitude dimension to slice along

# Initiate a figure and axis object with WCS projection information
fig = plt.figure(figsize=(18, 12))
ax = fig.add_subplot(111, projection=sub_cube_slab.wcs, slices=('y', lat_slice, 'x'))
# Above, we have specified to plot the longitude along the y axis, pick only the lat_slice 
# indicated, and plot the velocity along the x axis

# Display the slice
im = ax.imshow(sub_cube_slab[:, lat_slice, :].transpose().data)  # Display the image slice
ax.invert_yaxis()  # Flips the Y axis 

# Add axes labels
ax.set_xlabel("LSR Velocity (m/s)", fontsize=16)
ax.set_ylabel("Galactic Longitude (degrees)", fontsize=16)

# Add a colorbar
cbar = plt.colorbar(im, pad=.07, orientation='horizontal')
cbar.set_label('Temperature (K)', size=16)<jupyter_output><empty_output><jupyter_text>As we can see, the SMC seems to be only along positive velocities.  Try: Create a new spectral slab isolating just the SMC and slice along a different dimension to create a latitude-velocity diagram  Find and Download a Herschel ImageThis is great, but we want to compare the HI emission data with Herschel 350 micron emission to trace some dust. This can be done with [astroquery](http://www.astropy.org/astroquery/). We can query for the data by mission, take a quick look at the table of results, and download data after selecting a specific wavelength or filter. Since we are looking for Herschel data from an ESA mission, we will use the [astroquery.ESASky](http://astroquery.readthedocs.io/en/latest/esasky/esasky.html) class.Specifically, the `ESASKY.query_region_maps()` method allows us to search for a specific region of the sky either using an Astropy SkyCoord object or a string specifying an object name. In this case, we can just search for the SMC. A radius to search around the object can also be specified.<jupyter_code># Query for Herschel data in a 1 degree radius around the SMC
result = ESASky.query_region_maps('SMC', radius=1*u.deg, missions='Herschel')

print(result)<jupyter_output><empty_output><jupyter_text>Here, the result is a TableList which contains 24 Herschel data products that can be downloaded. We can see what information is available in this TableList by examining the keys in the Herschel Table.<jupyter_code>result['HERSCHEL'].keys()<jupyter_output><empty_output><jupyter_text>We want to find a 350 micron image, so we need to look closer at the filters used for these observations.<jupyter_code>result['HERSCHEL']['filter']<jupyter_output><empty_output><jupyter_text>Luckily for us, there is an observation made with three filters: 250, 350, and 500 microns.  This is the object we will want to download. One way to do this is by making a boolean mask to select out the Table entry corresponding with the desired filter. Then, the `ESASky.get_maps()` method will download our data provided a TableList argument. Note that the below command requires an internet connection to download five quite large files. It could take several minutes to complete.<jupyter_code>filters = result['HERSCHEL']['filter'].astype(str)  # Convert the list of filters from the query to a string

# Construct a boolean mask, searching for only the desired filters
mask = np.array(['250, 350, 500' == s for s in filters], dtype='bool')

# Re-construct a new TableList object containing only our desired query entry
target_obs = TableList({"HERSCHEL":result['HERSCHEL'][mask]})  # This will be passed into ESASky.get_maps()

IR_images = ESASky.get_maps(target_obs)  # Download the images
IR_images['HERSCHEL'][0]['350'].info()  # Display some information about the 350 micron image<jupyter_output><empty_output><jupyter_text>Since we are just doing some qualitative analysis, we only need the image, but you can also access lots of other information from our downloaded object, such as errors. Let's go ahead and extract just the WCS information and image data from the 350 micron image.<jupyter_code>herschel_header = IR_images['HERSCHEL'][0]['350']['image'].header
herschel_wcs = WCS(IR_images['HERSCHEL'][0]['350']['image'])  # Extract WCS information
herschel_imagehdu = IR_images['HERSCHEL'][0]['350']['image']  # Extract Image data
print(herschel_wcs)<jupyter_output><empty_output><jupyter_text>With this, we can display this image using matplotlib with [WCSAxes](http://docs.astropy.org/en/stable/visualization/wcsaxes/index.html) and the `LogNorm()` object so we can log scale our image.<jupyter_code># Set Nans to zero
himage_nan_locs = np.isnan(herschel_imagehdu.data)
herschel_data_nonans = herschel_imagehdu.data
herschel_data_nonans[himage_nan_locs] = 0

# Initiate a figure and axis object with WCS projection information
fig = plt.figure(figsize=(18, 12))
ax = fig.add_subplot(111, projection=herschel_wcs)

# Display the moment map image
im = ax.imshow(herschel_data_nonans, cmap='viridis', 
               norm=LogNorm(), vmin=2, vmax=50)
# ax.invert_yaxis() # Flips the Y axis 

# Add axes labels
ax.set_xlabel("Right Ascension", fontsize = 16)
ax.set_ylabel("Declination", fontsize = 16)
ax.grid(color = 'white', ls = 'dotted', lw = 2)

# Add a colorbar
cbar = plt.colorbar(im, pad=.07)
cbar.set_label(''.join(['Herschel 350'r'$\mu$m ','(', herschel_header['BUNIT'], ')']), size = 16)

# Overlay set of Galactic Coordinate Axes
overlay = ax.get_coords_overlay('galactic') 
overlay.grid(color='black', ls='dotted', lw=1)
overlay[0].set_axislabel('Galactic Longitude', fontsize=14)
overlay[1].set_axislabel('Galactic Latitude', fontsize=14)<jupyter_output><empty_output><jupyter_text>Overlay HI 21 cm Contours on the IR 30 micron ImageTo visually compare the neutral gas and dust as traced by HI 21 cm emission and IR 30 micron emission, we can use contours and colorscale images produced using the [WCSAxes](http://docs.astropy.org/en/stable/visualization/wcsaxes/index.html) framework and the `.get_transform()` method. The [WCSAxes.get_transform()](http://docs.astropy.org/en/stable/api/astropy.visualization.wcsaxes.WCSAxes.htmlastropy.visualization.wcsaxes.WCSAxes.get_transform) method returns a transformation from a specified frame to the pixel/data coordinates. It accepts a string specifying the frame or a WCS object.<jupyter_code># Initiate a figure and axis object with WCS projection information
fig = plt.figure(figsize=(18, 12))
ax = fig.add_subplot(111, projection=herschel_wcs)

# Display the moment map image
im = ax.imshow(herschel_data_nonans, cmap='viridis', 
               norm=LogNorm(), vmin=5, vmax=50, alpha=.8)
# ax.invert_yaxis() # Flips the Y axis 

# Add axes labels
ax.set_xlabel("Right Ascension", fontsize=16)
ax.set_ylabel("Declination", fontsize=16)
ax.grid(color = 'white', ls='dotted', lw=2)

# Extract x and y coordinate limits
x_lim = ax.get_xlim()
y_lim = ax.get_ylim()

# Add a colorbar
cbar = plt.colorbar(im, fraction=0.046, pad=-0.1)
cbar.set_label(''.join(['Herschel 350'r'$\mu$m ','(', herschel_header['BUNIT'], ')']), size=16)

# Overlay set of RA/Dec Axes
overlay = ax.get_coords_overlay('galactic')
overlay.grid(color='black', ls='dotted', lw=1)
overlay[0].set_axislabel('Galactic Longitude', fontsize=14)
overlay[1].set_axislabel('Galactic Latitude', fontsize=14)

hi_transform = ax.get_transform(hi_column_density.wcs)  # extract axes Transform information for the HI data

# Overplot column density contours 
levels = (2e21, 3e21, 5e21, 7e21, 8e21, 1e22)  # Define contour levels to use
ax.contour(hi_column_density.hdu.data, cmap='Greys_r', alpha=0.8, levels=levels,
           transform=hi_transform)  # include the transform information with the keyword "transform"

# Overplot velocity image so we can also see the Gas velocities
im_hi = ax.imshow(moment_1.hdu.data, cmap='RdBu_r', vmin=0, vmax=200, alpha=0.5, transform=hi_transform)

# Add a second colorbar for the HI Velocity information
cbar_hi = plt.colorbar(im_hi, orientation='horizontal', fraction=0.046, pad=0.07)
cbar_hi.set_label('HI 'r'$21$cm Mean Velocity (km/s)', size=16)

# Apply original image x and y coordinate limits
ax.set_xlim(x_lim)
ax.set_ylim(y_lim)<jupyter_output><empty_output><jupyter_text>Using reproject to match image resolutionsThe [reproject](https://reproject.readthedocs.io/en/stable/) package is a powerful tool allowing for image data to be transformed into a variety of projections and resolutions.  It's most powerful use is in fact to transform data from one map projection to another without losing any information and still properly conserving flux values within the data. It even has a method to perform a fast reprojection if you are not too concerned with the absolute accuracy of the data values. A simple use of the reproject package is to scale down (or up) resolutions of an image artificially. This could be a useful step if you are trying to get emission line ratios or directly compare the Intensity or Flux from a tracer to that of another tracer in the same physical point of the sky. From our previously made images, we can see that the IR Herschel Image has a higher spatial resolution than that of the HI data cube. We can look into this more by taking a better look at both header objects and using reproject to downscale the Herschel Image.<jupyter_code>print('IR Resolution (dx,dy) = ', herschel_header['cdelt1'], herschel_header['cdelt2'])

print('HI Resolution (dx,dy) = ', hi_column_density.hdu.header['cdelt1'], hi_column_density.hdu.header['cdelt1'])<jupyter_output><empty_output><jupyter_text>Note: Different ways of accessing the header are shown above corresponding to the different object types (coming from SpectralCube vs astropy.io.fits) As we can see, the IR data has over 10 times higher spatial resolution. In order to create a new projection of an image, all we need to specifiy is a new header containing WCS information to transform into.  These can be created manually if you wanted to completely change something about the projection type (i.e. going from a Mercator map projection to a Tangential map projection). For us, since we want to match our resolutions, we can just "steal" the WCS object from the HI data. Specifically, we will be using the [reproject_interp()](https://reproject.readthedocs.io/en/stable/api/reproject.reproject_interp.htmlreproject.reproject_interp) function. This takes two arguments: an HDU object that you want to reproject, and a header containing WCS information to reproject onto.<jupyter_code>rescaled_herschel_data, _ = reproject_interp(herschel_imagehdu, 
                                             # reproject the Herschal image to match the HI data
                                             hi_column_density.hdu.header) 

rescaled_herschel_imagehdu = fits.PrimaryHDU(data = rescaled_herschel_data, 
                                             # wrap up our reprojection as a new fits HDU object
                                             header = hi_column_density.hdu.header)<jupyter_output><empty_output><jupyter_text>`rescaled_herschel_imagehdu` will now behave just like the other FITS images we have been working with, but now with a degraded resolution matching the HI data. This includes having its native coordinates in Galactic rather than RA and Dec.<jupyter_code># Set Nans to zero
image_nan_locs = np.isnan(rescaled_herschel_imagehdu.data)
rescaled_herschel_data_nonans = rescaled_herschel_imagehdu.data
rescaled_herschel_data_nonans[image_nan_locs] = 0

# Initiate a figure and axis object with WCS projection information
fig = plt.figure(figsize = (18,12))
ax = fig.add_subplot(111,projection = WCS(rescaled_herschel_imagehdu))

# Display the moment map image
im = ax.imshow(rescaled_herschel_data_nonans, cmap = 'viridis', 
               norm = LogNorm(), vmin = 5, vmax = 50, alpha = .8)
#im = ax.imshow(rescaled_herschel_imagehdu.data, cmap = 'viridis', 
#               norm = LogNorm(), vmin = 5, vmax = 50, alpha = .8)
ax.invert_yaxis() # Flips the Y axis 

# Add axes labels
ax.set_xlabel("Galactic Longitude", fontsize = 16)
ax.set_ylabel("Galactic Latitude", fontsize = 16)
ax.grid(color = 'white', ls = 'dotted', lw = 2)

# Extract x and y coordinate limits
x_lim = ax.get_xlim()
y_lim = ax.get_ylim()

# Add a colorbar
cbar = plt.colorbar(im, fraction=0.046, pad=-0.1)
cbar.set_label(''.join(['Herschel 350'r'$\mu$m ','(', herschel_header['BUNIT'], ')']), size = 16)

# Overlay set of RA/Dec Axes
overlay = ax.get_coords_overlay('fk5')
overlay.grid(color='black', ls='dotted', lw = 1)
overlay[0].set_axislabel('Right Ascension', fontsize = 14)
overlay[1].set_axislabel('Declination', fontsize = 14)

hi_transform = ax.get_transform(hi_column_density.wcs) # extract axes Transform information for the HI data

# Overplot column density contours 
levels = (2e21, 3e21, 5e21, 7e21, 8e21, 1e22) # Define contour levels to use
ax.contour(hi_column_density.hdu.data, cmap = 'Greys_r', alpha = 0.8, levels = levels,
           transform = hi_transform) # include the transform information with the keyword "transform"

# Overplot velocity image so we can also see the Gas velocities
im_hi = ax.imshow(moment_1.hdu.data, cmap = 'RdBu_r', vmin = 0, vmax = 200, alpha = 0.5, transform = hi_transform)

# Add a second colorbar for the HI Velocity information
cbar_hi = plt.colorbar(im_hi, orientation = 'horizontal', fraction=0.046, pad=0.07)
cbar_hi.set_label('HI 'r'$21$cm Mean Velocity (km/s)', size = 16)

# Apply original image x and y coordinate limits
ax.set_xlim(x_lim)
ax.set_ylim(y_lim)<jupyter_output><empty_output>
StarcoderdataJupyterStructuredCleanDedup