Mini Tools for APOGEE data - astroNN.apogee

Note

astroNN only contains a limited amount of necessary tools. For a more comprehensive python tool to deal with APOGEE data, please refer to Jo Bovy’s APOGEE tools

The APO Galactic Evolution Experiment 1 (APOGEE-1) employed high-resolution, high signal-to-noise infrared spectroscopy to penetrate the dust that obscures significant fractions of the disk and bulge of our Galaxy. APOGEE surveyed over 100,000 red giant stars across the full range of the Galactic bulge, bar, disk, and halo. APOGEE-1 generated precise radial velocities and detailed chemical abundances, providing unprecedented insights into the dynamical structure and chemical history of the Galaxy. In conjunction with the planet-finding surveys, Kepler and CoRoT, APOGEE unravels problems in fundamental astrophysics.

SDSS APOGEE: http://www.sdss.org/surveys/apogee/

Continuum Normalization of APOGEE Spectra

You can access the default astroNN continuum mask fro APOGEE spectra by

import os
import astroNN
import numpy as np

dr = 14

dir = os.path.join(os.path.dirname(astroNN.__path__[0]), 'astroNN', 'data', f'dr{dr}_contmask.npy')
cont_mask = np.load(dir)

When you do continuum normalization using astroNN, you can just use con_mask=None to use default mask provided by Jo Bovy’s APOGEE Tools. astroNN will use a SINGLE continuum pixel mask to normalize all spectra you provided. Moreover, astroNN will normalize the spectra by chips instead of normalize them all together.

astroNN.apogee.apogee_continuum(spectra, spectra_err, cont_mask=None, deg=2, dr=None, bitmask=None, target_bit=None, mask_value=1.0)

It is designed only for apogee spectra by fitting Chebyshev polynomials to the flux values in the continuum mask by chips. The resulting continuum will have the same shape as fluxes.

Parameters:

• spectra (ndarray) – spectra
• spectra_err (ndarray) – spectra uncertainty, same shape as spectra
• cont_mask (ndarray[bool]) – continuum mask
• deg (int) – The degree of Chebyshev polynomial to use in each region, default is 2 which works the best so far
• dr (int) – apogee dr
• bitmask (ndarray) – bitmask array of the spectra, same shape as spectra
• target_bit (Union(int, list[int], ndarray[int])) – a list of bit to be masked
• mask_value (Union(int, float)) – if a pixel is determined to be a bad pixel, this value will be used to replace that pixel flux

Returns:

normalized spectra, normalized spectra uncertainty

Return type:

ndarray, ndarray

History:

2018-Mar-21 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import apogee_continuum

# spectra_errs refers to the 1-sigma error array provided by APOGEE
# spectra can be multiple spectra at a time
norm_spec, norm_spec_err = apogee_continuum(apogee_spectra, spectra_errs, cont_mask=None, deg=2, dr=14)

# If you deal with bitmask too and want to set some target bits to zero, you can add additional arguement in apogee_continuum()
# You target_bit=[a list of number] or target_bit=None to use default target_bit
apogee_continuum(apogee_spectra, spectra_errs, cont_mask=None, deg=2, dr=14, bitmask=apogee_bitmask, target_bit=None)

norm_spec refers to the normalized spectra while norm_spec_err refers to the normalized spectra error

Note

If you are planning to compile APOGEE dataset using astroNN, you can ignore this section as astroNN H5Compiler will load data from fits files directly and will take care everything.

You can use continuum() to normalize any spectra while apogee_continuum() is specifically designed for APOGEE spectra.

from astroNN.apogee import continuum

spec, spec_err = continuum(spectra, spectra_errs, cont_mask, deg=2)

Basics Tools related to APOGEE Spectra

Here are some basic tools to deal with APOGEE spectra

Retrieve Basic APOGEE Spectra Pixel Information

You can retrieve basic APOGEE spectra pixel information by

astroNN.apogee.chips_pix_info(dr=None)

To return chips info according to dr

Parameters:dr (Union(int, NoneType)) – data release Returns:

The starting and ending pixels location of APOGEE camera chips in the original 8575 pixels spectra

- list[0] refers to the location where blue chips starts

- list[1] refers to the location where blue chips ends

- list[2] refers to the location where green chips starts

- list[3] refers to the location where blue chips end

- list[4] refers to the location where red chips starts

- list[5] refers to the location where red chips ends

- list[6] refers to the total number of pixels after deleting gap

Return type:list History:

2017-Nov-27 - Written - Henry Leung (University of Toronto)

2017-Dec-16 - Updated - Henry Leung (University of Toronto)

from astroNN.apogee import chips_pix_info

info = chips_pix_info(dr=14)

# info[0] refers to the location where blue chips starts
# info[1] refers to the location where blue chips ends
# info[2] refers to the location where green chips starts
# info[3] refers to the location where blue chips end
# info[4] refers to the location where red chips starts
# info[5] refers to the location where red chips ends
# info[6] refers to the total number of pixels after deleting gap

APOGEE Spectra Wavelength Solution

astroNN.apogee.wavelength_solution(dr=None)

To return wavelegnth_solution, apStarWavegrid was provided by Jo Bovy’s apogee tools (Toronto)

Parameters:dr (Union(int, NoneType)) – data release Returns:

lambda_blue, lambda_green, lambda_red which are 3 wavelength solution array

- lambda_blue refers to the wavelength solution for each pixel in blue chips

- lambda_green refers to the wavelength solution for each pixel in green chips

- lambda_red refers to the wavelength solution for each pixel in red chips

Return type:ndarray History:

2017-Nov-20 - Written - Henry Leung (University of Toronto)

2017-Dec-16 - Updated - Henry Leung (University of Toronto)

You can retrieve APOGEE spectra wavelength solution by

from astroNN.apogee import wavelength_solution

lambda_blue, lambda_green, lambda_red = wavelength_solution(dr=14)

# lambda_blue refers to the wavelength solution for each pixel in blue chips
# lambda_green refers to the wavelength solution for each pixel in green chips
# lambda_red refers to the wavelength solution for each pixel in red chips

APOGEE Spectra Gap Delete

astroNN.apogee.gap_delete(spectra, dr=None)

To delete the gap between APOGEE CCDs from the original 8575 pixels spectra

Parameters:

• spectra (ndarray) – The original 8575 pixels spectrum/spectra
• dr (Union(int, NoneType)) – data release

Returns:

Gap deleted spectrum/spectra

Return type:

ndarray

History:

2017-Oct-26 - Written - Henry Leung (University of Toronto)

2017-Dec-16 - Updated - Henry Leung (University of Toronto)

You can delete the gap between raw spectra by

from astroNN.apogee import gap_delete

# original_spectra can be multiple spectra at a time
gap_deleted_spectra = gap_delete(original_spectra, dr=14)

Split APOGEE Spectra into Three Detectors

astroNN.apogee.chips_split(spectra, dr=None)

To split APOGEE spectra into RGB chips, will delete the gap if detected

Parameters:

• spectra (ndarray) – APOGEE spectrum/spectra
• dr (Union(int, NoneType)) – data release

Returns:

3 ndarrays which are spectra_blue, spectra_green, spectra_red

Return type:

ndarray

History:

2017-Nov-20 - Written - Henry Leung (University of Toronto)

2017-Dec-17 - Updated - Henry Leung (University of Toronto)

You can split APOGEE spectra into three detectors by

from astroNN.apogee import chips_split

# original_spectra can be multiple spectra at a time
spectra_blue, spectra_green, spectra_red = chips_split(original_spectra, dr=14)

chips_split() will delete the gap between the detectors if you give raw APOGEE spectra. If you give gap deleted spectra, then the function will simply split the spectra into three.

APOGEE Bitmask to Boolean Array

You can turn a APOGEE PIXMASK bitmask array into a boolean array provided you have some target bit you want to mask

Bitmask: http://www.sdss.org/dr14/algorithms/bitmasks/#collapseAPOGEE_PIXMASK

astroNN.apogee.bitmask_boolean(bitmask, target_bit)

Turn bitmask to boolean with provided bitmask array and target bit to mask

Parameters:

• bitmask (ndarray) – bitmask
• target_bit (list[int]) – target bit to mask

Returns:

boolean array, True for clean, False for masked

Return type:

ndarray[bool]

History:

2018-Feb-03 - Written - Henry Leung (University of Toronto)

Example:

from astroNN.apogee import bitmask_boolean
import numpy as np

spectra_bitmask = np.array([2048, 128, 1024, 512, 16, 8192, 4096, 64, 2, 32, 256, 8, 4, 16896])
boolean_output = bitmask_boolean(spectra_bitmask, target_bit=[0,1,2,3,4,5,6,7,9,12])
print(boolean_output)
>>> array([[False, True, False, True, True, False, True, True, True, True, False, True, True, True]])

Decompose APOGEE Bitmask into Constitute Bits

You can turn a APOGEE PIXMASK bit into its constitute bits

Bitmask: http://www.sdss.org/dr14/algorithms/bitmasks/#collapseAPOGEE_PIXMASK

astroNN.apogee.bitmask_decompositor(bit)

To decompose a bit from bitmask array to individual bit

Parameters:bit (int) – bitmask Returns:boolean array, True for clean, False for masked Return type:ndarray[bool] History:2018-Feb-03 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import bitmask_decompositor

decomposed_bits = bitmask_decompositor(single_bitmask)

Example:

from astroNN.apogee import bitmask_decompositor

# Create a simulated bit number
# Lets say this pixel is marked as 0, 5, 13 and 14 bit
bitmask = 2**0 + 2**5 + 2**13 + 2**14

decomposed_bits = bitmask_decompositor(bitmask)
# The function returns the set of original bits
>>> array([ 0,  5, 13, 14])

Retrieve ASPCAP Elements Window Mask

Original ASPCAP Elements Windows Mask: https://svn.sdss.org/public/repo/apogee/idlwrap/trunk/lib/ which is described in https://arxiv.org/abs/1502.04080

You can get ASPCAP elements window mask as a boolean array by providing an element name to this function,

astroNN.apogee.aspcap_mask(elem, dr=None)

To load ASPCAP elements window masks

DR14 Elements: 'C', 'CI', 'N', 'O', 'Na', 'Mg', 'Al', 'Si', 'P', 'S', 'K', 'Ca', 'TI', 'TiII', 'V', 'Cr', 'Mn', 'Fe', 'Co', 'Ni', 'Cu', 'Ge', 'Ce', 'Rb', 'Y', 'Nd'

Parameters:

• elem (str) – element name
• dr (int) – apogee dr

Returns:

mask

Return type:

ndarray[bool]

History:

2018-Mar-24 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import aspcap_mask

mask = aspcap_mask('Mg')  # for example you want to get ASPCAP Mg mask

APOGEE Data Downloader

astroNN APOGEE data downloader always act as functions that will return you the path of downloaded file(s), and download it if it does not exist locally. If the file cannot be found on server, astroNN will generally return False as the path.

General Way to Open Fits File

astropy.io.fits documentation: http://docs.astropy.org/en/stable/io/fits/

from astropy.io import fits

data = fits.open(local_path_to_file)

allstar file

Data Model: https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/allStar.html

astroNN.apogee.allstar(dr=None, flag=None)

Download the allStar file (catalog of ASPCAP stellar parameters and abundances from combined spectra)

Parameters:

• dr (int) – APOGEE DR
• flag (int) – 0: normal, 1: force to re-download

Returns:

full file path and download in background if not found locally, False if cannot be found on server

Return type:

str

History:

2017-Oct-09 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import allstar

local_path_to_file = allstar(dr=14)

allvisit file

Data Model: https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/allVisit.html

astroNN.apogee.allvisit(dr=None, flag=None)

Download the allVisit file (catalog of properties from individual visit spectra)

Parameters:

• dr (int) – APOGEE DR
• flag (int) – 0: normal, 1: force to re-download

Returns:

full file path and download in background if not found locally, False if cannot be found on server

Return type:

str

History:

2017-Oct-11 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import allvisit

local_path_to_file = allvisit(dr=14)

Combined Spectra (aspcapStar)

Data Model: https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/LOCATION_ID/aspcapStar.html

astroNN.apogee.combined_spectra(dr=None, location=None, apogee=None, telescope=None, verbose=1, flag=None)

Download the required combined spectra file a.k.a aspcapStar

Parameters:

• dr (int) – APOGEE DR
• location (int) – Location ID [Optional]
• apogee (str) – Apogee ID
• telescope (str) – Telescope ID, for example ‘apo25m’ or ‘lco25m’
• flag (int) – 0: normal, 1: force to re-download

Returns:

full file path and download in background if not found locally, False if cannot be found on server

Return type:

str

History:

2017-Oct-15 - Written - Henry Leung (University of Toronto)

2018-Aug-31 - Updated - Henry Leung (University of Toronto)

from astroNN.apogee import combined_spectra

local_path_to_file = combined_spectra(dr=14, location=a_location_id, apogee=a_apogee_id)

Visit Spectra (apStar)

Data Model: https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/TELESCOPE/LOCATION_ID/apStar.html

astroNN.apogee.visit_spectra(dr=None, location=None, apogee=None, telescope=None, verbose=1, flag=None, commission=False)

Download the required individual spectra file a.k.a apStar

Parameters:

• dr (int) – APOGEE DR
• location (int) – Location ID [Optional]
• apogee (str) – Apogee ID
• telescope (str) – Telescope ID, for example ‘apo25m’ or ‘lco25m’
• verbose (int) – verbose
• flag (int) – 0: normal, 1: force to re-download
• commission (bool) – whether the spectra is taken during commissioning

Returns:

full file path and download in background if not found locally, False if cannot be found on server

Return type:

str

History:

2017-Nov-11 - Written - Henry Leung (University of Toronto)

2018-Aug-31 - Updated - Henry Leung (University of Toronto)

from astroNN.apogee import visit_spectra

local_path_to_file = visit_spectra(dr=14, location=a_location_id, apogee=a_apogee_id)

Red Clumps of SDSS Value Added Catalogs

Introduction: http://www.sdss.org/dr14/data_access/value-added-catalogs/?vac_id=apogee-red-clump-rc-catalog

Data Model (DR14): https://data.sdss.org/datamodel/files/APOGEE_RC/cat/apogee-rc-DR14.html

astroNN.datasets.apogee_rc.load_apogee_rc(dr=None, metric='distance', extinction=True)

Load apogee red clumps (absolute magnitude measurement)

Parameters:

• dr (int) – Apogee DR
• metric (string) –

  which metric you want to get back

  • ”absmag” for k-band absolute magnitude
  • ”fakemag” for k-band fake magnitude
  • ”distance” for distance in parsec
• extinction (bool) – Whether to take extinction into account, only affect when metric is NOT ‘distance’

Returns:

numpy array of ra, dec, metrics_array

Return type:

ndarrays

History:

2018-Jan-21 - Written - Henry Leung (University of Toronto)

2018-May-12 - Updated - Henry Leung (University of Toronto)

from astroNN.apogee import apogee_vac_rc

local_path_to_file = apogee_vac_rc(dr=14)

Or you can use load_apogee_rc() to load the data by

from astroNN.datasets import load_apogee_rc

# metric can be 'distance' for distance in parsec, 'absmag' for k-band absolute magnitude
# 'fakemag' for astroNN's k-band fakemag scale
RA, DEC, metrics_array = load_apogee_rc(dr=14, metric='distance', extinction=True)  # extinction only effective if not metric='distance'

APOKASC in the Kepler Fields

from astroNN.datasets.apokasc import apokasc_load

ra, dec, logg = apokasc_load()

# OR you want the gold and basic standard separately
gold_ra, gold_dec, gold_logg, basic_ra, basic_dec, basic_logg = apokasc_load(combine=False)

APOGEE DR14-Based Distance Estimations

Introduction: http://www.sdss.org/dr14/data_access/value-added-catalogs/?vac_id=apogee-dr14-based-distance-estimations

Data Model (DR14): https://data.sdss.org/datamodel/files/APOGEE_DISTANCES/apogee_distances.html

astroNN.apogee.apogee_distances(dr=None, flag=None)

Download the Apogee Distances catalogue

Parameters:

• dr (int) – Apogee DR
• flag (int) – Force to download if flag=1

Returns:

full file path

Return type:

str

History:

2018-Jan-24 - Written - Henry Leung (University of Toronto)

from astroNN.apogee.downloader import apogee_distances

local_path_to_file = apogee_distances(dr=14)

astroNN.datasets.load_apogee_distances(dr=None, metric='distance', cuts=True, extinction=True, keepdims=False)

Load apogee distances (absolute magnitude from stellar model)

Parameters:

• dr (int) – Apogee DR
• metric (string) –

  which metric you want to get back

  • ”absmag” for absolute magnitude
  • ”fakemag” for fake magnitude
  • ”distance” for distance in parsec
• cuts (Union[boolean, float]) – Whether to cut bad data (negative parallax and percentage error more than 20%), or a float to set the threshold
• extinction (bool) – Whether to take extinction into account, only affect when metric is NOT ‘distance’
• keepdims (boolean) – Whether to preserve indices the same as APOGEE allstar DR14, no effect when cuts=False, set to -9999 for bad indices when cuts=True keepdims=True

Returns:

numpy array of ra, dec, metrics_array, metrics_err_array

Return type:

ndarrays

History:

2018-Jan-25 - Written - Henry Leung (University of Toronto)

Or you can use load_apogee_distances() to load the data by

from astroNN.datasets import load_apogee_distances

# metric can be 'distance' for distance in parsec, 'absmag' for k-band absolute magnitude
# 'fakemag' for astroNN's k-band fakemag scale
# cuts=True to cut out those unknown values (-9999.) and measurement error > 20%
RA, DEC, metrics_array, metrics_err_array = load_apogee_distances(dr=14, metric='distance', cuts=True, keepdims=False)

Cannon’s allstar

Introduction: https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/CANNON_VERS/cannonModel.html

Data Model (DR14): https://data.sdss.org/datamodel/files/APOGEE_REDUX/APRED_VERS/APSTAR_VERS/ASPCAP_VERS/RESULTS_VERS/CANNON_VERS/allStarCannon.html

astroNN.apogee.allstarcannon(dr=None, flag=None)

Download the allStarCannon file (catalog of Cannon stellar parameters and abundances from combined spectra)

Parameters:

• dr (int) – APOGEE DR
• flag (int) – 0: normal, 1: force to re-download

Returns:

full file path and download in background if not found locally, False if cannot be found on server

Return type:

str

History:

2017-Oct-24 - Written - Henry Leung (University of Toronto)

from astroNN.apogee import allstarcannon

local_path_to_file = allstarcannon(dr=14)