# Loss Functions and Metrics - astroNN.nn.losses¶

astroNN provides modified loss functions which are capable to deal with incomplete labels which are represented by magicnumber in astroNN configuration file or Magic Number in equations below. Since they are built on Tensorflow and follows Keras API requirement, all astroNN loss functions are fully compatible with Keras with Tensorflow backend, as well as directly be imported and used with Tensorflow, for most loss functions, the first argument is ground truth tensor and the second argument is prediction tensor from neural network.

Note

Always make sure when you are normalizing your data, keep the magic number as magic number. If you use astroNN normalizer, astroNN will take care of that.

Here are some explanations on variables in the following loss functions:

$$y_i$$ means the ground truth labels, always represented by python variable y_true in astroNN

$$\hat{y_i}$$ means the prediction from neural network, always represented by python variable y_pred in astroNN

## Correction Term for Magic Number¶

astroNN.nn.magic_correction_term(y_true)

Calculate a correction term to prevent the loss being lowered by magic_num

Parameters: y_true (tf.Tensor) – Ground Truth Correction Term tf.Tensor 2018-Jan-30 - Written - Henry Leung (University of Toronto) 2018-Feb-17 - Updated - Henry Leung (University of Toronto)

Since astroNN deals with magic number by assuming the prediction from neural network for those ground truth with Magic Number is right, so we need a correction term.

The correction term in astroNN is defined by the following equation and we call the equation $$\mathcal{F}_{correction}$$

$\mathcal{F}_{correction} = \frac{\text{Non-Magic Number Count} + \text{Magic Number Count}}{\text{Non Magic Number Count}}$

In case of no labels with Magic Number is presented, $$\mathcal{F}_{correction}$$ will equal to 1

## Mean Squared Error¶

astroNN.nn.losses.mean_squared_error(y_true, y_pred)

Calculate mean square error losses

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Squared Error tf.Tensor 2017-Nov-16 - Written - Henry Leung (University of Toronto)

MSE is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} (\hat{y_i}-y_i)^2 & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_squared_error, ...)


## Mean Absolute Error¶

astroNN.nn.losses.mean_absolute_error(y_true, y_pred)

Calculate mean absolute error, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Absolute Error tf.Tensor 2018-Jan-14 - Written - Henry Leung (University of Toronto)

MAE is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} \left| \hat{y_i}-y_i \right| & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_absolute_error, ...)


## Mean Error¶

astroNN.nn.losses.mean_error(y_true, y_pred)

Calculate mean error as a way to get the bias in prediction, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Error tf.Tensor 2018-May-22 - Written - Henry Leung (University of Toronto)

Mean Error is a metrics to evaluate the bias of prediction and is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} \hat{y_i}-y_i & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_error, ...)


## Regression Loss and Predictive Variance Loss for Bayesian Neural Net¶

astroNN.nn.losses.robust_mse(y_true, y_pred, variance, labels_err)

Calculate predictive variance, and takes account of labels error in Bayesian Neural Network

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction variance (Union(tf.Tensor, tf.Variable)) – Predictive Variance labels_err (Union(tf.Tensor, tf.Variable)) – Known labels error, give zeros if unknown/unavailable Robust Mean Squared Error, can be used directly with Tensorflow tf.Tensor 2018-April-07 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.mse_lin_wrapper(var, labels_err)

Calculate predictive variance, and takes account of labels error in Bayesian Neural Network

Parameters: Returns: var (Union(tf.Tensor, tf.Variable)) – Predictive Variance labels_err (Union(tf.Tensor, tf.Variable)) – Known labels error, give zeros if unknown/unavailable Robust MSE function for labels prediction neurones, which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Prediction Return (tf.Tensor): Robust Mean Squared Error 2017-Nov-16 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.mse_var_wrapper(lin, labels_err)

Calculate predictive variance, and takes account of labels error in Bayesian Neural Network

Parameters: Returns: lin (Union(tf.Tensor, tf.Variable)) – Prediction labels_err (Union(tf.Tensor, tf.Variable)) – Known labels error, give zeros if unknown/unavailable Robust MSE function for predictive variance neurones which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Predictive Variance Return (tf.Tensor): Robust Mean Squared Error 2017-Nov-16 - Written - Henry Leung (University of Toronto)

It is based on the equation implemented as robust_mse(), please notice $$s_i$$ is representing $$log((\sigma_{predictive, i})^2 + (\sigma_{known, i})^2)$$. Neural network not predicting variance directly to avoid numerical instability but predicting $$log((\sigma_{i})^2)$$

$\begin{split}Loss_i = \begin{cases} \begin{split} \frac{1}{2} (\hat{y_i}-y_i)^2 e^{-s_i} + \frac{1}{2}(s_i) & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{BNN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

They basically do the same things and can be used with Keras, you just have to import the functions from astroNN

def keras_model():

# model for the training process
model = Model(inputs=[input_tensor, labels_err_tensor], outputs=[output, variance_output])

# model for the prediction
model_prediction = Model(inputs=input_tensor, outputs=[output, variance_output])

variance_output = Dense(name='variance_output', ...)
output = Dense(name='output', ...)

predictive_variance_loss = mse_var_wrapper(output, labels_err_tensor)
output_loss = mse_lin_wrapper(predictive_variance, labels_err_tensor)

return model, model_prediction, output_loss, predictive_variance_loss

model, model_prediction, output_loss, predictive_variance_loss = keras_model()
# remember to import astroNN loss function first
model.compile(loss={'output': output_loss, 'variance_output': predictive_variance_loss}, ...)


To better understand this loss function, you can see the following plot of Loss vs Variance colored by squared difference which is $$(\hat{y_i}-y_i)^2$$

## Mean Squared Logarithmic Error¶

astroNN.nn.losses.mean_squared_logarithmic_error(y_true, y_pred)

Calculate mean squared logarithmic error, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Squared Logarithmic Error tf.Tensor 2018-Feb-17 - Written - Henry Leung (University of Toronto)

MSLE will first clip the values of prediction from neural net for the sake of numerical stability,

\begin{align}\begin{aligned}\begin{split}y_i = \begin{cases} \begin{split} \epsilon + 1 & \text{ for } y_i < \epsilon \\ y_i + 1 & \text{ for otherwise } \end{split} \end{cases}\end{split}\\\text{where } \epsilon \text{ is a small constant}\end{aligned}\end{align}

Then MSLE is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} (\log{(\hat{y_i})} - \log{(y_i)})^2 & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_squared_logarithmic_error, ...)


## Mean Absolute Percentage Error¶

astroNN.nn.losses.mean_absolute_percentage_error(y_true, y_pred)

Calculate mean absolute percentage error, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Absolute Percentage Error tf.Tensor 2018-Feb-17 - Written - Henry Leung (University of Toronto)

Mean Absolute Percentage Error will first clip the values of prediction from neural net for the sake of numerical stability,

\begin{align}\begin{aligned}\begin{split}y_i = \begin{cases} \begin{split} \epsilon & \text{ for } y_i < \epsilon \\ y_i & \text{ for otherwise } \end{split} \end{cases}\end{split}\\\text{where } \epsilon \text{ is a small constant}\end{aligned}\end{align}

Then Mean Absolute Percentage Error is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} 100 \text{ } \frac{\left| y_i - \hat{y_i} \right|}{y_i} & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_absolute_percentage_error, ...)


## Mean Percentage Error¶

astroNN.nn.losses.mean_percentage_error(y_true, y_pred)

Calculate mean percentage error, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Mean Percentage Error tf.Tensor 2018-Jun-06 - Written - Henry Leung (University of Toronto)

Mean Percentage Error will first clip the values of prediction from neural net for the sake of numerical stability,

\begin{align}\begin{aligned}\begin{split}y_i = \begin{cases} \begin{split} \epsilon & \text{ for } y_i < \epsilon \\ y_i & \text{ for otherwise } \end{split} \end{cases}\end{split}\\\text{where } \epsilon \text{ is a small constant}\end{aligned}\end{align}

Then Mean Percentage Error is based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} 100 \text{ } \frac{y_i - \hat{y_i}}{y_i} & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=mean_percentage_error, ...)


## Categorical Cross-Entropy¶

astroNN.nn.losses.categorical_crossentropy(y_true, y_pred, from_logits=False)

Categorical cross-entropy between an output tensor and a target tensor, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction from_logits (boolean) – From logits space or not. If you want to use logits, please use from_logits=True Categorical Cross-Entropy tf.Tensor 2018-Jan-14 - Written - Henry Leung (University of Toronto)

Categorical Cross-Entropy will first clip the values of prediction from neural net for the sake of numerical stability if the prediction is not coming from logits (before softmax activated)

\begin{align}\begin{aligned}\begin{split}\hat{y_i} = \begin{cases} \begin{split} \epsilon & \text{ for } \hat{y_i} < \epsilon \\ 1 - \epsilon & \text{ for } \hat{y_i} > 1 - \epsilon \\ \hat{y_i} & \text{ for otherwise } \end{split} \end{cases}\end{split}\\\text{where } \epsilon \text{ is a small constant}\end{aligned}\end{align}

and then based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} y_i \log{(\hat{y_i})} & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = - \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=categorical_crossentropy(from_logits=False), ...)


## Binary Cross-Entropy¶

astroNN.nn.losses.binary_crossentropy(y_true, y_pred, from_logits=False)

Binary cross-entropy between an output tensor and a target tensor, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction from_logits (boolean) – From logits space or not. If you want to use logits, please use from_logits=True Binary Cross-Entropy tf.Tensor 2018-Jan-14 - Written - Henry Leung (University of Toronto)

Binary Cross-Entropy will first clip the values of prediction from neural net for the sake of numerical stability if from_logits=False

\begin{align}\begin{aligned}\begin{split}\hat{y_i} = \begin{cases} \begin{split} \epsilon & \text{ for } \hat{y_i} < \epsilon \\ 1 - \epsilon & \text{ for } \hat{y_i} > 1 - \epsilon \\ \hat{y_i} & \text{ for otherwise } \end{split} \end{cases}\end{split}\\\text{where } \epsilon \text{ is a small constant}\end{aligned}\end{align}

and then based on the equation

$\begin{split}Loss_i = \begin{cases} \begin{split} y_i \log{(\hat{y_i})} + (1-y_i)\log{(1-\hat{y_i})} & \text{ for } y_i \neq \text{Magic Number}\\ \hat{y_i} \log{(\hat{y_i})} + (1-\hat{y_i})\log{(1-\hat{y_i})} & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

to avoid numerical instability if from_logits=True, we can reformulate it as

$\begin{split}Loss_i = \begin{cases} \begin{split} \max{(\hat{y_i}, 0)} - y_i \hat{y_i} + \log{(1+e^{-\|\hat{y_i}\|})} & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

And thus the loss for mini-batch is

$Loss_{NN} = - \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's loss function first
model.compile(loss=binary_crossentropy(from_logits=False), ...)


## Categorical Cross-Entropy and Predictive Logits Variance for Bayesian Neural Net¶

astroNN.nn.losses.robust_categorical_crossentropy(y_true, y_pred, logit_var)

Calculate categorical accuracy, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction in logits space logit_var (Union(tf.Tensor, tf.Variable)) – Predictive variance in logits space categorical cross-entropy tf.Tensor 2018-Mar-15 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.bayesian_categorical_crossentropy_wrapper(logit_var)
Categorical crossentropy between an output tensor and a target tensor for Bayesian Neural Network
equation (12) of arxiv:1703.04977
Parameters: Returns: logit_var (Union(tf.Tensor, tf.Variable)) – Predictive variance Robust categorical_crossentropy function for predictive variance neurones which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Prediction in logits space Return (tf.Tensor): Robust categorical crossentropy 2018-Mar-15 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.bayesian_categorical_crossentropy_var_wrapper(logits)
Categorical crossentropy between an output tensor and a target tensor for Bayesian Neural Network
equation (12) of arxiv:1703.04977
Parameters: Returns: logits (Union(tf.Tensor, tf.Variable)) – Prediction in logits space Robust categorical_crossentropy function for predictive variance neurones which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Predictive variance in logits space Return (tf.Tensor): Robust categorical crossentropy 2018-Mar-15 - Written - Henry Leung (University of Toronto)

It is based on Equation 12 from arxiv:1703.04977. $$s_i$$ is representing the predictive variance of logits

$\begin{split}Loss_i = \begin{cases} \begin{split} \text{Categorical Cross-Entropy} + \text{Distorted Categorical Cross-Entropy} + e^{s_i} - 1 & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

where Distorted Categorical Cross-Entropy is defined as

$\text{elu} (\text{Categorical Cross-Entropy}(y_i, \hat{y_i}) - \text{Categorical Cross-Entropy}(y_i, \mathcal{N}(\hat{y_i}, \sqrt{s_i})))$

And thus the loss for mini-batch is

$Loss_{BNN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

bayesian_categorical_crossentropy_wrapper is for the prediction neurones

bayesian_categorical_crossentropy_var_wrapper is for the predictive variance neurones

They basically do the same things and can be used with Keras, you just have to import the functions from astroNN

def keras_model():

# model for the training process
model = Model(inputs=[input_tensor], outputs=[output, variance_output])

# model for the prediction
model_prediction = Model(inputs=input_tensor, outputs=[output, variance_output])

variance_output = Dense(name='predictive_variance', ...)
output = Dense(name='output', ...)

predictive_variance_loss = bayesian_categorical_crossentropy_var_wrapper(output)
output_loss = bayesian_categorical_crossentropy_wrapper(predictive_variance)

return model, model_prediction, output_loss, predictive_variance_loss

model, model_prediction, output_loss, predictive_variance_loss = keras_model()
# remember to import astroNN loss function first
model.compile(loss={'output': output_loss, 'variance_output': predictive_variance_loss}, ...)


## Binary Cross-Entropy and Predictive Logits Variance for Bayesian Neural Net¶

astroNN.nn.losses.robust_binary_crossentropy(y_true, y_pred, logit_var)

Calculate binary accuracy, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction in logits space logit_var (Union(tf.Tensor, tf.Variable)) – Predictive variance in logits space categorical cross-entropy tf.Tensor 2018-Mar-15 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.bayesian_binary_crossentropy_wrapper(logit_var)
Binary crossentropy between an output tensor and a target tensor for Bayesian Neural Network
equation (12) of arxiv:1703.04977
Parameters: Returns: logit_var (Union(tf.Tensor, tf.Variable)) – Predictive variance Robust binary_crossentropy function for predictive variance neurones which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Prediction in logits space Return (tf.Tensor): Robust binary crossentropy 2018-Mar-15 - Written - Henry Leung (University of Toronto)
astroNN.nn.losses.bayesian_binary_crossentropy_var_wrapper(logits)
Binary crossentropy between an output tensor and a target tensor for Bayesian Neural Network
equation (12) of arxiv:1703.04977
Parameters: Returns: logits (Union(tf.Tensor, tf.Variable)) – Prediction in logits space Robust binary_crossentropy function for predictive variance neurones which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Predictive variance in logits space Return (tf.Tensor): Robust binary crossentropy 2018-Mar-15 - Written - Henry Leung (University of Toronto)

It is based on Equation 12 from arxiv:1703.04977. $$s_i$$ is representing the predictive variance of logits

$\begin{split}Loss_i = \begin{cases} \begin{split} \text{Binary Cross-Entropy} + \text{Distorted Binary Cross-Entropy} + e^{s_i} - 1 & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

where Distorted Binary Cross-Entropy is defined as

$\text{elu} (\text{Binary Cross-Entropy}(y_i, \hat{y_i}) - \text{Binary Cross-Entropy}(y_i, \mathcal{N}(\hat{y_i}, \sqrt{s_i})))$

And thus the loss for mini-batch is

$Loss_{BNN} = \frac{1}{D} \sum_{i=1}^{batch} (Loss_i \mathcal{F}_{correction, i})$

bayesian_binary_crossentropy_wrapper is for the prediction neurones

bayesian_binary_crossentropy_var_wrapper is for the predictive variance neurones

They basically do the same things and can be used with Keras, you just have to import the functions from astroNN

def keras_model():

# model for the training process
model = Model(inputs=[input_tensor], outputs=[output, variance_output])

# model for the prediction
model_prediction = Model(inputs=input_tensor, outputs=[output, variance_output])

variance_output = Dense(name='predictive_variance', ...)
output = Dense(name='output', ...)

predictive_variance_loss = bayesian_binary_crossentropy_var_wrapper(output)
output_loss = bayesian_binary_crossentropy_wrapper(predictive_variance)

return model, model_prediction, output_loss, predictive_variance_loss

model, model_prediction, output_loss, predictive_variance_loss = keras_model()
# remember to import astroNN loss function first
model.compile(loss={'output': output_loss, 'variance_output': predictive_variance_loss}, ...)


## Categorical Classification Accuracy¶

astroNN.nn.losses.categorical_accuracy(y_true, y_pred)

Calculate categorical accuracy, ignoring the magic number

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Categorical Classification Accuracy tf.Tensor 2018-Jan-21 - Written - Henry Leung (University of Toronto)

Categorical Classification Accuracy will first deal with Magic Number

$\begin{split}Loss_i = \begin{cases} \begin{split} y_i & \text{ for } y_i \neq \text{Magic Number}\\ 0 & \text{ for } y_i = \text{Magic Number} \end{split} \end{cases}\end{split}$

Then based on the equation

$\begin{split}Accuracy_i = \begin{cases} \begin{split} 1 & \text{ for } \text{Argmax}(y_i) = \text{Argmax}(\hat{y_i})\\ 0 & \text{ for } \text{Argmax}(y_i) \neq \text{Argmax}(\hat{y_i}) \end{split} \end{cases}\end{split}$

And thus the accuracy for is

$Accuracy = \frac{1}{D} \sum_{i=1}^{labels} (Accuracy_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's metrics function first
model.compile(metrics=categorical_accuracy, ...)


Note

Please make sure you use categorical_accuracy when using categorical_crossentropy as the loss function

## Binary Classification Accuracy¶

astroNN.nn.losses.binary_accuracy(from_logits=False)

Calculate binary accuracy, ignoring the magic number

Parameters: Returns: from_logits (boolean) – From logits space or not. If you want to use logits, please use from_logits=True Function for Binary classification accuracy which matches Keras losses API function function(y_true, y_pred) - y_true (tf.Tensor): Ground Truth - y_pred (tf.Tensor): Prediction Return (tf.Tensor): Binary Classification Accuracy 2018-Jan-31 - Written - Henry Leung (University of Toronto)

Binary Classification Accuracy will round the values of prediction if from_logits=False or will apply sigmoid first and then round the values of prediction if from_logits=True

$\begin{split}\hat{y_i} = \begin{cases} \begin{split} 1 & \text{ for } \hat{y_i} > 0.5 \\ 0 & \text{ for } \hat{y_i} \leq 0.5 \end{split} \end{cases}\end{split}$

and then based on the equation

$\begin{split}Accuracy_i = \begin{cases} \begin{split} 1 & \text{ for } y_i = \hat{y_i}\\ 0 & \text{ for } y_i \neq \hat{y_i} \end{split} \end{cases}\end{split}$

And thus the accuracy for is

$Accuracy = \frac{1}{D} \sum_{i=1}^{labels} (Accuracy_i \mathcal{F}_{correction, i})$

It can be used with Keras, you just have to import the function from astroNN

def keras_model():
return model

model = keras_model()
# remember to import astroNN's metrics function first
model.compile(metrics=binary_accuracy(from_logits=False), ...)


Note

Please make sure you use binary_accuracy when using binary_crossentropy as the loss function

## Zeros Loss¶

astroNN.nn.losses.zeros_loss(y_true, y_pred)

Always return zeros

Parameters: y_true (Union(tf.Tensor, tf.Variable)) – Ground Truth y_pred (Union(tf.Tensor, tf.Variable)) – Prediction Zeros tf.Tensor 2018-May-24 - Written - Henry Leung (University of Toronto)

zeros_loss is a loss function that will always return zero loss and the function matches Keras API. It is mainly designed to do testing or experiments.

It can be used with Keras, you just have to import the function from astroNN

def keras_model():