DISSERTATION DEFENSE

Author : Fawad Kirmani
Advisors: Dr. John Rose
Date: October 22, 2026
Time: 10:00 am
Location: Virtual
Link: https://teams.microsoft.com/l/meetup-join/19%3ameeting_NzI3MDUwODctMzVi…

Abstract

In deep learning classification tasks, class imbalance is a frequent challenge, because a model trained on imbalanced data tends to favor the majority class and miss the rare cases that are often of greatest interest. In this work, we introduce our custom loss function, which adds a symmetric Kullback--Leibler divergence term to categorical cross-entropy. It is designed to increase recall of the minority class while limiting the loss of recall in the majority class. We evaluated our custom loss function against categorical cross-entropy and focal loss using the first two classes of the CIFAR-10 dataset. Over 30 seeds and 12 class imbalance ratios, it increased the recall of the minority class most under severe imbalance, by 5.15 percentage points at the most imbalanced ratio, while keeping the decrease in the recall of the majority class below 2 percentage points at every ratio, at the cost of a lower ROC-AUC. In addition, we also evaluated our custom loss function on the German Credit dataset. These additional results are presented in an appendix. Having demonstrated the utility of our custom loss function, we applied it to two binary classification tasks in proteomics and astronomy. We conclude with a multiclass task in astronomy that does not use it.

First, we developed a bidirectional long short-term memory (BiLSTM) autoencoder for identifying proteotypic peptides. The model combines peptide embeddings with physicochemical peptide features and is trained with our custom loss function. Under 5x2 cross-validation, it achieved a mean ROC-AUC of 95.08%, sensitivity of 92.39%, and specificity of 77.55%. Relative to categorical cross-entropy, our custom loss function significantly reduced the misclassification rate, and an ensemble with an XGBoost model reduced it further.

Second, we developed a deep convolutional neural network to detect polar ring galaxies in Sloan Digital Sky Survey images, trained with our custom loss function. Across ten folds, the model achieved a mean test ROC-AUC of approximately 99% and a mean recall of 96% for the polar ring galaxy class, and it maintained strong performance on a heavily imbalanced subset of 20 polar ring galaxies and 1,000 other galaxies.

Third, we developed a deep learning model to detect gravitationally lensed supernovae (SNe), which are extremely rare and fade quickly. Instead of relying on magnification, our model identifies doubly-imaged SNe with small separations (<0.6 arcsec) from the shape of the transient in a single difference image, so it can give an early alert before a light curve is available. We trained a VGG19 network with a Convolutional Block Attention Module on simulated SNe planted into archival Hubble Space Telescope images. Across ten test folds, the model achieved a mean ROC-AUC of 99.29% and mean recall scores of 96.6%, 91.0%, and 97.1% for images with zero, one, and two simulated transients. Unlike the Source-Extractor baseline, which could not identify single transients, it separated single from doubly-imaged SNe. Further work is needed to assess its performance on real survey data.

Together, these studies show that our custom loss function can improve minority-class performance in imbalanced binary classification while making the trade-offs explicit, and they demonstrate deep learning applications in proteomics and astronomy, where identifying rare objects or events is an important scientific problem.