Towards Intrinsically Motivated Reinforcement Learning - A Video Prefetching Example

Monday, October 19, 2026 - 03:15 pm
Room 2267, Storey Innovation building

DISSERTATION DEFENSE

Author :Nawras Alkassab
Advisors: Dr. Chin-Tser Huang
Date: October 19, 2026
Time: 03:15 pm
Location: Room 2267, Storey Innovation building

Abstract
In psychology, humans who are intrinsically motivated tend to outperform those whose self-validation is more dependent on extrinsic rewards. In reinforcement learning, curiosity-driven agents suffer from the noisy TV problem, since calculating the error in predicting the future observations leaves the agents perplexed in extremely randomized environments. In this work, we tackle the video prefetching problem at edge networks, an NP-Hard problem, using intrinsically motivated reinforcement learning agents. The extrinsic rewards in the video prefetching problem are sparse and delayed, since they are received from access networks by edge networks infrequently. First, we explore the benefits and schemes of video prefetching from cloud networks to edge networks. Next, we explore the design and implementation of deep reinforcement learning for video prefetching at edge networks. By formulating the prefetching problem at edge networks as Partially Observable Markov Decision Process, we propose an intrinsically motivated reinforcement learning agent, Techie, to maximize both prefetching accuracy and prefetching coverage. Techie self-tunes its aggressiveness to manage the trade-offs between prefetching accuracy and prefetching coverage, given a long-term trajectory of video requests. Finally, we conclude that the prefetching problem at edge networks is susceptible to large action space and observation space, which exponentially increases as the size of the edge network's storage increases linearly. In addition, we believe that intrinsically motivated reinforcement learning agents for video prefetching at edge networks offer multiple benefits such as their sensitivity to the popularity of video items issued by end-users at different heterogeneous access networks and their ability to make intelligent prefetching decisions online without relying on access networks' metrics such as Quality of Experience metrics and bandwidth wastage.
 

Beyond Linear Thought in LLM Reasoning: Advancing Process Reward Models via Topological and Neuro-Symbolic Verification

Tuesday, October 20, 2026 - 09:00 am
AI Seminar Room 529

DISSERTATION DEFENSE

Author : Yuxin Zi
Advisors: Dr. Amit Sheth
Date: October 20, 2026
Time: 09:00 am
Location: AI Seminar Room 529
Link:  https://sc-edu.zoom.us/j/84342587325
Meeting ID: 843 4258 7325

Abstract

While Large Language Models (LLMs) possess a remarkable capacity to generate fluent, step-by-step explanations, their underlying reasoning processes remain epistemologically fragile. The prevailing paradigm of Chain-of-Thought (CoT) prompting forces formal, multi-dimensional logic into a one-dimensional textual sequence. This fundamental misalignment between the topology of deductive reasoning and the linearity of autoregressive generation conceals causal dependencies, leading to hallucination snowballs, context dilution, and catastrophic error cascading. Furthermore, current oversight mechanisms, such as linear Process Reward Models (PRMs), evaluate reasoning steps based on chronologically flat histories and symmetric distance metrics, thereby conflating mere topical relevance with rigorous logical entailment.

This dissertation introduces a paradigm shift in how artificial reasoning is represented, verified, grounded, and repaired. It argues that robust reasoning requires the explicit scaffolding of a machine-checkable Directed Acyclic Graph (DAG), rigorously governed by asymmetric geometry, neuro-symbolic boundaries, and continuous latent-space optimization. 

Artificial Intelligence for Physiological Time-Series Classification: From Population Models to Personalized Health Monitoring

Wednesday, October 21, 2026 - 02:00 pm
AI Seminar Room 529

DISSERTATION DEFENSE

Author : Andrew Kenneth Smith 
Advisors: Dr. Homayoun Valafar
Date: October 21, 2026
Time: 02:00 pm
Location: AI Seminar Room 529
Link:  https://teams.microsoft.com/meet/226969496196462?p=FsccssHidNdPu1F1S5

Abstract

Physiological signals such as EEG, accelerometry, and inertial motion carry rich information about health and behavior, yet building reliable classifiers that work outside the laboratory remains difficult. This dissertation advances a single thesis: machine learning classifiers for physiological time series generalize well across health behaviors, clinical contexts, and species when sufficient population data is available, but population models do not generalize uniformly to the individual. Closing the gap to the individual requires adapting the model itself: recalibrating a per-person decision threshold is a cheap, label-light first step, but it recovers only part of what fine-tuning achieves. How much it recovers depends on the person as well as the behavior, so the regimes that decide the right intervention (calibration-limited, representation-limited, and individual-invariant) belong to a person and behavior pair rather than to a behavior, and the central contribution is a framework that identifies which regime a person and behavior pair falls in and matches the personalization intervention accordingly.

The argument is developed through five studies that progress from population models toward personalization. A feature-engineering baseline establishes classical machine learning for rodent sleep staging, after which a deep convolutional recurrent model reproduces expert-level vigilance-state scoring at a macro F1 of 0.878. The same architectural principles transfer to human activity recognition, where a single model discriminates multiple health behaviors from a single wrist sensor at a macro F1 of 0.817, and to medication-adherence monitoring in opioid use disorder treatment, where neural gesture recognition detects therapeutic self-administration. A capstone study on smoking detection exposes the core obstacle directly: population models cluster gestures cleanly yet classify individuals only moderately. Recalibrating each participant’s decision threshold, with zero gradient updates, raises mean smoking-class F1 from 0.702 to 0.767, but this recovers only 43.1% of the gain that fine-tuning achieves (F1 0.853), and fine-tuning beats calibration significantly. The convolutional layers can stay frozen, but the recurrent layers must adapt, and for one participant calibration recovers almost nothing and only fine-tuning closes the gap. In the medication-adherence setting of opioid use disorder, distinguishing methadone from buprenorphine is representation-limited at the behavior level: threshold recalibration does not help in any of three runs, while fine-tuning does. Personalization therefore needs model adaptation, with calibration as its cheap first step, and whether that first step suffices must be judged for each person and behavior pair, not per behavior alone.

Alongside the empirical studies, the dissertation contributes a reproducible annotation and experimentation platform that supports the full pipeline from raw signal import to labeled datasets and trained models. Taken together, the studies show that the path from accurate population-level classification to deployable personalized health monitoring runs through diagnosing which regime an individual’s behavior falls in and matching the personalization intervention to that regime, not through ever-larger models, and they chart a practical route for that transition across diverse physiological sensing domains.
 

Improving Minority-Class Recall While Limiting Majority-Class Loss: A Divergence-Regularized Loss Function for Imbalanced Classification, with Applications in Proteomics and Astronomy

Thursday, October 22, 2026 - 10:00 am
online

DISSERTATION DEFENSE

Author : Fawad Kirmani
Advisors: Dr. John Rose
Date: October 22, 2026
Time: 10:00 am
Location: Virtual
Link: https://teams.microsoft.com/l/meetup-join/19%3ameeting_NzI3MDUwODctMzVi…

Abstract

In deep learning classification tasks, class imbalance is a frequent challenge, because a model trained on imbalanced data tends to favor the majority class and miss the rare cases that are often of greatest interest. In this work, we introduce our custom loss function, which adds a symmetric Kullback--Leibler divergence term to categorical cross-entropy. It is designed to increase recall of the minority class while limiting the loss of recall in the majority class. We evaluated our custom loss function against categorical cross-entropy and focal loss using the first two classes of the CIFAR-10 dataset. Over 30 seeds and 12 class imbalance ratios, it increased the recall of the minority class most under severe imbalance, by 5.15 percentage points at the most imbalanced ratio, while keeping the decrease in the recall of the majority class below 2 percentage points at every ratio, at the cost of a lower ROC-AUC. In addition, we also evaluated our custom loss function on the German Credit dataset. These additional results are presented in an appendix. Having demonstrated the utility of our custom loss function, we applied it to two binary classification tasks in proteomics and astronomy. We conclude with a multiclass task in astronomy that does not use it.

First, we developed a bidirectional long short-term memory (BiLSTM) autoencoder for identifying proteotypic peptides. The model combines peptide embeddings with physicochemical peptide features and is trained with our custom loss function. Under 5x2 cross-validation, it achieved a mean ROC-AUC of 95.08%, sensitivity of 92.39%, and specificity of 77.55%. Relative to categorical cross-entropy, our custom loss function significantly reduced the misclassification rate, and an ensemble with an XGBoost model reduced it further.

Second, we developed a deep convolutional neural network to detect polar ring galaxies in Sloan Digital Sky Survey images, trained with our custom loss function. Across ten folds, the model achieved a mean test ROC-AUC of approximately 99% and a mean recall of 96% for the polar ring galaxy class, and it maintained strong performance on a heavily imbalanced subset of 20 polar ring galaxies and 1,000 other galaxies.

Third, we developed a deep learning model to detect gravitationally lensed supernovae (SNe), which are extremely rare and fade quickly. Instead of relying on magnification, our model identifies doubly-imaged SNe with small separations (<0.6 arcsec) from the shape of the transient in a single difference image, so it can give an early alert before a light curve is available. We trained a VGG19 network with a Convolutional Block Attention Module on simulated SNe planted into archival Hubble Space Telescope images. Across ten test folds, the model achieved a mean ROC-AUC of 99.29% and mean recall scores of 96.6%, 91.0%, and 97.1% for images with zero, one, and two simulated transients. Unlike the Source-Extractor baseline, which could not identify single transients, it separated single from doubly-imaged SNe. Further work is needed to assess its performance on real survey data.

Together, these studies show that our custom loss function can improve minority-class performance in imbalanced binary classification while making the trade-offs explicit, and they demonstrate deep learning applications in proteomics and astronomy, where identifying rare objects or events is an important scientific problem.