DISSERTATION DEFENSE 

Author : Yuhang Lu

Advisor : Dr. Song Wang

Date : May 23, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

Meeting Link: https://zoom.us/j/7182193712

Abstract

Semantic segmentation that aims at grouping discrete pixels into connected regions is a fundamental step in many high-level computer vision tasks. In recent years, Convolutional Neural Networks (CNNs) have made breakthrough progresses in public semantic segmentation benchmarks. The ability of learning from large-scale labeled datasets empowers them to generalize to unseen images better than traditional non-learning-based methods. Nevertheless, the heavy dependency on labeled data also limits their applications in tasks where high-quality ground truth segmentation masks are scarce or difficult to acquire. In this dissertation, we study the problem of alleviating the data dependency for CNN-based segmentation with a focus on leveraging the shape prior knowledge of objects.

    Shape prior knowledge could provide rich learning-free information of object boundaries if properly utilized. However, this is not trivial for CNN-based segmentation because of its nature of pixel-wise classification. To address this problem, we propose novel methods to integrate three types of shape priors into CNN training, including implicit, explicit and class-agnostic priors. They cover from specific objects with strong prior to general objects with weak prior. To demonstrate the practical value of our methods, we present each of them within a challenging real-world image segmentation task. 1) We propose a weakly supervised segmentation method to extract curve structures stamped on cultural heritage objects, which implicitly takes advantage of the prior knowledge of their thin and elongated shape to relax the training label from pixel-wise curve mask to single-pixel curve skeleton, and outperforms fully supervised alternatives by at least 7.7% in F1 score. 2) We propose a one-shot segmentation method to learn to segment anatomical structure from X-ray images with only one labeled image, which is realized by explicitly model the shape and appearance prior knowledge of objects into the objective function of CNNs. It performs competitively compared to state-of-the-art fully supervised methods when using a single label, and could outperform them when a human-in-the-loop mechanism is incorporated. 3) Finally, we attempt to model shape priors in a universal form that is agnostic to object classes, where the knowledge can be distilled from a few labeled samples through a meta-learning strategy. Given a base model pretrained on existing large-scale dataset, our method could adapt it to any unseen domains with the help of a few labeled images and masks. Experimental results show that our method significantly improve the performance of base models in a variety of cross-domain segmentation tasks.

DISSERTATION DEFENSE 

Author : Zhenyao Wu

Advisor : Dr. Song Wang

Date : May 18, 2022

Time 9:00 am

Place : Virtual (Zoom link below)

Meeting Link: https://zoom.us/j/91863722659?pwd=KytCSmc3NGRRbHhPSmczM2EyUnpuQT09

Abstract

Estimating depth from images has become a very popular task in computer vision which aims to restore the 3D scene from 2D images and identify important geometric knowledge of the scene. Its performance has been significantly improved by convolutional neural networks in recent years, which surpass the traditional methods by a large margin. However, the natural scenes are usually complicated, and hard to build the correspondence between pixels across frames, such as the region containing moving objects, illumination changes, occlusions, and reflections. This research explores rich and comprehensive spatial correspondence across images and designs three new network architectures for depth estimation whose inputs can be a single image, stereo pairs, or monocular video. 

First,  we propose a novel semantic stereo network named SSPCV-Net, which includes newly designed pyramid cost volumes for describing semantic and spatial correspondence on multiple levels. The semantic features are inferred from a semantic segmentation subnetwork while the spatial features are constructed by hierarchical spatial pooling. In the end, we design a 3D multi-cost aggregation module to integrate the extracted multilevel correspondence and perform regression for accurate disparity maps. We conduct comprehensive experiments and comparisons with some recent stereo matching networks on Scene Flow, KITTI 2015 and 2012, and Cityscapes benchmark datasets, and the results show that the proposed SSPCV-Net significantly promotes the state-of-the-art stereo-matching performance. 

Second, we present a novel SC-GAN network with end-to-end adversarial training for depth estimation from monocular videos without estimating the camera pose and pose change over time. To exploit cross-frame relations, SC-GAN includes a spatial correspondence module that uses Smolyak sparse grids to efficiently match the features across adjacent frames and an attention mechanism to learn the importance of features in different directions. Furthermore, the generator in SC-GAN learns to estimate depth from the input frames, while the discriminator learns to distinguish between the ground-truth and estimated depth map for the reference frame. Experiments on the KITTI and Cityscapes datasets show that the proposed SC-GAN can achieve much more accurate depth maps than many existing state-of-the-art methods on monocular videos. 

Finally, we propose a new method for single image depth estimation which utilize the spatial correspondence from stereo matching. To achieve the goal, we incorporate a pre-trained stereo network as a teacher to provide depth cues for the features and output generated by the student network which is a monocular depth estimation network. To further leverage the depth cues, we developed a new depth-aware convolution operation that can adaptively choose subsets of relevant features for convolutions at each location. Specifically, we compute hierarchical depth features as the guidance, and then estimate the depth map using such depth-aware convolution which can leverage the guidance to adapt the filters.  Experimental results on the KITTI online benchmark and Eigen split datasets show that the proposed method achieves the state-of-the-art performance for single-image depth estimation. 

Abstract

Building machines that automatically write computer programs is a grand challenge in AI. Such a development would offer the potential to automatically build bug-free programs and to discover novel efficient algorithms. In this talk, I will describe progress towards this grand challenge, i.e. progress towards building an `Automatic Computer Scientist (AutoCS)`. I will focus on major recent breakthroughs in inductive logic programming (ILP), a form of machine learning based on mathematical logic, with wider applications in drug design, game playing, and visual reasoning.

Bio

I am a research fellow at the University of Oxford. I work on logic and machine learning, i.e. inductive logic programming. I run the Logic and Learning (LoL) group and the `Automatic Computer Scientist' project.

Location

In person: Swearingen Engineering Center in Room 2A31

 Virtual MS Teams

Abstract

Adaptive controllers employ online observations of system performance to determine control policies for driving a system toward a desired state. For example, the adaptive cruise control module in a car utilizes data from various sensors to steer the vehicle such that it maintains a safe following distance and stays within the speed limit. In this talk, I will introduce a set of learning algorithms to synthesize feedback control policies for dynamic systems. Specifically, I will discuss topics including event-triggered control, approximate dynamic programming, and the limits of learning-based controllers for real-time control.

Bio

Vignesh Narayanan (Member, IEEE) received the B.Tech. Electrical and Electronics Engineering degree from SASTRA University, Thanjavur, India, the M.Tech. degree with specialization in Control Systems from the National Institute of Technology Kurukshetra, Haryana, India, in 2012 and 2014, respectively, and the Ph.D. degree from the Missouri University of Science and Technology, Rolla, MO, USA, in 2017. He joined the Applied Mathematics Lab and Brain Dynamics and Control Research Group in the Dept. of Electrical and Systems Engineering at the Washington University in St. Louis, where he was a postdoctoral research associate. He is currently with the Dept. of Computer Science and Engineering and AI institute of University of South Carolina. He is also affiliated with CAN (Center for Autism and Neurodevelopmental disorders). His current research interests include learning and adaptation in dynamic population systems, complex dynamic networks, reinforcement learning, and computational neuroscience.

Location

In person: Swearingen Engineering Center in Room 2A31

Virtual MS Teams

Time

2:20-3:10pm

DISSERTATION DEFENSE 

Author : Xiaoguang Li

Advisor : Dr. Song Wang

Date : March 31, 2022

Time : 3:00 pm

Place : Virtual (Zoom link below)

Zoom link: https://zoom.us/j/3845952539?pwd=WkVxVmdETU4zcy9FcDNnOVNDdzE4UT09

Abstract

Dendrite core is the center point of the dendrite. The information of dendrite core is very helpful for material scientists to analyze the properties of materials. Therefore, detecting the dendrite core is a very important task in the material science field. Meanwhile, because of some special properties of the dendrites, this task is also very challenging. Different from the typical detection problems in the computer vision field, detecting the dendrite core aims to detect a single point location instead of the bounding-box. As a result, the existing regressing bounding-box based detection methods can not work well on this task because the calculated center point location based on the upper-left and lower-right corners of the bounding-box is usually not precise. In this work, we formulate the dendrite core detection problem as a segmentation task and proposed a novel detection method to detect the dendrite core directly. Our whole pipeline contains three steps: Easy Sample Detection (ESD), Hard Sample Detection (HSD), and Hard Sample Refinement (HSR). Specifically, ESD and HSD focus on the easy samples and hard samples of dendrite cores respectively. Both of them employ the same Central Point Detection Network (CPDN) but not sharing parameters. To make HSD only focus on the feature of hard samples of dendrite cores, we destroy the structure of the easy samples of dendrites which are detected by ESD and force HSD to learn the feature of hard samples. HSR is binary classifier which is used to filter out the false positive prediction of HSD. We evaluate our method on the dendrite dataset. Our method outperforms the state-of-the-art baselines on three metrics, i.e., Recall, Precision, and F-score

Women in Computing is hosting its first ever programming workshop! We will be learning the basics of Python! If you have an interest in learning coding, come on out! We hope to do more workshops in other languages in the future so come by and show your interest tonight, March 28th at 6pm in the Innovation Center Room 2277!

If you want to join virtually we will try to simultaneously share screen via Zoom so be sure to join our GroupMe to get access to the Zoom link.

GroupMe: https://groupme.com/join_group/34681325/pIJInQ

Everyone – all genders and majors is welcome!

Abstract

This study provides and overview of data system and linkage process for people living with HIV in South Carolina. The purpose of the study is developing and identifying best machine-learning based predictive model for HIV medical treatment status using historical data for a comprehensive established data repository. We provide findings from the study thus far.

Bio

Banky Olatosi is tenure track assistant professor in the Department of Health Services Policy and Management, at the Arnold School of Public Health, University of South Carolina (UofSC). He is published in peer-reviewed journals and his research interests are in the fields of Big Data Health Analytics, HIV/AIDS, COVID-19 and rural health. He has expertise in the field of Data Analytics and Data Mining, and currently has NIH grant funding in this area. He co-leads the UofSC national big data health science center (BDHSC). He is a Fellow of the American College of Healthcare Executives (FACHE). He is passionate and committed to the improvement of graduate healthcare education. He currently serves as the Chair of the CAHME Accreditation Council and is also a CAHME national board member. He is a UofSC 2021 Breakthrough Research award winner. Banky Olatosi earned his doctorate in Health Services Policy and Management from the University of South Carolina and earned his MPH in Public Health Administration and Policy from the University of Minnesota (Twin Cities). He also holds a master’s degree in biochemistry from the University of Lagos.

Location

In person

Swearingen Engineering Center in Room 2A31

Virtual MS Teams

Time

2:20-3:10pm

DISSERTATION DEFENSE

(will take place in hybrid fashion; both physical and virtual)

Author : Manas Gaur

Advisor : Dr. Amit P. Sheth

Date : March 25, 2022

Time 10:30 am

Location : Seminar Room, AI Institute, 5th Floor,

    1112 Greene Street (Science and Technology Building)
     Columbia, South Carolina-29208

Virtual Zoom Link

              Meeting ID: 844 013 9296
         Passcode: 12345

Abstract:

In DARPA’s view on three waves of AI, the first wave of AI termed symbolic AI, focused on explicit knowledge. The current second wave of AI is termed statistical AI. Deep learning techniques have been able to exploit large amounts of data and massive computational power to improve human levels of performance in narrowly defined tasks. Separately, knowledge graphs emerged as a powerful tool to capture and exploit an extensive amount and variety of explicit knowledge to make algorithms better understand the content, and enable the next generation of data processing, such as in semantic search. After initial hesitancy about the scalability of the knowledge creation process, the last decade has seen significant growth in developing and applying knowledge, usually in the form of knowledge graphs (KG). Examples range from the use of DBPedia in IBM’s Watson to Google Knowledge Graph in Google Semantic Search, and the application of ProteinBank in AlphaFold, recognized by many as the most significant AI breakthrough, as well as numerous domain-specific knowledge have been applied in improving AI methods in diverse domains such as medicine and healthcare, finance, manufacturing, and defense.

Now, we herald towards the third wave of AI built on what is termed as the neuro-symbolic approach that combines the strengths of statistical and symbolic AI. Combining the respective powers and benefits of using knowledge graphs and deep learning is particularly attractive. This has led to the development of an approach we have called knowledge-infused (deep) learning. This dissertation will advance the currently limited forms of combining the knowledge graphs and deep learning, called shallow and semi-infusion, with a more advanced form called deep-infusion, that will support stronger interleaving of more variety of knowledge at different levels of abstraction with layers in a deep learning architecture.

This dissertation will investigate the knowledge-infusion strategy in two important ways. The first is to infuse knowledge to make any classification task explainable. The second is to achieve explainability in any natural language generation tasks. I will demonstrate the effective strategies of knowledge infusion that bring five characteristic properties in any statistical AI model: (1) Context Sensitivity, (2) Handling Uncertainty and Risk, (3) Interpretable in learning, (4) User-level Explainability, and (5) Transferability across natural language understanding (NLU) tasks. Along with proven methodological contributions in AI made by the dissertation, I will show their applications for open-domain and close-domain NLU.
Furthermore, the dissertation will showcase the utility of incorporating diverse forms of knowledge: linguistic, commonsense, broad-based, and domain-specific. As the dissertation illustrates the success in various domains, achieving state-of-the-art in specific applications, and significant contributions towards improving the state of machine intelligence, I will walk through careful steps to prevent errors arising due to knowledge infusion. Finally, for future directions, I will discuss two exciting areas of research where knowledge infusion would be pivotal to propel machine understanding.

DISSERTATION DEFENSE

Author : Hanin Omar

Advisor : Dr. Homayoun Valafar

Date : March 21, 2022

Time: 9:30 am

Place: Virtual Teams Link

Abstract

A complete understanding of the structure-function relationship of proteins requires an analysis of their dynamic behaviors in addition to the static structure. However, all current approaches to the study of dynamics in proteins have their shortcomings. A conceptually attractive and alternative approach simultaneously characterizes a protein's structure and its intrinsic dynamics⁠⁠. Ideally, such an approach could solely rely on RDC data-carrying both structural and dynamical information. The major bottleneck in the utilization of RDC data in recent years has been attributed to a lack of RDC analysis tools capable of extracting the pertinent information embedded within this complex source of data.  

Here we present a comprehensive strategy for structure calculation and reconstruction of discrete state dynamics from RDC data based on the SVD method of order tensor estimation. In addition to structure determination, we provide a mechanism of producing an ensemble of conformations for the dynamical regions of a protein from RDC data. The developed methodology has been tested on simulated RDC data with ±1Hz of error from an 83 residue α protein (PDB ID 1A1Z). In nearly all instances, our method reproduced the protein structure, including the conformational ensemble, to within less than 2Å. Based on our investigations, arc motions with more than 30° of rotation are recognized as internal dynamics and are reconstructed with sufficient accuracy.  Furthermore, states with relative occupancies above 20% are consistently recognized and reconstructed successfully. Arc motions with a magnitude of 15° or relative occupancy of less than 10% are consistently unrecognizable as dynamical regions within the context of ±1Hz of error.  

 We also introduce a computational approach named REDCRAFT that allows for uncompromised and concurrent characterization of protein structure and dynamics. We have subjected DHFR (PDB-ID 1RX2), a 159-residue protein, to a fictitious but plausible, mixed-mode internal dynamics model. In this simulation, DHFR was segmented into 7 regions. The two dynamical and rigid-body segments experienced an average orientational modification of 7˚ and 12˚, respectively. Observable RDC data for backbone C'-N, N-H, and C'-H were generated from 102 frames that described the molecular trajectory. The Dynamic Profile generated by REDCRAFT allowed for the recovery of individual fragments with bb-rmsd of less than 1Å and the identification of different dynamical regions of the protein. Following the recovery of fragments, structural assembly correctly assembled the four rigid fragments with respect to each other, categorized the two domains that underwent rigid-body dynamics, and identified one dynamical region for which no conserved structure can be defined. In conclusion, our approach successfully identified dynamical domains, recovery of structure where it is meaningful, and relative assembly of the domains when possible.  

Abstract

In 2020, Dr. Amari N. Lewis had the opportunity to conduct research at Aalborg University in Denmark. This talk on how smart city infrastructure and Blockchain can reduce harmful vehicle emissions revisits the research discoveries from Dr. Lewis' time in Denmark. As a result, we discovered two methodologies to reduce harmful vehicle emissions through the use of technology and were able to validate our theory through simulation. The first is a Blockchain Emissions Trading System (B-ETS). The second method involves traffic theory, smart infrastructure and dissuasion methods to improve air quality in residential areas.

Bio

Amari N. Lewis is currently a Postdoctoral scholar in the Department of Computer Science & Engineering at the University of California San Diego. Her current research is in the area of Computer Science Education studying retention and experiences of students especially focused on students from marginalized populations. Amari earned her PhD in Computer Science from the University of California, Irvine where her research focused on technological advancement in transportation systems. In the last year of her PhD she conducted research in Denmark at Aalborg University. At Aalborg University the research focused on the use of Blockchain in smart infrastructure.

Location:

In person

Swearingen Engineering Center in Room 2A31

Virtual MS Teams

Time

2:20-3:10pm