COLLOQUIUM Zsolt Kira Abstract A large number of robot perception tasks have been revolutionized by machine learning and deep neural networks in particular. However, current learning methods are limited in several ways that hinder their large-scale use for critical robotics applications: They are often focused on individual sensor modalities, do not attempt to understand semantic information in a fine-grained temporal manner, and are beholden to strong assumptions about the data (e.g. that the data distribution is the same when deployed in the real world as when trained). In this talk, I will describe work on novel deep learning architectures for moving beyond current methods to develop a richer multi-modal and fine-grained scene understanding from raw sensor data. I will also discuss methods we have developed that can use transfer learning to deal with changes in the environment or the existence of entirely new, unknown categories in the data (e.g. unknown object types). I will focus especially on this latter work, where we use neural networks to learn how to compare objects and transfer such learning to new domains using one of the first deep-learning based clustering algorithms, which we developed. I will show examples of real-world robotic systems using these methods, and conclude by discussing future directions in this area, towards making robots able to continually learn and adapt to new situations as they arise. Dr. Zsolt Kira received his B.S. in ECE at the University of Miami in 2002 and M.S. and Ph.D. in Computer Science from the Georgia Institute of Technology in 2010. He is currently a Senior Research Scientist and Branch Chief of the Machine Learning and Analytics group at the Georgia Tech Research Institute (GTRI). He is also an Adjunct at the School of Interactive Computing and Associate Director of Georgia Tech’s Machine Learning Center (ML@GT). He conducts research in the areas of machine learning for sensor processing and robot perception, with emphasis on feature learning for multi-modal object detection, video analysis, scene characterization, and transfer learning. He has over 25 publications in these areas, several best paper/student paper and other awards, and has been invited to speak at related workshops in both academia and government venues. Date: Feb. 28, 2018 Time: 10:15-11:15 am Place: Innovation Center, Room 2277

DISSERTATION DEFENSE Zibo Meng Advisor : Dr. Yan Tong Abstract In spite of great progress achieved on posed facial display and controlled image acquisition, performance of facial action unit (AU) recognition degrades significantly for spontaneous facial displays. Furthermore, recognizing AUs accompanied with speech is even more challenging since they are generally activated at a low intensity with subtle facial appearance/geometrical changes during speech, and more importantly, often introduce ambiguity in detecting other co-occurring AUs, e.g., producing non-additive appearance changes. All the current AU recognition systems utilized information extracted only from visual channel. However, sound is highly correlated with visual channel in human communications. Thus, we propose to exploit both audio and visual information for AU recognition. Specifically, a feature-level fusion method combining both audio and visual features is first introduced. Specifically, features are independently extracted from visual and audio channels. The extracted features are aligned to handle the difference in time scales and the time shift between the two signals. These temporally aligned features are integrated via feature-level fusion for AU recognition. Second, a novel approach that recognizes speech-related AUs exclusively from audio signals based on the fact that facial activities are highly correlated with voice during speech is developed. Specifically, dynamic and physiological relationships between AUs and phonemes are modeled through a continuous time Bayesian network (CTBN); then AU recognition is performed by probabilistic inference via the CTBN model. Third, a novel audiovisual fusion framework, which aims to make the best use of visual and acoustic cues in recognizing speech-related facial AUs is developed. In particular, a dynamic Bayesian network (DBN) is employed to explicitly model the semantic and dynamic physiological relationships between AUs and phonemes as well as measurement uncertainty. AU recognition is then conducted by probabilistic inference via the DBN model. To evaluate the proposed approaches, a pilot AU-coded audiovisual database was collected. Experiments on this dataset have demonstrated that the proposed frameworks yield significant improvement in recognizing speech-related AUs compared to the state-of-the-art visual-based methods. Furthermore, more impressive improvement has been achieved for those AUs, whose visual observations are impaired during speech.

DISSERTATION DEFENSE Xiao Lin Advisor : Dr. Gabriel Terejanu Abstract Computational models play an important role in scientific discovery and engineering design. However, developing computational models is challenging, since the process always follows a path contaminated with errors and uncertainties. The uncertainties and errors inherent in computational models are the result of many factors, including experimental uncertainties, model structure inadequacies, uncertainties in model parameters and initial conditions, as well as errors due to numerical discretiza- tions. To realize the full potential in applications it is critical to systematically and economically reduce the uncertainties inherent in all computational models. The update and development of computational models is a recursive process between data assimilation and data selection. In data assimilation, measurements are incorporated into computational simulations to reduce the uncertainties of the model and in reverse, the simulations help determine where to acquire data such that most information can be provided. Currently, data assimilation techniques are overwhelmed by data volume and velocity and increased complexity of computational models. In this work, we develop a novel data assimilation approach EnLLVM which is based on linear latent variable model. There are several advantages of this approach. First, it works well with high dimensional dynamic systems, but only requires a small number of samples. Second, it can absorb model structure error and reflect the error in the uncertainty of data assimilation results. In addition, data assimilation is performed without calculating likelihood of observation, thus it can be applied to data assimilation problems in which likelihood is intractable. Obtaining informative data is also crucial, as data collection is an expensive endeavor for a number of science and engineering fields. Mutual information, which naturally measures information provided about one quantity by knowing the other quantity, has become a major design metric and has fueled a large body of work on experimental design. However, estimating mutual information is challenging and results are not reliable in high dimensions. In this work, we derive a lower bound of mutual information, which is computed in much lower dimensions. This lower bound can be applied to experimental design as well as other problems that require comparison of mutual information. At last, we develop a general framework for building computational models. In this framework, hypotheses about unknown model structure are generated by using EnLLVM for data assimilation and lower bound of mutual information for finding relations between state variables and unknown structure function. Then, different hypotheses can be ranked with model selection technique. This framework not only provides a way to infer model discrepancy, but also could further contribute to scientific discoveries.

COLLOQUIUM Pooyan Jamshidi Abstract A wide range of modern software-intensive systems (e.g., autonomous systems, big data analytics, robotics, deep neural architectures) are built configurable. These systems offer a rich space for adaptation to different domains and tasks. Developers and users often need to reason about the performance of such systems, making tradeoffs to change specific quality attributes or detecting performance anomalies. For instance, developers of image recognition mobile apps are not only interested in learning which deep neural architectures are accurate enough to classify their images correctly, but also which architectures consume the least power on the mobile devices on which they are deployed. Recent research has focused on models built from performance measurements obtained by instrumenting the system. However, the fundamental problem is that the learning techniques for building a reliable performance model do not scale well, simply because the configuration space is exponentially large that is impossible to exhaustively explore. For example, it will take over 60 years to explore the whole configuration space of a system with 25 binary options. In this talk, I will start motivating the configuration space explosion problem based on my previous experience with large-scale big data systems in industry. I will then present my transfer learning solution to tackle the scalability challenge: instead of taking the measurements from the real system, we learn the performance model using samples from cheap sources, such as simulators that approximate the performance of the real system, with a fair fidelity and at a low cost. Results show that despite the high cost of measurement on the real system, learning performance models can become surprisingly cheap as long as certain properties are reused across environments. In the second half of the talk, I will present empirical evidence, which lays a foundation for a theory explaining why and when transfer learning works by showing the similarities of performance behavior across environments. I will present observations of environmental changes‘ impacts (such as changes to hardware, workload, and software versions) for a selected set of configurable systems from different domains to identify the key elements that can be exploited for transfer learning. These observations demonstrate a promising path for building efficient, reliable, and dependable software systems. Finally, I will share my research vision for the next five years and outline my immediate plans to further explore the opportunities of transfer learning. Pooyan Jamshidi is a postdoctoral researcher at Carnegie Mellon University, where he works on transfer learning for building performance models to enable dynamic adaptation of mobile robotics software as a part of BRASS, a DARPA sponsored project. Prior to his current position, he was a research associate at Imperial College London, where he worked on Bayesian optimization for automated performance tuning of big data systems. He holds a Ph.D. from Dublin City University, where he worked on self-learning Fuzzy control for auto-scaling in the cloud. He has spent 7 years in industry as a developer and a software architect. His research interests are at the intersection of software engineering, systems, and machine learning, and his focus lies predominantly in the areas of highly-configurable and self-adaptive systems (more details: https://pooyanjamshidi.github.io/research/). Date: Feb. 26, 2018 Time: 10:15-11:15 am Place: Innovation Center, Room 2277

Abstract: The recent proliferation of acoustic devices, ranging from voice assistants to wearable health monitors, is leading to a sensing ecosystem around us -- referred to as the Internet of Acoustic Things or IoAT. My research focuses on developing hardware-software building blocks that enable new capabilities for this emerging future. In this talk, I will sample some of my projects. For instance, (1) I will demonstrate carefully designed sounds that are completely inaudible to humans but recordable by all microphones. (2) I will discuss our work with physical vibrations from mobile devices, and how they conduct through finger bones to enable new modalities of short range, human-centric communication. (3) Finally, I will draw attention to various acoustic leakages and threats that arrive with sensor-rich environments. I will conclude this talk with a glimpse of my ongoing and future projects targeting a stronger convergence of sensing, computing, and communications in tomorrow’s IoT, cyber-physical systems, and healthcare technologies. Bio: Nirupam Roy is a Ph.D. candidate in Electrical and Computer Engineering at the University of Illinois, Urbana-Champaign (UIUC). His research interests are in mobile sensing, wireless networking, and embedded systems with applications to IoT, cyber-physical-systems, and security. Roy is the recipient of the Valkenburg graduate research award, the Lalit Bahl fellowship, and the outstanding thesis awards from both his Bachelor's and Master's institutes. His recent research on "Making Microphones Hear Inaudible Sounds" received the MobiSys'17 best paper award and was selected for the ACM SIGMOBILE research highlights of the year in 2017.

Speaker: Dr. Jianjun Hu Abstract: Deep learning has led to tremendous progress in computer vision, speech recognition, and natural language processing. It has now crossed the boundary and has brought breakthroughs also in the area of bioinformatics. One interesting problem is developing accurate models for predicting peptide binding affinities to protein receptors such as MHC(Major Histocompatibility complex), which can shed understanding to adverse drug reaction and autoimmune diseases and lead to more effective protein therapy and design of vaccines. We proposed a deep convolutional neural network (CNN) based peptide binding prediction algorithm for achieving substantially higher accuracy as tested in MHC-I peptide binding affinity prediction. Our model takes raw binding peptide sequences and affinity scores or binding labels as input without needing any human-designed features. The back-propagation training algorithm allows it to learn nonlinear relationships among the amino acid positions of the peptides. It also can naturally handle the peptide length variation, MHC polymorphasim, and unbalanced training samples of MHC proteins with different alleles via a simple amino acid padding scheme. Our experiments showed that DeepMHC can achieve the state-of-the-art prediction performance on most of the IEDB benchmark datasets with a single model architecture and without using any consensus or composite ensemble classifier models. Bio: I joined CSE department of the University of South Carolina in August 2007. I am now working on integrative functional genomics and especially integrative analysis of microarray data. I am also interested in motif discovery for understanding gene expression mechanisms involved in diseases. I got my Ph.D. in Computer Science in the area of machine learning and particularly evolutionary computation at the Genetic Algorithm Research and Application Group (GARAGe) of Michigan State University. My dissertation focuses on sustainable evolutionary computation algorithms and automated computational synthesis. I have worked on the DNA motif discovery problem as Postdoc at Kihara Bioinformatics Lab, Purdue University and microarray analysis at the Computational Molecular Biology Division at the University of Southern California (another USC).

Abstract: Patching is one of the most important measures to continuously uphold security throughout the life of a software system. Patch generation and deployment are probably the most critical tasks in the process of patching. However, patch generation is typically a lengthy procedure (according to Symantec, it takes an average of 28 days to release a patch for fixing a critical security bug); and patch deployment risks system stability due to new bugs contained in patches. From the perspective of speeding up patch generation and avoiding bugs in patches, we examine the notorious heap vulnerabilities, including heap buffer overflows (such as Heartbleed), uninitialized-read, use-after-free, and double-free, and explore the following two important but less-investigated problems: (1) How fast can heap patches be generated? (2) How to ensure zero bugs in the generated patches? While quick patch generation and bug-free patches are two naturally desired goals, they contradict each other in practice. Rushed patch generation tends to introduce bugs, while creating a quality patch requires significant time for debugging, testing, and even system redesign. Thus, how to achieve the two inherently contradictory goals simultaneously has been challenging. Inspired by “targeted therapy”, a cancer treatment that precisely recognizes and kills cancer cells, we propose Targeted Heap Therapy to pinpoint and treat vulnerable buffers, which are buffers that can be exploited to launch attacks, with instantly generated bug-free patches. This talk will also present some of the important problems and future prospects on Internet of Things security as well as our ongoing work on these problems. Bio: Dr. Qiang Zeng is a Tenure-Track Assistant Professor in the Department of Computer & Information Sciences at Temple University. He received his Ph.D. in Computer Science and Engineering from the Pennsylvania State University. He has rich industry experience and has worked in the IBM T.J. Watson Research Center, the NEC Lab America, Symantec and Yahoo. Dr. Zeng’s main research interest is Systems and Software Security. He currently works on Mobile Security, IoT Security, and deep learning for solving security problems. He has published papers in PLDI, NDSS, MobiSys, CGO, DSN and TKDE.