Inference Framework for Model Update and Development
- 39 views
Monday, February 26, 2018 - 01:30 pm
Meeting room 2265, Innovation Center
DISSERTATION DEFENSE
Xiao Lin
Advisor : Dr. Gabriel Terejanu
Abstract
Computational models play an important role in scientific discovery and engineering design. However, developing computational models is challenging, since the process always follows a path contaminated with errors and uncertainties. The uncertainties and errors inherent in computational models are the result of many factors, including experimental uncertainties, model structure inadequacies, uncertainties in model parameters and initial conditions, as well as errors due to numerical discretiza- tions. To realize the full potential in applications it is critical to systematically and economically reduce the uncertainties inherent in all computational models.
The update and development of computational models is a recursive process between data assimilation and data selection. In data assimilation, measurements are incorporated into computational simulations to reduce the uncertainties of the model and in reverse, the simulations help determine where to acquire data such that most information can be provided.
Currently, data assimilation techniques are overwhelmed by data volume and velocity and increased complexity of computational models. In this work, we develop a novel data assimilation approach EnLLVM which is based on linear latent variable model. There are several advantages of this approach. First, it works well with high dimensional dynamic systems, but only requires a small number of samples. Second, it can absorb model structure error and reflect the error in the uncertainty of data assimilation results. In addition, data assimilation is performed without calculating likelihood of observation, thus it can be applied to data assimilation problems in which likelihood is intractable.
Obtaining informative data is also crucial, as data collection is an expensive endeavor for a number of science and engineering fields. Mutual information, which naturally measures information provided about one quantity by knowing the other quantity, has become a major design metric and has fueled a large body of work on experimental design. However, estimating mutual information is challenging and results are not reliable in high dimensions. In this work, we derive a lower bound of mutual information, which is computed in much lower dimensions. This lower bound can be applied to experimental design as well as other problems that require comparison of mutual information.
At last, we develop a general framework for building computational models. In this framework, hypotheses about unknown model structure are generated by using EnLLVM for data assimilation and lower bound of mutual information for finding relations between state variables and unknown structure function. Then, different hypotheses can be ranked with model selection technique. This framework not only provides a way to infer model discrepancy, but also could further contribute to scientific discoveries.
Abstract
A wide range of modern software-intensive systems (e.g., autonomous systems, big data analytics, robotics, deep neural architectures) are built configurable. These systems offer a rich space for adaptation to different domains and tasks. Developers and users often need to reason about the performance of such systems, making tradeoffs to change specific quality attributes or detecting performance anomalies. For instance, developers of image recognition mobile apps are not only interested in learning which deep neural architectures are accurate enough to classify their images correctly, but also which architectures consume the least power on the mobile devices on which they are deployed. Recent research has focused on models built from performance measurements obtained by instrumenting the system. However, the fundamental problem is that the learning techniques for building a reliable performance model do not scale well, simply because the configuration space is exponentially large that is impossible to exhaustively explore. For example, it will take over 60 years to explore the whole configuration space of a system with 25 binary options.
In this talk, I will start motivating the configuration space explosion problem based on my previous experience with large-scale big data systems in industry. I will then present my transfer learning solution to tackle the scalability challenge: instead of taking the measurements from the real system, we learn the performance model using samples from cheap sources, such as simulators that approximate the performance of the real system, with a fair fidelity and at a low cost. Results show that despite the high cost of measurement on the real system, learning performance models can become surprisingly cheap as long as certain properties are reused across environments. In the second half of the talk, I will present empirical evidence, which lays a foundation for a theory explaining why and when transfer learning works by showing the similarities of performance behavior across environments. I will present observations of environmental changes‘ impacts (such as changes to hardware, workload, and software versions) for a selected set of configurable systems from different domains to identify the key elements that can be exploited for transfer learning. These observations demonstrate a promising path for building efficient, reliable, and dependable software systems. Finally, I will share my research vision for the next five years and outline my immediate plans to further explore the opportunities of transfer learning.
Pooyan Jamshidi is a postdoctoral researcher at Carnegie Mellon University, where he works on transfer learning for building performance models to enable dynamic adaptation of mobile robotics software as a part of BRASS, a DARPA sponsored project. Prior to his current position, he was a research associate at Imperial College London, where he worked on Bayesian optimization for automated performance tuning of big data systems. He holds a Ph.D. from Dublin City University, where he worked on self-learning Fuzzy control for auto-scaling in the cloud. He has spent 7 years in industry as a developer and a software architect. His research interests are at the intersection of software engineering, systems, and machine learning, and his focus lies predominantly in the areas of highly-configurable and self-adaptive systems (more details:
Abstract:
The recent proliferation of acoustic devices, ranging from voice assistants to wearable health monitors, is leading to a sensing ecosystem around us -- referred to as the Internet of Acoustic Things or IoAT. My research focuses on developing hardware-software building blocks that enable new capabilities for this emerging future. In this talk, I will sample some of my projects. For instance, (1) I will demonstrate carefully designed sounds that are completely inaudible to humans but recordable by all microphones. (2) I will discuss our work with physical vibrations from mobile devices, and how they conduct through finger bones to enable new modalities of short range, human-centric communication. (3) Finally, I will draw attention to various acoustic leakages and threats that arrive with sensor-rich environments. I will conclude this talk with a glimpse of my ongoing and future projects targeting a stronger convergence of sensing, computing, and communications in tomorrow’s IoT, cyber-physical systems, and healthcare technologies.
Bio:

