Dr. Yan Tong, a Computer Science and Engineering assistant professor at the University of South Carolina, College of Engineering and Computing, has received the National Science Foundation's CAREER award. The Faculty Early Career Development (CAREER) Program is NSF's most prestigious award in support of the early career-development activities of junior faculty. The award will support her research on facial recognition, that is, on developing algorithms that can detect whether a person in a video is smiling, angry, confused, etc. This work is innovative in that it blends both visual and auditory information so the software uses both what it sees and what it hears in the video to determine how the person's face looks like. In more detail:

This project develops a unified multimodal and multialgorithm fusion framework to recognize facial action units such as “lip corner raiser” and “lips apart”, which describe complex and rich facial behaviors. This framework systematically captures the inherent interactions between the visual and audio channels in a global context of human perception of facial behavior. Advanced machine learning techniques are developed to integrate these relationships together with uncertainties associated with various visual and audio measurements in the fusion framework to achieve a robust and accurate understanding of facial activity. It is these coordinated and consistent interactions that produce a meaningful facial display. The basic research in this unified fusion framework can foster advanced computer vision and machine learning technologies with applications across a wide range of fields varying from entertainment to psychiatry to human-computer interaction. An integration of research and education promotes cutting-edge training on human-computer interactions to K-12, undergraduate, and graduate students, especially encourages the participation of women in engineering and computing.

The NSF CAREER program recognizes and supports the early career-development activities of those teacher-scholars who are most likely to become the academic leaders of the 21st century. CAREER awardees are selected on the basis of creative career-development plans that effectively integrate research and education within the context of the mission of their institution and department.