CockcyHacks: Weekend Hackathon
- 32 views
Abstract:
Large Language Models (LLMs) have dramatically transformed the landscape of Generative AI, making profound impacts across a broad spectrum of domains. From enhancing Recommender Systems to advancing the frontiers of Natural Language Processing (NLP), LLMs have become indispensable. Their versatility extends into specialized sectors, such as finance with the development of BloombergGPT, and healthcare through MedLlama, showcasing their adaptability and potential for industry-specific innovations.
In this presentation, we will embark on a comprehensive exploration of the evolution of Large Language Models. Our journey will trace the origins of LLMs, highlighting key milestones and breakthroughs, and proceed to examine the latest advancements and research directions in the field. To mirror the structured and layered nature of LLMs themselves, our discussion will be organized into distinct sections. We'll begin with the foundational aspect of prompting, delve into the intricacies of their architecture, and discuss pivotal strategies such as Pretraining, Fine-tuning, and Parameter Efficient Fine-Tuning (PEFT). Furthermore, we'll address the challenges and solutions related to the mitigation of hallucination, a critical aspect of ensuring the reliability and accuracy of LLM-generated content.
Speaker Bio:
Vinija Jain brings to the table an extensive background in machine learning, with significant expertise in developing recommender systems at Amazon and spearheading NLP initiatives at Oracle. Her passion for artificial intelligence was ignited during her time in the Stanford AI program, which served as a catalyst for her deep dive into the field. Currently, Vinija is actively engaged in fundamental research and collaborates with the Artificial Intelligence Institute of South Carolina (AIISC) at the University of South Carolina. Her latest work with AIISC on AI-Generated Text Detection has been recognized with an outstanding paper award at EMNLP '23, underscoring her contributions to advancing AI research and application
https://www.linkedin.com/events/thellmjourney7178098223411036160/about/
Abstract: Several real-world applications of machine learning (ML) systems such as robotics, autonomous cars, assistive technologies, smart manufacturing, and many other Internet-of-Things (IoT) applications require real-time inference with low energy consumption. The surge in demand for specialized hardware for AI applications has resulted in a rapidly expanding industry for edge AI accelerators. Anticipating this trend, several companies have developed their own specialized accelerators such as the NVIDIA Jetson Nano, Intel NCS2, and Google TPU. While many conventional neural networks can be readily deployed on many of these platforms, the support for deploying more advanced and larger models such as transformers on them has yet to be researched and developed. In this talk, we discuss two of our recent projects in which we utilize optimization mechanism such neural architecture search (NAS) and system-level innovations such as modifying the computational graphs, partitioning, and refactoring the unsupported operations to efficiently deploy ML models on edge accelerators for computer vision and natural language processing tasks.
Bio: Dr. Ramtin Zand is an assistant professor of the Computer Science and Engineering and the principal investigator of the Intelligent Circuits, Architectures, and Systems (iCAS) Lab at the University of South Carolina. The iCAS lab has close collaborations with and is supported by several multinational companies including Intel, AMD, and Juniper Networks, as well as federal agencies such as National Science Foundation (NSF). Dr. Zand has authored more than 50 journal and conference articles and two book chapters and received recognition from ACM/IEEE including the best paper runner-up of ACM GLSVLSI’18, the best poster of ACM GLSVLSI’19, and best paper of IEEE ISVLSI’21, as well as featured paper in IEEE Transactions on Emerging Topics in Computing. He has received the NSF CAREER award in 2024. His research focus is on neuromorphic computing, edge computing, processing-in-memory, and AI/ML hardware acceleration.
Online at: https://us06web.zoom.us/j/8440139296?pwd=b09lRCtJR0FCTWcyeGtCVVlUMDNKQT09&omn=85050122519
DISSERTATION DEFENSE
Department of Computer Science and Engineering
University of South Carolina
Author : Madushan Abeysinghe
Advisor : Dr. Jason Bakos
Date : March 18, 2024
Time: 10 am – 11: 00 pm
Place : Teams
Meeting ID: 234 681 711 471
Passcode: wiatfJ
Abstract
Digital signal processors (DSP), which are characterized by statically-scheduled Very Long Instruction Word architectures and software-defined scratchpad memory, are currently the go-to processor type for low-power embedded vision systems, as exemplified by the DSP processors integrated into systems-on-chips from NVIDIA, Samsung, Qualcomm, Apple, and Texas Instruments. DSPs achieve performance by statically scheduling workloads, both in terms of data movement and instructions. We developed a method for scheduling buffer transactions across a data flow graph using data-driven performance models, yielding a 25% average reduction in execution time and a reduction of up to 85% DRAM utilization for randomly-generated data flow graphs. We also developed a heuristic instruction scheduler that serves as a performance model to guide the selection of loops from a target data flow graph to be fused. By strategically selecting loops to fuse, performance gains can be achieved by eliminating unnecessary transactions with memory and increasing functional unit utilization. This approach has helped us achieve up to 1.9x speedup on average for sufficiently large data flow graphs used in image processing.
CSE Graduate Student research poster presentations.
Below is a tentative agenda for the day. If you plant to attend the luncheon, please RSVP by at this link by Monday, March 11.
Location: 550 Assembly St, Room 2277, Columbia, SC 29201
# | Title | Presenter |
---|---|---|
1. | Information Competition Simulator: A High-Performance Approach to Modeling Opinion Dynamics in Large Populations | Erik Connerty |
3. | Orthogonal Dictionary Guided Shape Completion Network for Point Cloud | Pingping Cai |
5. | Expressive and Flexible Simulation of Information Spread Strategies in Social Networks Using Planning | Bharath Muppasani |
7. | Can I eat this food or not? Explainable food recommendation through multi-contextual Grounding | Revathy Venkataramanan |
9. | Automating the Analysis of Rodent Ultrasonic Vocalizations | Sabah S. Anis |
11. | Understanding Information Spread in Dynamic Networks: A Graph Neural Network and Reinforcement Learning Based Approach | Protik Nag |
13. | Promoting Research Collaboration with Open Data Driven Team Recommendation in Response to Call for Proposals | Siva Likitha Valluru |
15. | Systematic comparison of EEG brain connectivity metrics | Diksha Srishyla |
17. | Smartwatch-Based Smoking Detection Using Accelerometer Data and Masked AutoEncoder based Neural Networks | Musa Azeem |
19. | Predicting Language Outcomes from MRI Post-Stroke: A Machine Learning Approach | Deepa Tilwani |
21. | Deep Learning For Human Vascular Analysis | Ali Firooz |
23. | Adaptive Channel Switching for Connected Vehicles under Extreme Weather Conditions: A Reinforcement Learning Based Approach | Jian Liu |
25. | RI2AP: Robust and Interpretable 2D Anomaly Prediction in Assembly Pipelines | Chathurangi Shyalika; Renjith Prasad Kaippilly Mana |
27. | Crystal structure prediction using neural network potential and age-fitness Pareto genetic algorithm | Sadman Sadeed Omee |
29. | An Integrated Transformer-Based Framework for Enhanced Longitudinal Prediction of Low Birthweight | Yang Ren |
31. | mLIFE Student Preliminary Results | Lexington Whalen |
33. | SafeChat: A Framework to Build Safe and Trustworthy Chatbots | Kausik Lakkaraju |
35. | Enhancing Injection Molding with Industry 4.0: Towards Smart Manufacturing and Real-Time Quality Prediction | Xiaoyi Liu |
37. | Crystal structure prediction using neural network potential and age-fitness Pareto genetic algorithm | Sadman Sadeed Omee; Lai Wei; Ming Hu; Jianjun Hu |
39. | Knowledge Graphs Construction & Alignment At Scale | Hong Yung Yip |
41. | Foundation Model for Pathfinding Problems | Vedant Khandelwal |
43. | PixelPrune: Sparse Object Detection for AIoT Systems via In-Sensor Segmentation and Adaptive Data Transfer | Mohammadreza Mohammadi |
45. | Computing Heuristics by Relaxation | Ross Foultz; Marco Valtorta |
# | Title | Presenter |
---|---|---|
2. | Rethinking Robust Contrastive Learning from the Adversarial Perspective | Fatemeh Ghofrani |
4. | MilliCar: Accurate 3D Bounding Box Prediction of Vehicles and Pedestrians in All Weather Conditions | Reza Tavasoli; Hem Regmi |
6. | Analysis of cancer patients molecular and clinical data | Ali Firooz |
8. | Cross modal few-shot point cloud semantic segmentation | ziyu zhao |
10. | MilliCar: Accurate 3D Bounding Box Prediction of Moving Vehicles with Millimeter-Wave Radar in All Weather Conditions | Reza Tavasoli; Hem Regmi; Joseph Telaak |
12. | Learning Discrete World Models for Planning | Misagh Soltani |
14. | Realtime Facial Expression Recognition: Neuromorphic Hardware vs. Edge AI Accelerators | James Seekings |
16. | Assessing Cognitive Measures in Aging through Discourse Tasks | Yong Yang |
18. | Web Application for Searching and Displaying Cancer Patient Data | Savannah Noblitt |
20. | Physics Guided Dual Self-supervised Learning for Structure-based Material Property Prediction | Nihang Fu |
22. | Flex-TPU: A Flexible TPU Architecture with Runtime Reconfigurable Dataflow | Peyton Chandarana |
24. | Explainable Chemical Reaction Predictions using Deep Approximate Value Iteration | Christian Geils |
26. | Flex-TPU: A Flexible TPU Architecture with Runtime Reconfigurable Dataflow | Peyton Chandarana |
28. | RDR: the Recap, Deliberate, and Respond Method for Enhanced Language Understanding | Yuxin Zi |
30. | Efficient Deployment of Transformer Models on Edge TPU Accelerators: A Real System Evaluation | Mohammadreza Mohammadi |
32. | Deep Learning-Based Classification of Gamma Photon Interaction in Room-Temperature Semiconductor Radiation Detectors | Qinyang Li |
34. | Auto-req: Automatic detection of pre-requisite dependencies between academic videos | Rushil Thareja |
36. | GENE EXPRESSION PROFILES UNDER THE INFLUENCE OF KYNURENIC ACID DURING PROGENY | Hamed Abdollahi |
38. | CURE: Simulation-Augmented Auto-Tuning in Robotics | Md Abir Hossen |
40. | Bidirectional Autoregessive Diffusion Model for Dance Generation | Canyu Zhang |
42. | SMARPchain: A Smart Marker Based Reputational Probabilistic Blockchain for Multi-Agent Systems | Matthew Sharp |
Time | Presenter | Lab |
---|---|---|
03:00 PM | Abir Hossen | Jamshidi Lab |
03:05 PM | Siva Likitha Valluru | AI4Society / Srivastava Lab |
03:10 PM | Fawad Kirmani | Rose Lab |
03:15 PM | Xeerak Muhammad | Tong Lab |
03:20 PM | Misagh Soltani | Agostinelli Lab |
03:25 PM | Ioannis Rekleitis | Rekleitis Lab |
03:30 PM | Peng Fu | Fu Lab |
03:35 PM | Dipannoy Gupta | Zhang Lab |
03:40 PM | Christian O'Reilly | O'Reilly Lab |
03:45 PM | Jian Liu | Huang Lab |
03:50 PM | Rongzhi Dong | Hu Lab |
03:55 PM | Ramtin Zand | Zand Lab |
04:00 PM | Pingping Cai | Wang Lab |
DISSERTATION DEFENSE
Author : Liang Zhao
Advisor : Dr. Song Wang
Date : March 11, 2024
Time: 8:30am-10:00am
Place : Teams Link
Abstract
Scene texts refer to arbitrary texts presented in an image captured by a camera in the real world. The tasks of scene text detection and recognition from complex images play a crucial role in computer vision, with potential applications in scene understanding, information retrieval, robotics, autonomous driving, etc. Despite the notable progress made by existing deep-learning methods, achieving accurate text detection and recognition remains challenging for robust real-world applications. The challenges in scene text detection and recognition stem from: 1) diverse text shapes, fonts, colors, styles, layouts, etc.; 2) countless combinations of characters with unstable attributes for complete detection, coupled with background interruptions that obscure character strokes and shapes in text recognition; and 3) the need for effective coordination of multiple sub-tasks in end-to-end learning. The fundamental issue lies in the absence of a particularly discriminative representation for the detection task, which involves locating exact complete words with unfixed attributes, and for the recognition task, which entails differentiating similar characters within words. Our research aims to address these challenges and enhance scene text detection and recognition by improving text discriminative representation. In this study, we focus on two interconnected problems: 1) Scene Text Recognition (STR), which involves recognizing text from scene images, and 2) Scene Text Spotting (STS), which entails simultaneously detecting and recognizing multiple texts in scene images.
Addressing the challenges of Scene Text Recognition (STR), the presence of text variations and complex backgrounds remain significant hurdles due to their impact on text feature representation. Numerous existing methods attempt to mitigate these issues by employing attentional regions, bounding boxes, or polygons. Despite these efforts, the text regions identified by such methods often retain undesirable background interference. In response, we propose a Background-Insensitive Network (BINet) that explicitly incorporates text Semantic Segmentation (SSN) to enhance the text representation and reduce the background interruptions. This approach eliminates the need for extensive pixel-level annotations in the STR training data. To maximize the benefits of semantic cues, we introduce novel segmentation refinement and embed?ding modules that refine text masks and strengthen visual features. Experimental results demonstrate that our proposed method significantly improves text recognition in the presence of complex backgrounds, achieving state-of-the-art performance across multiple public datasets.
In tackling the problem of Scene Text Spotting (STS), we introduce two novel developments. Given that the task involves a multi-task model dedicated to locating and recognizing texts in scenes, the coordination of multiple sub-tasks can exert a significant impact on each other and, subsequently, on the overall performance. Current end-to-end text spotters commonly incorporate independent sequential pipelines to conduct different multi-tasks. However, this unidirectional pipeline leads to information loss and error propagation among sub-tasks. In light of these observations, we present CommuSpotter, designed to enhance multi-task communication by explicitly and concurrently exchanging compatible information throughout the scene text spotting process. Experimental results demonstrate that our improved text representation for both sub-tasks enhances performance across public datasets.
Another prominent limitation in multi-tasks coordination in Scene Text Spotting (STS) lies in the capability of extracting and refining text representation of instances for multiple sub-tasks. Existing methods often utilize features from Convolutional Neural Networks (CNNs) and shrink the text regions in representation to perform sequential tasks. Nevertheless, the effectiveness of these methods is primarily constrained by the contextual biases inherent in the representation of CNN backbones. These biases are challenging to filter out, complicating the identification of randomly appearing texts and introducing confusion in discerning the similar characters within text instances. In response to these challenges, we propose a novel approach named Assembling Text Spotter (ATS) to mitigate the problem. ATS initially decouples image contextual information from text structure information through the separation of dual backbones. The disentanglement of image and text information eliminates the need for filtering out one from the other. Subsequently, they are dynamically and purposely aligned to generate discriminative representations for different sub-tasks. Extensive experiments conducted on existing scene text datasets demonstrate competitive performance results across multiple benchmarks for scene text spotting.
DISSERTATION DEFENSE
Author : Mohammed Essa Fawzy Essa
Advisor : Dr. Ramtin Zand
Date : Feb 29, 2024
Time: 4:00 pm – 6:00 pm
Place : Teams
Link: https://teams.microsoft.com/dl/launcher/launcher.html?url=%2F_%23%2Fl%2…
Abstract
Machine learning (ML) has become ubiquitous, integrating into numerous real-life applications. However, meeting the computational demands of ML systems is challenging, as existing computing platforms are constrained by memory bandwidth, and technology scaling no longer yields substantial improvements in system performance. This work introduces novel hardware architectures to accelerate ML workloads, addressing both compute and memory challenges.
In the compute domain, we explore various approximate computing techniques to assess their efficacy in accelerating ML computations. Subsequently, we propose the Approximate Tensor Processing Unit (APTPU), a hardware accelerator that utilizes approximate processing elements to replace direct quantization of inputs and weights in ML models, thereby enhancing performance, power efficiency, and area utilization. The APTPU achieves significant throughput gains while maintaining comparable accuracies. In the memory domain, we present the In-Memory Analog Computing (IMAC) architecture as an effective solution to the data movement issues faced by von Neumann architectures. IMAC employs memristive devices in crossbars and analog activation functions to efficiently execute matrix-vector multiplication (MVM) and non-linear vector operations in ML workloads. Finally, we integrate IMAC with TPU and APTPU architectures to capitalize on their combined strengths in accelerating MVM and matrix-matrix multiplication operations across various ML workloads. This integration leads to noteworthy performance enhancements and reduced memory requirements. This work provides design baselines and automation tools for architects to seamlessly incorporate the proposed compute and memory solutions into their custom systems.
This talk on the intersection of Physics & AI will give a high level overview of physics based neural networks, applications of physics in NLP, healthcare, data generation, autonomous vehicles, data generation etc. The talk will also have a deep dive on "Incorporating Physics into Data-Driven Computer Vision" which will cover approaches like modifications to datasets, network design, loss function, optimization and regularization schemes.
Bio: Parth Patwa is a data scientist at Amazon, working on generative AI. He graduated from UCLA. Previously he was working as a ML scientist at MIT Pathcheck. His publication in NLP and CV have over 1000 citations.
About
Human society of this century is facing several fundamental challenges including global climate change, energy crisis, and public health crisis such as cancers and COVID-19. Common to their solutions are the discovery of novel materials, molecules, proteins, and drugs. Designing these functional atomic structures is challenging due to the astonishing complexity of the interatomic interactions, sophisticated physical/chemical/geometric constraints and patterns to form stable structures, and how the structures relate to their functions. Like most other engineering design activities, currently, the mainstream paradigm of material design is the rational design approach, which emphasizes a causal understanding of the structure-function relationship and depends on heuristic expert knowledge and explicit design rules. However, the traditional material design paradigm is facing increasing challenges in designing extraordinary functional materials that can effectively meet our needs: it usually leads to sub-optimal solutions in the huge chemical design space due to their limited search capability; it is difficult to handle huge amount of implicit knowledge and constraints, and cannot exploit such rules for efficient design space exploration; it needs too many explicit design rules; it is difficult to design highly constrained structures such as the periodic inorganic crystals.
In this talk, I will introduce the transformative shift from rational materials design to the data-driven deep generative material design paradigm, in which known materials data are fed to the deep generative models to learn explicit and implicit knowledge of atomic structures and then exploit them for efficient structure generation. This is inspired by the deep learning based Artificial Intelligence Generated Content (AIGC) technologies that have been accelerating in generating authentic images, videos, texts, music, and human voices. Our work shows that designing images and texts shares many characteristics with the task of designing proteins, molecules, and materials, in which building blocks of different levels are assembled together to form specific stable or meaningful structures that satisfy diverse grammatical, physical, chemical or geometric constraints. While Nature has used the physical apparatus of DNA as the information carrier of synthesis rules for protein synthesis and biochemistry through evolution, deep neural networks can also be exploited similarly to achieve Nature's way of material design by learning the designing rules from known materials or from computational simulations. Just as a female frog can give birth to a frog without knowing how a frog is grown from a zygote through a developmental process, we show that our deep generative materials design works in a similar design-without-understanding process.
-----------------------
Bio: Dr. Jianjun Hu is professor of computer science at the University of South Carolina. He directs the Machine Learning and Evolution Laboratory (http://mleg.cse.sc.edu). Dr. Hu received his Ph.D. of computer science in 2004 from Michigan State University in the areas of machine learning and evolutionary computation and then conducted postdoc studies in bioinformatics at Purdue University and University of Southern California from 2004 to 2007. His current research interests include AI for science, machine learning, deep learning, evolutionary algorithms, and their application in material informatics, bioinformatics, health informatics, and automated design synthesis with a total of more than 200 papers. Dr. Hu is the winner of the National Science Foundation Career Award. His research has been funded by NSF, DOE, NIH. He can be reached at jianjunh@cse.sc.edu
Details: https://www.linkedin.com/events/7162579233444192256/about/
Held in person on the third Friday of each month at the AI Institute 1112 Greene St. Columbia, SC 29208
1112 Greene St. Columbia, SC 29208