An Efficient End-to-End Large Language Model (LLM) Accelerator Chip Optimized for Edge Computing
Technology Introduction:
Our team developed MMA, a high-efficiency accelerator optimized for Transformer architecture and LLM inference. Featuring low power consumption and high performance, it is currently the only solution capable of fully running LLMs on edge devices. Compared to Georgia Tech’s MicroScopiQ, Seoul National University’s Tender, and Korea University’s OPAL, our design is the first to fully realize the vision of LLM on Edge, making secure and real-time AI possible.
Industry Applicability:
LLMs like ChatGPT are widely used in daily life. Our chip design enables real-time LLM inference without relying on the cloud or GPUs, reducing energy use and protecting data privacy. Imagine a future where anyone can run an LLM directly on their smartphone, or interact with a compact personal chatbot anytime, anywhere, completely offline and without worrying about data leaks. The solution not only enhances convenience but also resolves concerns around privacy, bringing AI into everyday life.
Our team developed MMA, a high-efficiency accelerator optimized for Transformer architecture and LLM inference. Featuring low power consumption and high performance, it is currently the only solution capable of fully running LLMs on edge devices. Compared to Georgia Tech’s MicroScopiQ, Seoul National University’s Tender, and Korea University’s OPAL, our design is the first to fully realize the vision of LLM on Edge, making secure and real-time AI possible.
Industry Applicability:
LLMs like ChatGPT are widely used in daily life. Our chip design enables real-time LLM inference without relying on the cloud or GPUs, reducing energy use and protecting data privacy. Imagine a future where anyone can run an LLM directly on their smartphone, or interact with a compact personal chatbot anytime, anywhere, completely offline and without worrying about data leaks. The solution not only enhances convenience but also resolves concerns around privacy, bringing AI into everyday life.
Industrial Applications:
-
National Yang Ming Chiao Tung University
National Yang Ming Chiao Tung University (NYCU) was formed in 2021 through the merger of National Yang Ming University and National Chiao Tung University. Located in Hsinchu, Taiwan, NYCU is a leading institution specializing in technology, engineering, medicine, and social sciences. The university is known for its strengths in research and innovation, particularly in areas such as information technology, biomedicine, and artificial intelligence. NYCU fosters interdisciplinary collaboration, global partnerships, and aims to nurture professionals with strong academic foundations and leadership skills to address societal challenges and contribute to technological advancements.
