Multimodal TAIDE Taiwanese-English-Japanese Interactive Learning Robot
Technology Introduction:
1. NUTN–Jen-Pin–NCHC TAIDE Taiwanese–English–Japanese AI Learning Companion SystemThis exhibit presents a multilingual learning application that integrates teaching materials for Taiwanese, the TAIDE Large Language Model (LLM), and generative AI (GAI) technologies. Visitors can experience interactive activities such as "Speak Taiwanese, Learn English, and Communicate in Japanese," as well as multimodal learning functions that generate AI images from spoken Taiwanese, English, or Japanese.
2. Taiwanese–English–Japanese Human–Machine Co-Learning Online Platform
This exhibit demonstrates a multilingual human–machine co-learning approach that is not limited to specific textbooks or learning environments. Through hands-on use of the online platform, visitors can explore how learners practice Taiwanese, English, and Japanese through interactive activities based on real-life situations.
3. NUTN TAIDE Taiwanese–English–Japanese Conversational AI Robot
This exhibit showcases a multimodal interactive application that integrates the TAIDE LLM with a physical robot. Visitors can operate the robot and engage in real-time conversations using Chinese, Taiwanese, English, or Japanese.
Industrial Applications:
1. Intelligent EducationIntegrate multilingual interactions in Taiwanese, English, and Japanese with generative AI technologies to promote the internationalization of local language learning.
2. Leisure and Tourism
Empower tour guides and food-ordering robots to provide enhanced multilingual services in hotels, accommodation facilities, restaurants, and local tourist attractions.
3. Industrial Training
Support deployment in small and medium-sized enterprises (SMEs) and manufacturing environments, helping new employees quickly acquire industry-specific terminology and operational knowledge.
4. Cultural Preservation
Support the digital archiving and preservation of local languages and cultural heritage in Malaysia and other Southeast Asian countries.
5. Healthcare
Customize automatic speech recognition (ASR) models for healthcare and long-term care settings, providing voice-based education and interactive support to help older adults manage chronic diseases and medications.
National University of Tainan、JEN-PIN ENTERPRISE CO., LTD.、National Institutes of Applied Research、Academia Sinica
The Multimodal TAIDE Taiwanese–English–Japanese Interactive Learning Robot was developed under a research project funded by Taiwan's National Science and Technology Council (NSTC). It utilizes a fine-tuned version of OpenAI Whisper-Large-V3-Turbo (openai/whisper-large-v3-turbo) and is part of the NSTC TAIDE Taiwanese–English Family Pilot Project. In collaboration with Jen-Pin Co. Ltd., the system was fine-tuned using elementary and seconday school teaching materials, along with students' learning data, to enhance Taiwanese speech recognition for the Jen-Pin textbook series. Furthermore, in partnership with the National Center for High-Performance Computing (NCHC) of the National Applied Research Laboratories (NARLabs), the project leverages NCHC's computing infrastructure and the TAIDE fine-tuned models developed by the Institute of Information Science (IIS), Academia Sinica to jointly build an AI-powered human–machine collaborative learning robot for Taiwanese, English, and Japanese language education in elementary and seconday schools.
