On October 17, 2024, Infineon and Archetype AI introduced the first-ever foundation model capable of understanding real time sensor data. We presented a demo at OktoberTech™ Silicon Valley, Infineon's annual technology forum. Read on to learn more about the event and our approach to building a large behavior model that can reason about events in the physical world.
Infineon is a global semiconductor manufacturer and leader in sensing and IoT. This year’s OktoberTech™ showcased the latest innovations in sensing to over 500 of Infineon’s customers and partners. It also featured keynote speeches on energy, mobility, IoT, sustainability, and of course, artificial intelligence. "AI will be like water or electricity, ubiquitous. We won't think of a company as an AI company, just like we don't think of a company as an Internet company," said Bill Gross, founder of the technology incubator Idealab and a keynote speaker.
At last year’s OktoberTech, Archetype AI and Infineon announced our strategic partnership to accelerate the future development of sensor-based chips with AI functionality. This year, we followed up to demonstrate Newton, Archetype AI’s large behavior model that combines real time sensor data with natural language. This fusion allows devices to become aware of what's going on in the world around them. Smart home devices, for example, can then understand context to provide proactive assistance and more helpful responses to users' queries. Today these assistants only know what a user explicitly tells them, so they are missing a rich contextual understanding of real world user activity and intent.
"We're honored to partner with Infineon to bridge the gap between devices and real world context based on Newton's ability to interpret multimodal sensor data," said Dr. Ivan Poupyrev, CEO of Archetype AI. "Using Newton with Infineon's radars and microphones, it's possible for developers to rapidly build interactive Physical AI applications for their sensor-based use cases with minimal fine-tuning. Any developer can create unique user experiences for phones, smart home devices, or appliances."
The OktoberTech demo focused on Newton's ability to interpret real world situations by combining radar-based understanding of human activity with microphone-based understanding of auditory context. No cameras or other visual data were necessary for this demo, which is key for its applicability to private smart home use cases. In the demo scenario, Newton powers a home assistant in a kitchen setting. At one moment, it’s assisting a user through their morning routine — in the next, it’s helping that user keep their family safe when the smoke alarm goes off.
This demo highlights the potential of AI to understand both behavior and context in the physical world. Different actions like walking or running can have different meanings depending on the situation, such as whether an alarm is on or off. Each moment in the demo represents a turning point where changes in events (as detected by sensors) reshape the entire scenario — and what the smart home assistant should do to help.
“We're thrilled with the progress we've made in our partnership with Archetype AI over the past year. The collaboration has been incredibly fruitful, and we're excited to showcase the results of our joint efforts,” said Laurent Remont, Business Line Head for RF & Sensors at Infineon. “Our demo is now up and running, and it's amazing to see how seamlessly Infineon’s XENSIV™ Radar and MEMS microphones, enhanced with AI functionalities, are working together to make devices more intuitive, efficient, and sustainable. We're confident that this technology will have a significant impact on various industries, and we're looking forward to continuing our work with Archetype AI to drive innovation and decarbonization."
Curious to learn more about Newton? Sign up for our newsletter to hear about future events.
