2.3K views · 61 reactions | Vision Language Models (VLMs) understand natural language prompts and perform visual question answering. ➡️ https://nvda.ws/4cTW5Ox Learn how you can build VLM-powered visual AI agents for a wide range of apps. #SIGGRAPH2024 | NVIDIA AI | Facebook

High-capacity vision-language models (VLMs) are trained on web-scale datasets, making these systems remarkably good at recognising visual or language patterns and operating across different languages. But for robots to achieve a similar level of competency, they would need to collect robot data, first-hand, across every object, environment, task, and situation. Here, researchers have introduced Robotic Transformer 2 (RT-2), a novel vision-language-action (VLA) model that learns from both web and