Search

More Filters

Show More
Add Listing
Go Back

Magma

0
New
Multi-Agent Platforms & Frameworks
Report

Report Abuse

Basic Information

This repository provides a research-grade foundation model and codebase for building, training, evaluating, and deploying multimodal AI agents. It centers on Magma, a vision-language-action foundation model designed to perceive images and videos, reason about spatial-temporal content, and produce goal-driven action plans across digital and physical tasks. The repo includes pretraining and finetuning pipelines, data preprocessing tools, SoM and ToM generation code, inference examples, model checkpoints referenced for Magma-8B, evaluation scripts, and several agent demos (UI, gaming, robot visual planning). It also supplies a FastAPI server and Docker deployment for serving the model as a REST API. The materials are intended for researchers and developers to reproduce experiments, adapt the model for downstream tasks, and prototype multimodal agent behaviors.

Links

Stars
1768
App URL
https://github.com/microsoft/Magma
Github Repository
https://github.com/microsoft/Magma/blob/main/README.md

App Details

Features
Magma combines unified pretraining objectives that bridge text, image, and action modalities and introduces auxiliary tasks named Set-of-Mark and Trace-of-Mark to align outputs. The repo contains large-scale data handling for unlabeled videos and agentic datasets, scripts for pretraining and finetuning (including Open-X and Magma-820K workflows), and tools to generate SoM/ToM traces. Inference is supported via Hugging Face transformers, local model code, and quantized bitsandbytes loading. It supports multi-image and video inputs, benchmarking of latency and memory, evaluation via lmms-eval and SimplerEnv, and ready-to-run agent demos for UI navigation, gaming, and robot planning. A FastAPI server and dockerized deployment enable REST endpoints for image-based predictions and action outputs.
Use Cases
The project provides an end-to-end platform for researchers to develop and evaluate multimodal agent behaviors, accelerating experiments in visual planning, UI navigation, and robotics manipulation. It offers reproducible training and finetuning recipes, data preprocessing utilities, and demo apps that illustrate action grounding and long-horizon planning. The included inference examples and server simplify model integration into prototypes and services, while benchmarking and evaluation scripts help quantify performance and resource tradeoffs. By releasing model checkpoints, processor code, and data-generation tools, the repo lowers the barrier to adapt the foundation model for image/video captioning, question answering, UI automation, and robot control tasks, and serves as a baseline for further multimodal agent research.

Categories

Content Creation (5042)
Monetization & Business (2246)
Development Tools (1407)
Marketing & Social Media (1403)
Education & Career (1380)
Multi-Agent Platforms & Frameworks (1009)
Health & Life (957)
AI Resources (920)
Agents & Automation (778)
Data & Analytics (764)
Productivity Tools (679)
Writing & Editing (666)

Similar Listings

listing image

virtual lab

0 (0 review)
listing image

mnemo

0 (0 review)
listing image

EdgeChains

0 (0 review)
listing image

RAG Agents Accelerator

0 (0 review)
listing image

LLM Zero to Hundred

0 (0 review)

Featured Listings

Tags

Please fill the required fields*