MarketGen: A Scalable Simulation Platform with Auto-Generated Embodied Supermarket Environments
Abstract
The development of embodied agents for complex commercial environments is hindered by a critical gap in existing robotics datasets and benchmarks, which primarily focus on household or tabletop settings with short-horizon tasks. To address this limitation, we introduce MarketGen, a scalable simulation platform with automatic scene generation for complex supermarket environments. MarketGen features a novel agent-based Procedural Content Generation (PCG) framework. It uniquely supports multi-modal inputs (text and reference images) and integrates real-world design principles to automatically generate complete, structured, and realistic supermarkets. We also provide an extensive and diverse 3D asset library with a total of 1100+ supermarket goods and parameterized facilities assets. Building on this generative foundation, we propose a novel benchmark for assessing supermarket agents, featuring two daily tasks in a supermarket: (1) Checkout Unloading: long-horizon tabletop tasks for cashier agents, and (2) In-Aisle Item Collection: complex mobile manipulation tasks for salesperson agents. We validate our platform and benchmark through extensive experiments, including the deployment of a modular agent system and successful sim-to-real transfer. MarketGen provides a comprehensive framework to accelerate research in embodied AI for complex commercial applications.
Automatic Scene Generation
PCG Workflow
With the layout serving as the blueprint for the overall scene structure, the PCG system automatically instantiates and configures these components. This process consists of three primary stages: Asset Retrieval, Adjustment, and Placement.
Drawing from analyses of real-world supermarket facilities, we deconstruct key infrastructure, particularly shelving systems, into their minimal constituent units. By programmatically adjusting parameters, such as the number of vertical tiers, the spacing, and the unit's depth and length, the system can dynamically assemble a wide variety of shelving configurations
Scene Comparison
MarketGen Benchmark
To evaluate the capabilities of embodied robots within supermarket environments, we establish a dedicated benchmark primarily focused on long-horizon manipulation tasks. The design of this benchmark is grounded in practical applications, drawing inspiration directly from the daily operational duties performed by human supermarket staff.
Experiments
Scene Generation
Layout Generation
Different Combination of Facilities
Auto Product Fulling with different goods categories
Benchmark Experiments
Video demonstration of benchmark experiments.
Additional Generated Scenes
BibTeX
@article{YourPaperKey2024,
title={Your Paper Title Here},
author={First Author and Second Author and Third Author},
journal={Conference/Journal Name},
year={2024},
url={https://your-domain.com/your-project-page}
}