diff --git a/README.md b/README.md index 5143ca5..c0953e8 100644 --- a/README.md +++ b/README.md @@ -4,9 +4,9 @@ [![πŸ“– Documentation](https://img.shields.io/badge/πŸ“–-Documentation-blue)](https://horizonrobotics.github.io/EmbodiedGen/docs/) [![GitHub](https://img.shields.io/badge/GitHub-EmbodiedGen-black?logo=github)](https://github.com/HorizonRobotics/EmbodiedGen) [![πŸ“„ arXiv](https://img.shields.io/badge/πŸ“„-arXiv-b31b1b)](https://arxiv.org/abs/2506.10600) -[![πŸŽ₯ Video](https://img.shields.io/badge/πŸŽ₯-Video-red)](https://www.youtube.com/watch?v=rG4odybuJRk) +[![πŸŽ₯ Video](https://img.shields.io/badge/πŸŽ₯-Video-red)](https://youtu.be/swjDFEvjn14) [![πŸ€— Dataset](https://img.shields.io/badge/πŸ€—-Dataset-blue)](https://huggingface.co/datasets/HorizonRobotics/EmbodiedGenData) -[![中文介绍](https://img.shields.io/badge/中文介绍-07C160?logo=wechat&logoColor=white)](https://mp.weixin.qq.com/s/HH1cPBhK2xcDbyCK4BBTbw) + [![πŸ€— Hugging Face](https://img.shields.io/badge/πŸ€—-EmbodiedGen_Asset_Gallery-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Gallery-Explorer) [![πŸ€— Hugging Face](https://img.shields.io/badge/πŸ€—-Image_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Image-to-3D) [![πŸ€— Hugging Face](https://img.shields.io/badge/πŸ€—-Text_to_3D_Demo-blue)](https://huggingface.co/spaces/HorizonRobotics/EmbodiedGen-Text-to-3D) @@ -15,7 +15,7 @@ > **From intent to executable 3D worlds.** > ***EmbodiedGen*** compiles language, images, and edit commands into **simulation-ready 3D worlds** β€” physically plausible assets, large-scale scenes, and task-driven interactive environments, deployable across every major robotics simulator. -EmbodiedGen V2 Overview +EmbodiedGen V2 Overview --- @@ -273,8 +273,7 @@ If you use EmbodiedGen in your research or projects, please cite: ## πŸ™Œ Acknowledgement EmbodiedGen builds upon the following amazing projects and models: -🌟 [Trellis](https://github.com/microsoft/TRELLIS) | 🌟 [Hunyuan-Delight](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) | 🌟 [Segment Anything](https://github.com/facebookresearch/segment-anything) | 🌟 [Rembg](https://github.com/danielgatis/rembg) | 🌟 [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | 🌟 [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | 🌟 [Kolors](https://github.com/Kwai-Kolors/Kolors) | 🌟 [Aesthetic Score](http://captions.christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html) | 🌟 [Pano2Room](https://github.com/TrickyGo/Pano2Room) | 🌟 [Diffusion360](https://github.com/ArcherFMY/SD-T2I-360PanoImage) | 🌟 [Kaolin](https://github.com/NVIDIAGameWorks/kaolin) | 🌟 [diffusers](https://github.com/huggingface/diffusers) | 🌟 [gsplat](https://github.com/nerfstudio-project/gsplat) | 🌟 [ChatGPT](https://platform.openai.com/docs/models/gpt-4o) | 🌟 [SD3.5](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 🌟 [ManiSkill](https://github.com/haosulab/ManiSkill) | 🌟 [SAM3D](https://github.com/facebookresearch/sam-3d-objects) | 🌟 [infinigen](https://github.com/princeton-vl/infinigen) -🌟 [Trellis](https://github.com/microsoft/TRELLIS) | 🌟 [Hunyuan-Delight](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) | 🌟 [Hunyuan3D-Part](https://github.com/Tencent-Hunyuan/Hunyuan3D-Part) | 🌟 [GraspGen](https://github.com/NVlabs/GraspGen) | 🌟 [Segment Anything](https://github.com/facebookresearch/segment-anything) | 🌟 [Rembg](https://github.com/danielgatis/rembg) | 🌟 [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | 🌟 [Stable Diffusion x4](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler) | 🌟 [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | 🌟 [Kolors](https://github.com/Kwai-Kolors/Kolors) | 🌟 [ChatGLM3](https://github.com/THUDM/ChatGLM3) | 🌟 [Aesthetic Score](http://captions.christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html) | 🌟 [Pano2Room](https://github.com/TrickyGo/Pano2Room) | 🌟 [Diffusion360](https://github.com/ArcherFMY/SD-T2I-360PanoImage) | 🌟 [Kaolin](https://github.com/NVIDIAGameWorks/kaolin) | 🌟 [diffusers](https://github.com/huggingface/diffusers) | 🌟 [gsplat](https://github.com/nerfstudio-project/gsplat) | 🌟 [QWEN-2.5VL](https://github.com/QwenLM/Qwen2.5-VL) | 🌟 [GPT4o](https://platform.openai.com/docs/models/gpt-4o) | 🌟 [SD3.5](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 🌟 [ManiSkill](https://github.com/haosulab/ManiSkill) | 🌟 [SAM3D](https://github.com/facebookresearch/sam-3d-objects) | 🌟 [infinigen](https://github.com/princeton-vl/infinigen) +🌟 [Trellis](https://github.com/microsoft/TRELLIS) | 🌟 [Hunyuan-Delight](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) | 🌟 [Hunyuan3D-Part](https://github.com/Tencent-Hunyuan/Hunyuan3D-Part) | 🌟 [GraspGen](https://github.com/NVlabs/GraspGen) | 🌟 [Segment Anything](https://github.com/facebookresearch/segment-anything) | 🌟 [Rembg](https://github.com/danielgatis/rembg) | 🌟 [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | 🌟 [Stable Diffusion x4](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler) | 🌟 [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | 🌟 [Kolors](https://github.com/Kwai-Kolors/Kolors) | 🌟 [ChatGLM3](https://github.com/THUDM/ChatGLM3) | 🌟 [Aesthetic Score](http://captions.christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html) | 🌟 [Pano2Room](https://github.com/TrickyGo/Pano2Room) | 🌟 [Diffusion360](https://github.com/ArcherFMY/SD-T2I-360PanoImage) | 🌟 [Kaolin](https://github.com/NVIDIAGameWorks/kaolin) | 🌟 [diffusers](https://github.com/huggingface/diffusers) | 🌟 [gsplat](https://github.com/nerfstudio-project/gsplat) | 🌟 [QWEN-2.5VL](https://github.com/QwenLM/Qwen2.5-VL) | 🌟 [SD3.5](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 🌟 [ManiSkill](https://github.com/haosulab/ManiSkill) | 🌟 [SAM3D](https://github.com/facebookresearch/sam-3d-objects) | 🌟 [infinigen](https://github.com/princeton-vl/infinigen) --- diff --git a/apps/image_to_3d.py b/apps/image_to_3d.py index 09158cb..3e92b4e 100644 --- a/apps/image_to_3d.py +++ b/apps/image_to_3d.py @@ -83,7 +83,7 @@ def preprocess_example_image( πŸ’» GitHub - + πŸŽ₯ Video

diff --git a/apps/text_to_3d.py b/apps/text_to_3d.py index 3f63456..5e99073 100644 --- a/apps/text_to_3d.py +++ b/apps/text_to_3d.py @@ -63,7 +63,7 @@ πŸ’» GitHub - + πŸŽ₯ Video

diff --git a/apps/texture_edit.py b/apps/texture_edit.py index 01afe13..a03eaf4 100644 --- a/apps/texture_edit.py +++ b/apps/texture_edit.py @@ -64,15 +64,13 @@ def active_btn_by_content(mesh_content: gr.Model3D, text_content: gr.Textbox): πŸ’» GitHub - + πŸŽ₯ Video

🎨 Generate visually rich textures for 3D mesh. - """.format( - VERSION=VERSION - ), + """.format(VERSION=VERSION), elem_classes=["header"], ) diff --git a/apps/visualize_asset.py b/apps/visualize_asset.py index 4161967..b82b12a 100644 --- a/apps/visualize_asset.py +++ b/apps/visualize_asset.py @@ -56,6 +56,13 @@ HF_REPO_ID = "HorizonRobotics/EmbodiedGenData" HF_LOCAL_DIR = "EmbodiedGenData" CAMERA_ZOOM = 3.2 +# Gradio's default per-event concurrency limit is 1, which would make every +# user's HF snapshot_download (potentially several seconds) serialize behind +# whichever user clicked first. Give the HF-fetching events their own shared +# pool so several users can download/preview assets in parallel, bounded to +# avoid hammering the HF API with too many simultaneous requests. +HF_IO_CONCURRENCY_ID = "hf_asset_io" +HF_IO_CONCURRENCY_LIMIT = 6 # Compatible with huggingface space zero GPU import spaces @@ -124,6 +131,7 @@ def ensure_hf_files(rel_patterns: list[str]) -> None: css = """ .gradio-container .gradio-group { box-shadow: 0 2px 4px rgba(0,0,0,0.05) !important; } #asset-gallery { border: 1px solid #E5E7EB; border-radius: 8px; padding: 8px; background-color: #F9FAFB; } +#download-row button { white-space: nowrap; } """ lighting_css = """ @@ -724,13 +732,33 @@ def download_mjcf(asset_dir: str, req: gr.Request) -> str | None: return zip_path +def download_affordance(asset_dir: str, req: gr.Request) -> str | None: + """Package the ``affordance/`` folder.""" + if not asset_dir or not os.path.isdir(asset_dir): + gr.Warning("Please select an asset first.") + return None + + gr.Info("⏳ Preparing affordance asset for download, please wait...") + ensure_hf_files([f"{_rel_dir(asset_dir)}/affordance/**"]) + affordance_dir = os.path.join(asset_dir, "affordance") + if not os.path.isdir(affordance_dir): + gr.Warning("No affordance folder available for this asset.") + return None + + zip_path, uid, zip_name = _fmt_zip_path(asset_dir, "affordance", req) + _zip_items(zip_path, [(affordance_dir, "affordance")]) + gr.Info(f"βœ… {zip_name} is ready.") + return zip_path + + # Download buttons keyed by format; order matches the outputs list used when # locking/unlocking them together. -_DL_KEYS = ("urdf", "usd", "mjcf") +_DL_KEYS = ("urdf", "usd", "mjcf", "affordance") _DL_LABELS = { "urdf": "⬇️ Download URDF", "usd": "⬇️ Download USD", "mjcf": "⬇️ Download MJCF", + "affordance": "⬇️ Affordance", } @@ -796,7 +824,7 @@ def end_session(req: gr.Request) -> None: πŸ’» GitHub - + πŸŽ₯ Video

@@ -931,7 +959,7 @@ def end_session(req: gr.Request) -> None: urdf_file = gr.Textbox( label="URDF File Path", interactive=False, lines=2 ) - with gr.Row(): + with gr.Row(elem_id="download-row"): # DownloadButtons that build their zip into their own value # on click; a chained JS handler then triggers the browser # download from that value (one-click). If the JS ever @@ -952,6 +980,11 @@ def end_session(req: gr.Request) -> None: variant="primary", interactive=False, ) + affordance_dl_btn = gr.DownloadButton( + label="⬇️ Affordance", + variant="primary", + interactive=False, + ) search_button.click( fn=search_assets, @@ -1008,6 +1041,8 @@ def update_assets(p, s, c): gallery.select( fn=show_asset_from_gallery, inputs=[primary, secondary, category, search_box, gallery_df_state], + concurrency_id=HF_IO_CONCURRENCY_ID, + concurrency_limit=HF_IO_CONCURRENCY_LIMIT, outputs=[ (visual_path_state := gr.State()), (collision_path_state := gr.State()), @@ -1030,7 +1065,7 @@ def update_assets(p, s, c): ], ).success( fn=_reset_downloads, - outputs=[urdf_dl_btn, usd_dl_btn, mjcf_dl_btn], + outputs=[urdf_dl_btn, usd_dl_btn, mjcf_dl_btn, affordance_dl_btn], ) # After the zip is built into the button's own value, pass that file value @@ -1050,33 +1085,61 @@ def update_assets(p, s, c): } """ - dl_btns = [urdf_dl_btn, usd_dl_btn, mjcf_dl_btn] + dl_btns = [urdf_dl_btn, usd_dl_btn, mjcf_dl_btn, affordance_dl_btn] # Each flow: lock all buttons (prevent spam clicks) -> build the zip -> # JS-trigger the browser download -> unlock. Selecting another asset # resets the buttons independently (see `.success` above). urdf_dl_btn.click( - fn=lambda: _lock_downloads("urdf"), outputs=dl_btns + fn=lambda: _lock_downloads("urdf"), outputs=dl_btns, queue=False ).then( - fn=download_urdf, inputs=[asset_folder], outputs=[urdf_dl_btn] + fn=download_urdf, + inputs=[asset_folder], + outputs=[urdf_dl_btn], + concurrency_id=HF_IO_CONCURRENCY_ID, + concurrency_limit=HF_IO_CONCURRENCY_LIMIT, ).then(fn=lambda *a: None, inputs=[urdf_dl_btn], js=download_js).then( fn=_unlock_downloads, outputs=dl_btns ) - usd_dl_btn.click(fn=lambda: _lock_downloads("usd"), outputs=dl_btns).then( - fn=download_usd, inputs=[asset_folder], outputs=[usd_dl_btn] + usd_dl_btn.click( + fn=lambda: _lock_downloads("usd"), outputs=dl_btns, queue=False + ).then( + fn=download_usd, + inputs=[asset_folder], + outputs=[usd_dl_btn], + concurrency_id=HF_IO_CONCURRENCY_ID, + concurrency_limit=HF_IO_CONCURRENCY_LIMIT, ).then(fn=lambda *a: None, inputs=[usd_dl_btn], js=download_js).then( fn=_unlock_downloads, outputs=dl_btns ) mjcf_dl_btn.click( - fn=lambda: _lock_downloads("mjcf"), outputs=dl_btns + fn=lambda: _lock_downloads("mjcf"), outputs=dl_btns, queue=False ).then( - fn=download_mjcf, inputs=[asset_folder], outputs=[mjcf_dl_btn] + fn=download_mjcf, + inputs=[asset_folder], + outputs=[mjcf_dl_btn], + concurrency_id=HF_IO_CONCURRENCY_ID, + concurrency_limit=HF_IO_CONCURRENCY_LIMIT, ).then(fn=lambda *a: None, inputs=[mjcf_dl_btn], js=download_js).then( fn=_unlock_downloads, outputs=dl_btns ) + affordance_dl_btn.click( + fn=lambda: _lock_downloads("affordance"), + outputs=dl_btns, + queue=False, + ).then( + fn=download_affordance, + inputs=[asset_folder], + outputs=[affordance_dl_btn], + concurrency_id=HF_IO_CONCURRENCY_ID, + concurrency_limit=HF_IO_CONCURRENCY_LIMIT, + ).then( + fn=lambda *a: None, inputs=[affordance_dl_btn], js=download_js + ).then(fn=_unlock_downloads, outputs=dl_btns) + demo.load(start_session) demo.unload(end_session) diff --git a/docs/documentation/acknowledgement.md b/docs/documentation/acknowledgement.md index 2dd4f29..ec5a399 100644 --- a/docs/documentation/acknowledgement.md +++ b/docs/documentation/acknowledgement.md @@ -1,7 +1,7 @@ # πŸ™Œ Acknowledgement EmbodiedGen builds upon the following amazing projects and models: -🌟 [Trellis](https://github.com/microsoft/TRELLIS) | 🌟 [Hunyuan-Delight](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) | 🌟 [Segment Anything](https://github.com/facebookresearch/segment-anything) | 🌟 [Rembg](https://github.com/danielgatis/rembg) | 🌟 [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | 🌟 [Stable Diffusion x4](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler) | 🌟 [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | 🌟 [Kolors](https://github.com/Kwai-Kolors/Kolors) | 🌟 [ChatGLM3](https://github.com/THUDM/ChatGLM3) | 🌟 [Aesthetic Score](http://captions.christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html) | 🌟 [Pano2Room](https://github.com/TrickyGo/Pano2Room) | 🌟 [Diffusion360](https://github.com/ArcherFMY/SD-T2I-360PanoImage) | 🌟 [Kaolin](https://github.com/NVIDIAGameWorks/kaolin) | 🌟 [diffusers](https://github.com/huggingface/diffusers) | 🌟 [gsplat](https://github.com/nerfstudio-project/gsplat) | 🌟 [QWEN-2.5VL](https://github.com/QwenLM/Qwen2.5-VL) | 🌟 [GPT4o](https://platform.openai.com/docs/models/gpt-4o) | 🌟 [SD3.5](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 🌟 [ManiSkill](https://github.com/haosulab/ManiSkill) | 🌟 [SAM3D](https://github.com/facebookresearch/sam-3d-objects) +🌟 [Trellis](https://github.com/microsoft/TRELLIS) | 🌟 [Hunyuan-Delight](https://huggingface.co/tencent/Hunyuan3D-2/tree/main/hunyuan3d-delight-v2-0) | 🌟 [Hunyuan3D-Part](https://github.com/Tencent-Hunyuan/Hunyuan3D-Part) | 🌟 [GraspGen](https://github.com/NVlabs/GraspGen) | 🌟 [Segment Anything](https://github.com/facebookresearch/segment-anything) | 🌟 [Rembg](https://github.com/danielgatis/rembg) | 🌟 [RMBG-1.4](https://huggingface.co/briaai/RMBG-1.4) | 🌟 [Stable Diffusion x4](https://huggingface.co/stabilityai/stable-diffusion-x4-upscaler) | 🌟 [Real-ESRGAN](https://github.com/xinntao/Real-ESRGAN) | 🌟 [Kolors](https://github.com/Kwai-Kolors/Kolors) | 🌟 [ChatGLM3](https://github.com/THUDM/ChatGLM3) | 🌟 [Aesthetic Score](http://captions.christoph-schuhmann.de/aesthetic_viz_laion_sac+logos+ava1-l14-linearMSE-en-2.37B.html) | 🌟 [Pano2Room](https://github.com/TrickyGo/Pano2Room) | 🌟 [Diffusion360](https://github.com/ArcherFMY/SD-T2I-360PanoImage) | 🌟 [Kaolin](https://github.com/NVIDIAGameWorks/kaolin) | 🌟 [diffusers](https://github.com/huggingface/diffusers) | 🌟 [gsplat](https://github.com/nerfstudio-project/gsplat) | 🌟 [QWEN-2.5VL](https://github.com/QwenLM/Qwen2.5-VL) | 🌟 [SD3.5](https://huggingface.co/stabilityai/stable-diffusion-3.5-medium) | 🌟 [ManiSkill](https://github.com/haosulab/ManiSkill) | 🌟 [SAM3D](https://github.com/facebookresearch/sam-3d-objects) | 🌟 [infinigen](https://github.com/princeton-vl/infinigen) --- diff --git a/docs/documentation/index.md b/docs/documentation/index.md index 040dba2..14b932c 100644 --- a/docs/documentation/index.md +++ b/docs/documentation/index.md @@ -9,7 +9,7 @@ hide: [![GitHub](https://img.shields.io/badge/GitHub-EmbodiedGen-black?logo=github)](https://github.com/HorizonRobotics/EmbodiedGen) [![πŸ“„ arXiv](https://img.shields.io/badge/πŸ“„-arXiv-b31b1b)](https://arxiv.org/abs/2506.10600) [![πŸ€— Dataset](https://img.shields.io/badge/πŸ€—-Dataset-blue)](https://huggingface.co/datasets/HorizonRobotics/EmbodiedGenData) -[![πŸŽ₯ Video](https://img.shields.io/badge/πŸŽ₯-Video-red)](https://www.youtube.com/watch?v=rG4odybuJRk) +[![πŸŽ₯ Video](https://img.shields.io/badge/πŸŽ₯-Video-red)](https://youtu.be/swjDFEvjn14) *EmbodiedGen V2*: An Agentic, Simulation-Ready 3D World Engine for Embodied AI. diff --git a/docs/documentation/install.md b/docs/documentation/install.md index 0c5ade1..c0100f5 100644 --- a/docs/documentation/install.md +++ b/docs/documentation/install.md @@ -38,5 +38,5 @@ Update the API key in file: `embodied_gen/utils/gpt_config.yaml`. You can choose between two backends for the GPT agent: -- **`gpt-4o`** (Recommended) – Use this if you have access to **Azure OpenAI**. -- **`qwen2.5-vl`** – An alternative with free usage via OpenRouter, apply a free key [here](https://openrouter.ai/settings/keys) and update `api_key` in `embodied_gen/utils/gpt_config.yaml` (50 free requests per day) +- **`gpt-5.4`** (or Higher, Recommended) – Use this if you have access to **Azure OpenAI**. +- **`gemma-4-31b`** – A free multimodal alternative (`google/gemma-4-31b-it:free`) via OpenRouter, apply a free key [here](https://openrouter.ai/settings/keys) and update `api_key` in `embodied_gen/utils/gpt_config.yaml`. Free-tier availability on OpenRouter can change over time β€” check the [model page](https://openrouter.ai/google/gemma-4-31b-it:free) before relying on it. diff --git a/docs/landing/assets/img/logo_horizon.png b/docs/landing/assets/img/logo_horizon.png new file mode 100644 index 0000000..462d859 Binary files /dev/null and b/docs/landing/assets/img/logo_horizon.png differ diff --git a/docs/landing/index.html b/docs/landing/index.html index 622f8ed..85ba0a2 100644 --- a/docs/landing/index.html +++ b/docs/landing/index.html @@ -47,6 +47,7 @@ +

EmbodiedGenV2

@@ -70,7 +71,7 @@

EmbodiedGenPaper Code - + Video HuggingFace @@ -450,7 +451,7 @@

BibTeX