TRELLIS
TRELLIS is a deep learning-based 3D generation model that takes a single image or text prompt as input and generates multi-format 3D assets with rich 3D geometry and textures. Similar to how a natural language processing model like GPT understands context and outputs various forms of text, this tool precisely transforms the input source into a unified, structured latent representation (Structured Latents, SLAT) and then simultaneously decodes it into various downstream 3D representation formats. Existing 3D generation models typically focus on one of the following: NeRF, 3D Gaussian Splatting, or Polygon Mesh.
TRELLIS is a deep learning-based 3D generation model that takes a single image or text prompt as input and generates multi-format 3D assets with rich 3D geometry and textures. Similar to how a natural language processing model like GPT understands context and outputs various forms of text, this tool precisely transforms the input source into a unified, structured latent representation (SLAT) and then simultaneously decodes it into various downstream 3D representation formats. Existing 3D generation models have been limited to a single format among NeRF, 3D Gaussian Splatting, and Polygon Mesh, or have suffered significant quality degradation during format conversion. To fundamentally overcome this problem of "format fragmentation," TRELLIS introduces a Rectified Flow Transformer architecture that encompasses multiple formats.
The Rectified Flow Transformer backbone is designed with 1.2B to 2.0B parameters, enabling it to extract undistorted geometric shapes and detailed PBR textures in a very short time, typically within 10 seconds. It significantly reduces the geometric distortion and blurring that existing meshing models experience, and it accurately reconstructs objects with complex topological structures. The generated assets exhibit excellent versatility, ensuring that they can be immediately imported and used in commercial pipelines such as game development engines or virtual reality renderers without conversion loss.
Researchers in the medical and biotechnology visualization fields can quickly prototype high-precision 3D structural data based on a single graphic example or cross-sectional image. By inputting 2D screenshots of proteins predicted using AlphaFold or anatomical cross-sectional structures, they can generate assets for a web-based 3D viewer with real-time rendering capabilities or polygon meshes for surgical simulations, significantly streamlining the research workflow.
💻 System Requirements
NVIDIA GPU 16GB or higher required (officially validated in enterprise environments such as A100, A6000, etc.)
Approximately 5GB for model weight installation, and 15GB or more including the entire library and temporary compilation.
⚡ Installation
4-1. Quick Start
Clone repository and submodulesgit clone --recurse-submodules https://github.com/microsoft/TRELLIS.git cd TRELLIS
Create Conda virtual environment and install essential compilation dependencies in one go. ./setup.sh --new-env --basic --xformers --flash-attn --diffoctreerast --spconv --mipgaussian --kaolin --nvdiffrast
4-2. Detailed installation
Example of a Basic Inference Script Using the Python API (minimal_example.py)import os os.environ['SPCONV_ALGO'] = 'native' # Reduce benchmark time in single execution mode
from PIL import Image from trellis.pipelines import TrellisImageTo3DPipeline from trellis.utils import render_utils, postprocessing_utils
1. Load the pre-trained image-3D pipeline and allocate GPU resources.pipeline = TrellisImageTo3DPipeline.from_pretrained("microsoft/TRELLIS-image-large") pipeline.cuda()
2. Load single image for analysisimage = Image.open("input_sample.png")
3. 3Driving the D Asset Creation Processoutputs = pipeline.run(image, seed=1)
4. Apply simplification filters and texture map sizes to extract the final GLB mesh file.glb = postprocessing_utils.to_glb( outputs['gaussian'][0], outputs['mesh'][0], simplify=0.95, # 95% Polygon Simplification texture_size=1024 # Texture resolution 1024x1024 settings ) glb.export("output_mesh.glb")
FAQ
What is TRELLIS?
TRELLIS is a deep learning-based 3D generation model that takes a single image or text prompt as input and generates multi-format 3D assets with rich 3D geometry and textures. Similar to how a natural language processing model like GPT understands context and outputs various forms of text, this tool precisely transforms the input source into a unified, structured latent representation (SLAT) and then simultaneously decodes it into various downstream 3D representation formats. Existing 3D generation models have been limited to a single format among NeRF, 3D Gaussian Splatting, and Polygon Mesh, or have suffered significant quality degradation during format conversion. To fundamentally overcome this problem of "format fragmentation," TRELLIS introduces a Rectified Flow Transformer architecture that encompasses multiple formats. The Rectified Flow Transformer backbone is designed with 1.2B to 2.0B parameters, enabling it to extract undistorted geometric shapes and detailed PBR textures in a very short time, typically within 10 seconds. It significantly reduces the geometric distortion and blurring that existing meshing models experience, and it accurately reconstructs objects with complex topological structures. The generated assets exhibit excellent versatility, ensuring that they can be immediately imported and used in commercial pipelines such as game development engines or virtual reality renderers without conversion loss. Researchers in the medical and biotechnology visualization fields can quickly prototype high-precision 3D structural data based on a single graphic example or cross-sectional image. By inputting 2D screenshots of proteins predicted using AlphaFold or anatomical cross-sectional structures, they can generate assets for a web-based 3D viewer with real-time rendering capabilities or polygon meshes for surgical simulations, significantly streamlining the research workflow.
When should I use TRELLIS?
TRELLIS is a deep learning-based 3D generation model that takes a single image or text prompt as input and generates multi-format 3D assets with rich 3D geometry and textures. Similar to how a natural language processing model like GPT understands context and outputs various forms of text, this tool precisely transforms the input source into a unified, structured latent representation (Structured Latents, SLAT) and then simultaneously decodes it into various downstream 3D representation formats. Existing 3D generation models typically focus on one of the following: NeRF, 3D Gaussian Splatting, or Polygon Mesh.
📝 Update Notes
No update notes yet.
🧪 Related Code of Life
No related Code of Life posts yet.