Research vision and trajectory

Building World Models through 3D Intelligence

My research asks how machines can reconstruct, generate, and eventually simulate coherent three-dimensional environments. The path connects reliable spatial representations with scalable generative priors: from neural reconstruction and editable scene understanding, through pretrained 3D asset generation, toward open-ended world models for embodied intelligence.

Research directions

World Models and Infinite 3D Worlds

Generate large, continuous environments whose geometry and appearance remain coherent as the world expands.

  • WorldGrow β€” hierarchical, block-based infinite 3D scene generation.
  • WorldAct β€” a project I initiated and mentored (NeurIPS 2026); it turns static 3D worlds into object-centric scenes ready for editing and interaction.

Pretrained and Controllable 3D Generation

Learn scalable 3D priors while exposing control over appearance, semantics, and editable object structure.

  • SUPRA-Tex β€” my latest first-author work; it turns a single image into a PBR 3D asset whose textures stay aligned with the input.
  • UniLat3D β€” unified geometry-appearance latents for single-stage textured asset generation; I led the mesh decoder.
  • TIGON β€” joint text-image conditioned generation.
  • SCULPT β€” co-first-authored and mentored by me; it generates part-aware 3D assets by subtractive composition.

3D Reconstruction and Spatial Representation

Recover high-quality, navigable 3D content from sparse observations and make scene representations useful for downstream interaction.

  • GaussianObject β€” high-quality Gaussian Splatting reconstruction from only four images.
  • NeRFVS β€” free-view indoor synthesis with geometry scaffolds.
  • SA3D β€” interactive segmentation in neural 3D scenes.

Dynamic and Medical Spatial Intelligence

Model deformable scenes and challenging surgical observations where geometry, motion, and reliability matter together.

  • LerPlane β€” fast 4D reconstruction of deformable tissues.
  • EndoGSLAM β€” dense endoscopic reconstruction and tracking.
  • ForPlane β€” efficient dynamic neural representations.

Research trajectory

  1. 2021–2023 Β· RepresentationReliable neural scene representations. NeRFVS, SA3D, and LerPlane explored geometry priors, interactive understanding, and efficient dynamic reconstruction.
  2. 2024 Β· ReconstructionHigh-quality 3D from limited observations. GaussianObject and EndoGSLAM brought Gaussian Splatting to extremely sparse views and medical scenes.
  3. 2025–2026 Β· GenerationScalable and controllable 3D priors. UniLat3D, TIGON, SCULPT, and SUPRA-Tex move from unified representations to multimodal control, editable parts, and image-aligned PBR textures.
  4. 2025 onward Β· Product model R&DXiaoyi 3D foundation model. I own its end-to-end R&D, spanning technical direction, roadmap, experimentation, and core implementation; the model powers the team product Xiaoyi 3D Γ— Pixel Artisan PRO.
  5. 2026 onward Β· World modelsOpen-ended spatial intelligence. WorldGrow and WorldAct connect generative 3D priors to large, extendable, object-centric environments for embodied agents.

Open research and reproducible comparison

Representative projects provide public code or inspectable project pages, repositories, demos, and detailed paper records. The purpose is to make independent inspection and fair comparison easierβ€”not simply to showcase selected results.

  • Public project materials. WorldAct currently provides a repository with project-page assets, demos, and supporting materials; the method implementation has not yet been released.
  • Open implementations. SUPRA-Tex, SCULPT, TIGON, UniLat3D, WorldGrow, GaussianObject, SA3D, and LerPlane expose source code or reproducible implementation artifacts.
  • Verifiable records. Stable paper pages connect each method to its paper, DOI or arXiv identifier, code, available data or models, BibTeX, and research keywords.
  • Substantive baselines. The work covers reconstruction, generation, multimodal control, editable structure, and scene scale, enabling comparisons across a coherent technical trajectory.
  • Machine-readable records. The complete portfolio is available as publication JSON, BibTeX, an Atom feed, and a researcher profile.

What this body of work demonstrates

Together, these projects span world models, 3D generation, spatial intelligence, 3D reconstruction, and embodied AI. Rather than a single isolated result, they form a continuous path from problem formulation and open implementation to large-scale generation and product delivery.

  • Technical depth: contributions span neural representations, Gaussian Splatting, generative 3D latent spaces, image-aligned PBR textures, multimodal conditioning, compositional assets, floor-plan-to-house generation, and interaction-ready object-centric worlds.
  • Research continuity: reconstruction and scene understanding progressively connect to controllable 3D generation and foundation world models.
  • Research-to-product execution: the Xiaoyi 3D foundation model powers the consumer creation app Xiaoyi 3D Γ— Pixel Artisan PRO, which reached more than 40,000 users within 50 hours of launch.
  • Open evaluation: public implementations and stable bibliographic records allow other researchers and teams to inspect, reproduce, and compare the work directly.

Questions I am pursuing

  • How can a foundation world model maintain consistent geometry, appearance, and semantics over long spatial horizons?
  • How can pretrained 3D generators expose compositional structure without sacrificing asset quality?
  • How can reconstruction, generation, and simulation share one scalable spatial representation?
  • How can these models support embodied agents that understand, imagine, and interact with physical environments?