Few examples from the demo. You can generate various styles from a single prompt. You can either pick semantic matching images, or not and get unexpected results.

[1] unsplash.com/photos/a-boa...
[2] unsplash.com/photos/mount...
[3] unsplash.com/photos/man-i...
[4] unsplash.com/photos/macro...

Thanks to my co-authors Andreas Engelhardt, Simon Donné, Varun Jampani

Also check out the HF demo huggingface.co/spaces/stabi..., the code github.com/Stability-AI..., and the explainer youtu.be/ckcSgf0s-jI

This can be used for multiple applications such as color matching or diffusion guidance. Here, we showcase the diffusion process of generating a medieval house with the reference to the right.

Variance in MC is quite common in computer graphics so we combined ReSTIR -- more precisely the weighted reservoir sampling -- with SWD to keep more impactful random directions in the optimization.

I’ll also be sharing these and other works at the AI4CC workshop on the 12th at 11:00. ai4cc.net

There was SV3D where we mostly discarded any SDS loss (only for unseen areas). I was mainly working on the 3D part and it required quite a few tricks to make it work. sv3d.github.io

3️⃣ MARBLE: Edit materials effortlessly using simple CLIP feature manipulation, supporting exemplar-based interpolation or parametric edits across various styles. Check it out: marblecontrol.github.io

2️⃣ SPAR3D (follow-up to SF3D): Integrates a fast point diffusion module, enhancing depth, backside modeling, and enabling easier editing. Project page: spar3d.github.io

1️⃣ SF3D: Generate textured, UV-unwrapped 3D assets with additional materials incredibly fast (<0.3s)! More details here: stable-fast-3d.github.io

Check out the HF demo to test the model: huggingface.co/spaces/stabi.... The model (huggingface.co/stabilityai/...) is also available with code and Comfy Nodes github.com/Stability-AI.... We also have a project page available at spar3d.github.io

One neat implication is that we can edit the point cloud to fix missing features or wrong scaling. We even created a small gradio component for simple edits in the demo (pypi.org/project/grad...)

A single procedural modeling system is a huge undertaking when you aim for a high quality level. Take speed tree for example which combines procedural aspects with hand authored elements and it’s an entire company dedicated to that.

Yes I agree for certain things it can work. Simple cities (Manhattan style) and natural landscapes are rather well fitting and are explored heavily in video games already. Going for interiors or any object is another beast.

The realistic rendering is not the problem and even full path tracing scenes is doable for room scale scenes on GPU. It still requires some denoising tho as otherwise rendering times are too long to generate any meaningful amount of data. But even then data is the bottleneck

It’s hard to scale 3D data similarly to image or video. We run around with capable cameras all the time. Only few people can model 3D and it’s takes time and isn’t offered for free (rightfully). So even if we would pay all artists in the world, we still won’t hit the scale of image and video.

I recently went with recreating the rooms in Blender. A lot of furniture websites now have 3D viewers and you can download the models from devtools. They are also metric sized. Then blender becomes sims pro and you can iterate quite fast.

Would love to be added too ;)

bsky.app/profile/cspr... markboss.me/publication/... :D