Stable Diffusion 3 medium #SD3

Stable Diffusion 3 medium #SD3

1. It’s a good model with a blend of speed & performance
2. It was iteratively trained by Robin’s team & rest of Stability AI team to blend wide use but also be good out of the box
3. It’s clear some of the safety alignment stuff got wonky at the last stage, we’ve seen this with DALL-E, Google models etc
4. In particular it doesn’t like folk laying on grass. The safety stuff is needed due to regulatory obligations & more but is an art versus a science. Stability AI models also get way more use than any others so obligation is heavier – you may not care if models are used in bad ways but I can tell you it gave me sleepless nights.
5. Unlike DALL-E or Imagen etc the model weights are available and while being great for the vast majority of stuff can be adjusted to fix the issues as well as become even better.
6. Model perturbation, ELLA, MoE’ing, prompt augmentation, SPIN’ing & others are likely to have good results
7. This will also emphasise how SD3 will fit nicely in pipelines, just like the ultra API is a pipeline like Midjourney, dall-e, ideogram and other image “models”
8. The new license changes seem a bit confusing but from responses seem fine for creators as they basically cover inference services. Do give feedback.
9. It’s nice there are optimised versions for various hardware. Tuning will take some time to get right as it’s a bit difference, but I think we will see loads more leg work and impact with loras and ip adapters etc due to the quality of the base model, Vae upgrade etc