SWAYSTAR123
Swayam Bhanded

Swayam Bhanded

IMAGE GEN RESEARCH LUMA LABS AI INDIA

I'm Swayam Bhanded, a research engineer at Luma Labs AI. My work focuses on making diffusion models more efficient to train and run.

01

Research

I'm the author of Speedrunning ImageNet Diffusion, which introduces SR-DiT (Speedrun Diffusion Transformer) — a framework that combines a bunch of training techniques to achieve a ~360x training speedup on ImageNet-256. Getting 3.14 FID with only a 140M parameter model, rivaling models 5x its size whilst training for significantly fewer iterations.

Grid of 24 images generated by SR-DiT-B/1 on ImageNet-512: birds, animals, interiors, objects and landscapes.
Figure 1: SR-DiT-B/1 samples on ImageNet-512.
Log-scale plot of FID against training iterations. SR-DiT-B/1 sits well below SiT-B/2, SiT-B/2 + REPA and SiT-B/2 + REG at every checkpoint, reaching FID 3.49 at 400K iterations.
Figure 2: Training convergence comparison on ImageNet-256. SR-DiT-B/1 achieves strong performance with substantial convergence speedup.

I'm also a co-author of Abra: Scaling Diffusion Image Training, a Luma paper out of the pretraining and scaling team, and a compute-optimal scaling law study for text-to-image diffusion spanning 1019 to 1022 FLOPs. Diffusion models scale about as predictably as language models do — but compute optimality sits near 200 image tokens per parameter, roughly ten times the Chinchilla prescription.

EMA averaged loss against training compute for six model sizes from 60M to 2B parameters. Each run traces a curve that descends onto a common straight frontier on log axes; gold stars mark each size's compute-optimal point.
Abra, Figure 1: loss against training compute for six model sizes. Stars mark each size's compute-optimal point; the dashed line is the fitted frontier.
Best linear-probe top-1 accuracy rising from 0.43 to 0.67 against image-token training FLOPs, on a log x-axis, with separate fitted curves for 200 and 400 image tokens per parameter.
Abra, Figure 8(c): best linear-probe top-1 accuracy against image-token training compute.

You can see other projects on my GitHub.

I mostly post on X, but I also sometimes write blogposts, such as my recent “Diffusion models are schizophrenia machines”.

02

Previously

Before Luma I was at Bria AI, where I trained FIBO-lite, distilled from their 8B-parameter FIBO model.

Earlier than that I worked at Fuel Labs, where I was a contributor to the Sway smart contract language.

03

Other

I made this AI animated version of Bad Apple a while ago.

It didnt get much views on youtube or X, but managed to get millions of views when reposted by people on chinese social media.

I speak Japanese and Chinese, learning through my own website, languagespeedrun.com. I wrote about the Japanese half of that in N5 to N1 in one year.