Awesome
Congrats on v4 release!
Thank you! It was a big undertaking, lots of little hurdles. "SMART" training really changed the game on how the model performs - SMART training minimizes the normal next-token cross-entropy plus auxiliary losses on hidden states and weights: an entropy term penalizes output entropy scaled by a learned “knowledge-mass” estimate, a holographic-depth term matches layerwise representation entropy to a 1/depth target profile, a differentiable topology term discourages disconnected/holey latent structure, and a manifold term penalizes projection weights with uneven row/column sums. Together these constraints steer predictions, depth dynamics, latent geometry, and weight structure toward more stable representations than cross-entropy alone... The qualitative samples I've gathered show a clear winner with smart training checkpoint-by-checkpoint, even when loss metrics are higher.
V5 aka Z-Image-Engineer-FINAL will be done training in a few days on a dataset 2x the size of V4 (55k+ more synthetic prompt enhancement pairs!) - If all goes to plan it should be able to "restyle" prompts better than V4 - I also increased system instruction diversity by 35%.
any word on v5 yet? and do reach out to me, wanna discuss another training of interest...
Hit me up on x @bennydaball_og