feat: add logit-normal scheduler

[stduhpf]

Summary

Adds support for the logit-normal schedule that supposed to be the default noise schedule for Ideogram4 (see https://github.com/ideogram-oss/ideogram4/blob/main/docs/pipeline.md#noise-schedule).

It's now set as default scheduler when running an Ideogram4 model.

Related Issue / Discussion

N/A

Additional Information

Uses Acklam's Algorithm to approximate the Inverse Normal CDF to avoid adding a boost dependency.

This scheduler has two tweakable parameters: muand std. Here are the values recommended for Ideogram (source: https://github.com/ideogram-oss/ideogram4/blob/main/src/ideogram4/sampler_configs.py)

• 48 steps: mu=0.0, std = 1.5
• 20 steps: mu=0.0, std = 1.75 (default)
• 12 steps: mu=0.5, std = 1.75

They are set via the --extra-sample-args parameter. For example, for 12 steps generations, you should add --extra-sample-args "mu=0.5,std=1.75" to the cli args.

12 steps

Discrete scheduler	Logit_normal scheduler (mu=0.5,std=1.75)

20 steps

Discrete scheduler	Logit_normal scheduler (mu=0,std=1.75)

48 steps

Discrete scheduler	Logit_normal scheduler (mu=0,std=1.5)

Checklist

• I have read and confirmed this PR follows the contribution guidelines.

[stduhpf]

I played around with the formula a bit, by carefully chosing the mean and std parameters, it seems to be possible to approximate roughly any other schedule...

Examples:

mean	std	closest schedule
0	1.7	discrete
0	2.6	smoothstep
-3.3	2.6	exponential

https://www.desmos.com/calculator/t8ji3ml3aq

For some reason, the recommended parameters seem to give subpar results compared to discrete (the current default), especially at high resolution (at 512x512 is should be pretty close but the resolution-aware thing steers it away at higher resolution).

Maybe it's just that I'm comparing with prompts/seeds that were cherry-picked because they were working great with the discrete scheduler.

[stduhpf]

@leejet @Green-Sky @rmatif @daniandtheweb or anyone who actualy understand this stuff, could you take a look?

I'm a bit confused by some things, like why is t_to_sigma not the inverse of sigma_to_t for Flow denoisers, and why some schedulers ignore t_to_sigma but others use it etc...

[stduhpf]

https://github.com/Comfy-Org/ComfyUI/blob/master/comfy_extras/nodes_ideogram4.py

Looks like they flip the order and do 1-x here, I will try that and see how it looks lilke.

Edit: yeah, that's it. I must have missed something when looking at the original reference implementation.

[stduhpf]

PR is pretty much ready to merge, only thing I'd still like a second opinion on is part.

[daniandtheweb]

As far as I know in flow denoisers time = noise so there should be no need to be using any t_to_sigma to calculate the values.

[daniandtheweb]

@leejet @Green-Sky @rmatif @daniandtheweb or anyone who actualy understand this stuff, could you take a look?

I'm a bit confused by some things, like why is t_to_sigma not the inverse of sigma_to_t for Flow denoisers, and why some schedulers ignore t_to_sigma but others use it etc...

It may not be the best explanation as I'm not really an expert on it but I'll try.

For your first question t_to_sigma is not the inverse of sigma_to_t for flow denoisers because of the need to add a shift during the denoising phase. If they were exact inverses they'd cancel the shift out.

About t_to_sigma being ignored by some schedulers I think it's mostly because some schedulers need to lookup the noise level while others calculate the noise curve independently.

[stduhpf]

For your first question t_to_sigma is not the inverse of sigma_to_t for flow denoisers because of the need to add a shift during the denoising phase. If they were exact inverses they'd cancel the shift out.

I see, but even when there is no shifting, there's an off-by-one difference. I guess it's close enough?

some schedulers need to lookup the noise level while others calculate the noise curve independently.

Hmm. When the scheduler is calculating the noise curve independantly, doesn't it break compatibility with the different kinds of denoisers (or settings like flow shift)?

[wbruna]

I had notice that difference before (#1372). Note they are called in very different contexts; I suspect they are not really intended to be inverses of each other, despite their names.

or settings like flow shift

Hm. Yeah, I believe the schedulers should always call t_to_sigma, at least to apply the shift.

[daniandtheweb]

I see, but even when there is no shifting, there's an off-by-one difference. I guess it's close enough?

From what I understand: yes, it's close enough.

Hmm. When the scheduler is calculating the noise curve independantly, doesn't it break compatibility with the different kinds of denoisers (or settings like flow shift)?

I haven't tested it but in theory not all schedulers should work for flow models.