---
**Author:**  [@keturn](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fketurn)%5C
**Post date:** [August 20, 2023, 5:20pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautoencoderkl-scaling-factor-and-vaeimageprocessor%2F51367%2F3 "2023-08-20T17:20:12Z")
So the deprecation message is a misdirect, and the class it references does not replace it?
In that case, why deprecate the `decode_latents` method at all?
> [@madebyollin](#):
>
> Diffusers applies `vae.config.scaling_factor` _manually_ within the image generation pipelines,
Is there ever a time when one would call `vae.encode` or `vae.decode` _without_ applying this scaling factor?
---
**Author:**  [@madebyollin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmadebyollin)%5C
**Post date:** [August 20, 2023, 5:37pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautoencoderkl-scaling-factor-and-vaeimageprocessor%2F51367%2F4 "2023-08-20T17:37:52Z")
> So the deprecation message is a misdirect, and the class it references does not replace it?
Kinda. [`decode_latents`](https://github.com/huggingface/diffusers/blob/main/src/diffusers/pipelines/stable_diffusion/pipeline_stable_diffusion.py#L428) was doing several things:
1. Applying the `scaling_factor`
2. Running the VAE decoder
3. Rescaling decoded outputs from [-1, 1] → [0, 1] and clamping
4. Converting tensor format from NCHW torch GPU to NHWC numpy CPU.
The `VaeImageProcessor` only does [steps 3 and 4](https://github.com/huggingface/diffusers/blob/main/src/diffusers/image_processor.py#L239) of that, so it’s a partial replacement.
> Is there ever a time when one would call `vae.encode` or `vae.decode` _without_ applying this scaling factor?
I’m not aware of any (this is why TAESD does not have a `scaling_factor`). I suspect the separate `scaling_factor` is inherited from Stability / CompVis code where they treat the `scaling_factor` as a [parameter of the diffusion model](https://github.com/Stability-AI/generative-models/blob/477d8b9a7730d9b2e92b326a770c0420d00308c9/sgm/models/diffusion.py#L118) (i.e. “how much did we scale the VAE latents down when training this particular diffusion model?”) rather than as a property of the VAE.
---
**Author:**  [@keturn](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fketurn)%5C
**Post date:** [August 23, 2023, 12:03am UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautoencoderkl-scaling-factor-and-vaeimageprocessor%2F51367%2F5 "2023-08-23T00:03:30Z")
Right. Given that (a) misleading deprecation warnings are bad, and (b) methods that never return the correct values are bad, I request that we either
1. Complete the migration of the VAE `scaling_factor` away from the Pipeline class and in to the Autoencoder class. Always apply it to the values leaving `encode` and going in to `decode`. Or,
2. Provide methods wrapping the autoencoder that do the necessary thing. This could mean keeping `decode_latents` (and a corresponding `encode`) method on the Pipeline, or it could mean adding that scaling to the VaeImageProcessor’s preprocessing and postprocessing methods.
What’ll it be?
---
**Author:**  [@madebyollin](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fmadebyollin)%5C
**Post date:** [August 28, 2023, 3:16pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautoencoderkl-scaling-factor-and-vaeimageprocessor%2F51367%2F6 "2023-08-28T15:16:39Z")
It looks like the “do scaling in encode / decode” approach was already proposed in [[[wip] init scale\_value on vae by williamberman · Pull Request #1515 · huggingface/diffusers · GitHub](https://github.com/huggingface/diffusers/pull/1515)] but rejected in favor of [[make scaling factor a config arg of vae/vqvae by patil-suraj · Pull Request #1860 · huggingface/diffusers · GitHub](https://github.com/huggingface/diffusers/pull/1860)] due to backwards compatibility concerns (since lots of external code was _already_ applying the scaling factor outside the VAE).
The “create new encode / decode methods that also scale / unscale, and use those wherever possible” approach (also [mentioned](https://github.com/huggingface/diffusers/pull/1860#issuecomment-1368873331) - but not implemented) sounds like a good idea to me. It’s plausible that the diffusers team would accept a PR doing just that. But it’s not my call (I’m just some random dev 🙂 )
---
**Author:**  [@keturn](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Fu%2Fketurn)%5C
**Post date:** [August 29, 2023, 7:01pm UTC](/static-proxy?url=https%3A%2F%2Fdiscuss.huggingface.co%2Ft%2Fautoencoderkl-scaling-factor-and-vaeimageprocessor%2F51367%2F7 "2023-08-29T19:01:20Z")
Thank you Ollin! I’ve been hoping to draw out the maintainers who _do_ make the API decisions, but I must have posted during summer vacation. Having the links to the previous discussions is handy though.