Skip to content

Latest commit

 

History

History
115 lines (82 loc) · 9.14 KB

README.md

File metadata and controls

115 lines (82 loc) · 9.14 KB
Raw

StyleGANv3

Alias-Free Generative Adversarial Networks

Abstract

We observe that despite their hierarchical convolutional nature, the synthesis process of typical generative adversarial networks depends on absolute pixel coordinates in an unhealthy manner. This manifests itself as, e.g., detail appearing to be glued to image coordinates instead of the surfaces of depicted objects. We trace the root cause to careless signal processing that causes aliasing in the generator network. Interpreting all signals in the network as continuous, we derive generally applicable, small architectural changes that guarantee that unwanted information cannot leak into the hierarchical synthesis process. The resulting networks match the FID of StyleGAN2 but differ dramatically in their internal representations, and they are fully equivariant to translation and rotation even at subpixel scales. Our results pave the way for generative models better suited for video and animation.

Results and Models

Results (compressed) from StyleGAN3 config-T converted by MMGeneration

We perform experiments on StyleGANv3 paper settings and also experimental settings. For user convenience, we also offer the converted version of official weights.

Paper Settings

Model Dataset Iter FID50k Config Log Download
stylegan3-t ffhq 1024x1024 490000 3.37* config log model
stylegan3-t-ada metface 1024x1024 130000 15.09 config log model

Note*: This setting still needs a few days to run through, we put out currently the best checkpoint, and we will update the results the first time on the end of the experiment.

Experimental Settings

Model Dataset Iter FID50k Config Log Download
stylegan3-t ffhq 256x256 740000 7.65 config log model

Converted Weights

Model Dataset Comment FID50k EQ-T EQ-R Config Download
stylegan3-t ffhqu 256x256 official weight 4.62 63.01 13.12 config model
stylegan3-t afhqv2 512x512 official weight 4.04 60.15 13.51 config model
stylegan3-t ffhq 1024x1024 official weight 2.79 61.21 13.82 config model
stylegan3-r ffhqu 256x256 official weight 4.50 66.65 40.48 config model
stylegan3-r afhqv2 512x512 official weight 4.40 64.89 40.34 config model
stylegan3-r ffhq 1024x1024 official weight 3.07 64.76 46.62 config model

Interpolation

We provide a tool to generate video by walking through GAN's latent space. Run this command to get the following video.

python apps/interpolate_sample.py configs/styleganv3/stylegan3_t_afhqv2_512_b4x8_official.py https://download.openmmlab.com/mmgen/stylegan3/stylegan3_t_afhqv2_512_b4x8_cvt_official.pkl --export-video --samples-path work_dirs/demos/ --endpoint 6 --interval 60 --space z --seed 2022 --sample-cfg truncation=0.8
lerp.mp4

Equivarience Visualization && Evaluation

We also provide a tool to visualize the equivarience properties for StyleGAN3. Run these commands to get the results below.

python tools/utils/equivariance_viz.py configs/styleganv3/stylegan3_r_ffhqu_256_b4x8_official.py https://download.openmmlab.com/mmgen/stylegan3/stylegan3_r_ffhqu_256_b4x8_cvt_official.pkl --translate_max 0.5 --transform rotate --seed 5432

python tools/utils/equivariance_viz.py configs/styleganv3/stylegan3_r_ffhqu_256_b4x8_official.py https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmgen/stylegan3/stylegan3_r_ffhqu_256_b4x8_cvt_official.pkl --translate_max 0.25 --transform x_t --seed 5432

python tools/utils/equivariance_viz.py configs/styleganv3/stylegan3_r_ffhqu_256_b4x8_official.py https://openmmlab-share.oss-cn-hangzhou.aliyuncs.com/mmgen/stylegan3/stylegan3_r_ffhqu_256_b4x8_cvt_official.pkl --translate_max 0.25 --transform y_t --seed 5432
eq-rotate-seed-5432.mp4
eq-x_t-seed-5432.mp4
eq-y_t-seed-5432.mp4

If you want to get EQ-Metric for StyleGAN3, just add following codes into config.

metrics = dict(
    eqv=dict(
        type='Equivariance',
        num_images=50000,
        eq_cfg=dict(
            compute_eqt_int=True, compute_eqt_frac=True, compute_eqr=True)))

And we highly recommend you to use slurm_eval_multi_gpu script to accelerate evaluation time.

Citation

@inproceedings{Karras2021,
  author = {Tero Karras and Miika Aittala and Samuli Laine and Erik H\"ark\"onen and Janne Hellsten and Jaakko Lehtinen and Timo Aila},
  title = {Alias-Free Generative Adversarial Networks},
  booktitle = {Proc. NeurIPS},
  year = {2021}
}