> ## Content Index
> Fetch the complete content index at: https://sudormrf.run/llms.txt
> Use this file to discover other available public pages before exploring further.

# BicycleGAN : Image Translation with GAN (5)
- URL: https://sudormrf.run/2018/04/03/image-translation-with-gan-5/
- Published: 2018-04-03T04:24:31.000Z
- Updated: 2018-04-05T12:22:45.000Z
- Author: Junho Cho
- Tags: Computer Vision, Deep Learning, #Import 2026-10-24 03:36

## Limitations of pix2pix, DTN, DiscoGAN & CycleGAN?

- They produce single answer.
- They are deterministic models.  
  - Translates an image in one-to-one

![right fit](https://ws3.sinaimg.cn/large/006tNc79gy1fpa9y2frdcj31400kajtc.jpg)

- Paired set, One-to-One : pix2pix (CVPR2017)
- Unpaired set, One-to-One : DTN (ICLR2017), CycleGAN (ICCV2017)
- Paired set, One-to-Many : ???

# BicycleGAN:

Toward Multimodal Image-to-Image Translation (NIPS2017)  
[BicycleGAN github](https://github.com/junyanz/BicycleGAN?ref=sudormrf.run)

![inline](https://ws2.sinaimg.cn/large/006tNc79gy1fpa9zhv6ivj30os07yabc.jpg)  
![fit](https://junyanz.github.io/BicycleGAN/index_files/day2night.gif)  
![inline](https://ws2.sinaimg.cn/large/006tNc79gy1fpa9zpkf7aj31200p6ait.jpg)

## Easy approach:

- Adopt stochastically sampled noise $N(z)$ to the deterministic generator
- Hope noise act as latent code to produce diverse results

![inline](https://ws1.sinaimg.cn/large/006tNc79gy1fpaa7m2wjkj30n20eyt9o.jpg)  
![inline](https://ws4.sinaimg.cn/large/006tNc79gy1fpaae06sovj314006cmyk.jpg)

### However, it causes mode collapse

![inline](https://ws4.sinaimg.cn/large/006tNc79gy1fpaaeq1d28j30y20eo78u.jpg)

- Generates from multiple noises but mapped to similar outputs
- Generator do not care noise
- Generate learns to ignore random noise when conditioned on relevant context (input image)

## Encoder

- Encourage bijection between the output <-> latent space
- Disturb two different latent codes to generate same output.  
  - Avoid mode collapse

![inline](https://ws4.sinaimg.cn/large/006tNc79gy1fpaaiosreqj30zs0f640q.jpg)

## Conditional Variational Autoencoder-GAN

![inline](https://ws1.sinaimg.cn/large/006tNc79gy1fpaam8bn1sj30rc0f6dha.jpg)  
![inline](https://ws2.sinaimg.cn/large/006tNc79gy1fpaalwfmobj3186086wjx.jpg)

- Encoder predicts Gaussian
- Encoding is trained with real data (B)
- Generator takes latent code with rich info of $B$ and input $A$.
- At test time, generated from random latent code may produce unrealistic image.
- Generator never see random noise.
- Discriminator never see samples from generated from random noise

## Conditional Latent Regressor GAN

![inline](https://ws1.sinaimg.cn/large/006tNc79gy1fpaat3h3x0j30o20e40tv.jpg)  
![inline](https://ws2.sinaimg.cn/large/006tNc79gy1fpaatil0yaj310a04s3zb.jpg)

- Encoder is latent code regressor.
- Generated sample is encoded and mapped back to random noise.
- Latent code is easily and randomly sampled, as test time.
- Generator never sees ground truth $B$.
- More vulnerable to mode collapse, probably small dimension of $z$ and $L\_1$ loss on $z$ and $\\hat{z}$ is not enough to prevent generator easily fool discriminator?

# BicycleGAN

![inline](https://ws1.sinaimg.cn/large/006tNc79gy1fpaaukzcrmj314k0gin09.jpg)

### Train both model together, with benefit of cycle-loss in $z$ and $B$.

## Result

- Pix2pix + noise : similar realistic outputs
- cVAE-GAN : adds variation but artifacts caused from random sample at test
- cLR-GAN : less variant in output and sometimes mode collapse
- BicycleGAN : hybrid results both diverse and realistic

![inline](https://ws4.sinaimg.cn/large/006tNc79gy1fpaavasw1pj313q0g00xq.jpg)

## Quantative experiment

![inline](https://ws3.sinaimg.cn/large/006tNc79gy1fpaavpgf1fj30ye0dg42l.jpg)

## Conclusion

- Propose solution to mode collapse in the conditional generative setting
- Combine multiple objectives for encouraging a bijective mapping between the latent and output spaces
- Produce both realistic and diverse
- Latent code could be replaced with user controllable parameter in the future

## Trends

- Paired set, One-to-One : [pix2pix](https://sudormrf.run/2018/04/02/image-translation-with-gan-2) (CVPR2017)
- Unpaired set, One-to-One : [DTN](https://sudormrf.run/2018/04/02/image-translation-with-gan-3) (ICLR2017), [DiscoGAN](https://sudormrf.run/2018/04/02/image-translation-with-gan-4) (ICML2017), [CycleGAN](https://sudormrf.run/2018/04/02/image-translation-with-gan-4) (ICCV2017)
- Paired set, One-to-Many : BicycleGAN (NIPS2017)
- In the future:  
  - Unpaired set, One-to-Many : [Augmented CycleGAN](https://arxiv.org/abs/1802.10151?ref=sudormrf.run) (probabily ICML2018 submitted), [XGAN](https://arxiv.org/abs/1711.05139?ref=sudormrf.run) (ICLR2018 [rejected](https://openreview.net/forum?id=rkWN3g-AZ&ref=sudormrf.run))
  - Multi domains. Not only a source domain to a target domain: [StarGAN](https://sudormrf.run/2018/04/05/image-translation-with-gan-6) (CVPR2018 accepted)
  - User controllable noise vector in BicycleGAN