๐ŸŽฅ

Generative Adversarial Networks

Tags
Generative Adversarial Networks
Created
2021/01/31 10:50
Publication
NIPS'14
Rate
5
Source
https://arxiv.org/abs/1406.2661
Summary
๊ทธ ์œ ๋ช…ํ•œ GAN์ด๋‹ค. ๊ฐ€์งœ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•ด๋‚ด๋Š” ๋ชจ๋ธ generator์™€ ์ฃผ์–ด์ง„ ๋ฐ์ดํ„ฐ๊ฐ€ ๊ฐ€์งœ์ธ์ง€ ์ง„์งœ์ธ์ง€ ํŒ๋ณ„ํ•ด๋‚ด๋Š” ๋ชจ๋ธ discriminator๋ฅผ ๊ฒฝ์Ÿํ•˜๋“ฏ ํ•™์Šต์‹œํ‚ด์œผ๋กœ์จ ๋‘ ๋ชจ๋ธ์˜ ์„ฑ๋Šฅ์„ ๋™์‹œ์— ๋Œ์–ด์˜ฌ๋ฆฌ๊ณ , ์ด๋ ‡๊ฒŒ ํ•™์Šต๋œ generator๋ฅผ ํ†ตํ•ด ์‹ค์ œ ๋ฐ์ดํ„ฐ์™€ ์œ ์‚ฌํ•œ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•ด๋‚ด๊ณ ์ž ํ•œ๋‹ค. (์ƒ์„ธ ํŽ˜์ด์ง€ ์ฐธ๊ณ )

Reference

โ€ข
โ€ข
Tutorial Code: Colab Notebook
๊ทธ ์œ ๋ช…ํ•œ GAN์ด๋‹ค. GAN์€ 1) Generative: ํŠน์ •ํ•œ ๋ถ„ํฌ(i.e. ์‚ฌ๋žŒ์˜ ์–ผ๊ตด ์ด๋ฏธ์ง€์˜ ํ”ฝ์…€ ๊ฐ’ ๋ถ„ํฌ)์™€ ์ตœ๋Œ€ํ•œ ์œ ์‚ฌํ•œ ๋ถ„ํฌ์˜ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๋ชจ๋ธ์ธ๋ฐ, 2) Adversarial: ํŠน์ •ํ•œ ๋ถ„ํฌ๋ฅผ ๋ชจ์‚ฌํ•˜์—ฌ ๋ฐ์ดํ„ฐ๋ฅผ ์ƒ์„ฑํ•˜๋Š” ๋ชจ๋ธ(Generator)๊ณผ ์ฃผ์–ด์ง„ ๋ฐ์ดํ„ฐ๊ฐ€ ๋ชฉ์ ํ•˜๋Š” ๋ถ„ํฌ์—์„œ ๋ฐœ์ƒํ–ˆ๋Š”์ง€, ๊ทธ๋ ‡์ง€ ์•Š์€์ง€(G์—์„œ ์ƒ์„ฑ๋œ ๋ฐ์ดํ„ฐ์ธ์ง€)๋ฅผ ๊ตฌ๋ณ„ํ•˜๋Š” ๋ชจ๋ธ(Discriminator)์„ ๊ฒฝํ•ฉ์‹œํ‚ด์œผ๋กœ์จ Generator๋ฅผ ํ•™์Šต์‹œํ‚ค๋ฉฐ, 3) Net: ์ธ๊ณต ์‹ ๊ฒฝ๋ง์œผ๋กœ ๋ชจ๋ธ์„ ๊ตฌ์„ฑํ•œ๋‹ค. ํ•œ ๋งˆ๋””๋กœ ๋‘ ๋ฐ˜๋Œ€๋˜๋Š” ๋ชฉ์ ์„ ๊ฐ€์ง„ ๋ชจ๋ธ G์™€ D๋ฅผ ๊ฒฝ์Ÿ์‹œํ‚ด์œผ๋กœ์จ ํŠน์ • ๋ฐ์ดํ„ฐ์˜ ๋ถ„ํฌ๋ฅผ ๋ชจ์‚ฌํ•˜๋Š” ๋ชจ๋ธ G์˜ ์„ฑ๋Šฅ์„ ๋Œ์–ด์˜ฌ๋ฆฌ๋Š” ์•Œ๊ณ ๋ฆฌ์ฆ˜์ด๋‹ค.

Adversarial Nets

๋ชจ๋ธ ์ž์ฒด๋Š” ๊ฐ„๋‹จํ•˜๋‹ค. ์šฐ์„  ๋ชจ๋ธ GG๋Š” random noise variable zโˆผpzz \sim p_z๋กœ๋ถ€ํ„ฐ ๋ชฉ์ ํ•˜๋Š” ๋ฐ์ดํ„ฐ์˜ ๋ถ„ํฌ์ธ pdp_d๋กœ์˜ mapping์„ ํ•™์Šตํ•˜๋Š” ๋ชจ๋ธ์ด๋‹ค. ์ด ๋•Œ mapping์€ zโ†’G(z;ฮธg)โˆผpgz \rightarrow G(z; \theta_g) \sim p_g์™€ ๊ฐ™์ด ํ‘œ๊ธฐํ•  ์ˆ˜ ์žˆ๊ณ , ์ด๋Ÿฌํ•œ mapping์€ ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•œ ๋‹ค์ธต ์‹ ๊ฒฝ๋ง์œผ๋กœ ํ‘œํ˜„๋˜๋Š” ํ•จ์ˆ˜์ด๋‹ค. ํ•œํŽธ ๋ชจ๋ธ DD๋Š” ์–ด๋– ํ•œ ๋ฐ์ดํ„ฐ๊ฐ€ ์ฃผ์–ด์กŒ์„ ๋•Œ ๊ทธ ๋ฐ์ดํ„ฐ๊ฐ€ ์›๋ž˜ ๋ถ„ํฌ์ธ pdp_d๋กœ๋ถ€ํ„ฐ ์™”๋Š”์ง€, ๊ทธ๋ ‡์ง€ ์•Š์€์ง€(pgp_g๋กœ๋ถ€ํ„ฐ ์™”๋Š”์ง€)๋ฅผ ๊ตฌ๋ณ„ํ•ด๋‚ด๋Š” ๋ชจ๋ธ์ด๋‹ค. ์ฆ‰ D(x;ฮธd)D(x; \theta_d)๋Š” ๋ฐ์ดํ„ฐ xx๊ฐ€ ์›๋ž˜ ๋ถ„ํฌ pdp_d๋กœ๋ถ€ํ„ฐ ์™”์„ ํ™•๋ฅ ์„ ๋‚˜ํƒ€๋‚ธ๋‹ค. DD ์—ญ์‹œ ๋ฏธ๋ถ„ ๊ฐ€๋Šฅํ•œ ๋‹ค์ธต ์‹ ๊ฒฝ๋ง์œผ๋กœ ํ‘œํ˜„๋˜๋Š” ํ•จ์ˆ˜์ด๋‹ค.
์ด์ œ GG์˜ ํ•™์Šต ๋ชฉํ‘œ๋Š” DD๋ฅผ ์ตœ๋Œ€ํ•œ ์†์ด๋Š” ๊ฒƒ์ด๋ฉฐ, DD์˜ ํ•™์Šต ๋ชฉํ‘œ๋Š” GG๋ฅผ ์ตœ๋Œ€ํ•œ ์•Œ์•„์ฐจ๋ฆฌ๋Š” ๊ฒƒ์ด๋‹ค. ์ฆ‰ GG๋Š” D(G(z))D(G(z))๊ฐ€ ์ตœ๋Œ€ํ•œ 1์— ๊ฐ€๊นŒ์šด ๊ฐ’์„ ์ถœ๋ ฅํ•˜๋„๋ก ํ•™์Šต๋˜์–ด์•ผ ํ•œ๋‹ค. ํ•œํŽธ DD๋Š” D(x)xโˆผpdD(x)_{x \sim p_d}๊ฐ€ ์ตœ๋Œ€ํ•œ 1์— ๊ฐ€๊นŒ์šด ๊ฐ’์„ ์ถœ๋ ฅํ•˜๋Š” ๋™์‹œ์— D(G(z))zโˆผpzD(G(z))_{z \sim p_z}๊ฐ€ ์ตœ๋Œ€ํ•œ 0์— ๊ฐ€๊นŒ์šด ๊ฐ’์„ ์ถœ๋ ฅํ•˜๋„๋ก ํ•™์Šต๋˜์–ด์•ผ ํ•œ๋‹ค. ์ด๋Š” ๋‹ค์Œ๊ณผ ๊ฐ™์€ ํ•จ์ˆ˜ V(G,D)V(G, D)์— ๋Œ€ํ•œ two-player minimax game์œผ๋กœ ํ‘œํ˜„๋  ์ˆ˜ ์žˆ๋‹ค.
minGmaxDV(D,G)=Exโˆผpd[logโกD(x)]+Ezโˆผpz[logโก(1โˆ’D(G(z)))]min_{G} max_{D} V(D, G) = E_{x \sim p_d}[ \log{ D(x) } ] + E_{z \sim p_z}[ \log{( 1 - D(G(z)) )} ]
์ด์ œ ์œ„์™€ ๊ฐ™์€ ๋ชฉ์  ํ•จ์ˆ˜๋ฅผ ๊ฐ€์ง€๊ณ  GG์™€ DD๋ฅผ ๋ฒˆ๊ฐˆ์•„๊ฐ€๋ฉฐ ํ•™์Šตํ•˜๋ฉด ๋œ๋‹ค. ์ด ๋•Œ ๋…ผ๋ฌธ์—์„œ๋Š” ํ•œ ๋ฒˆ์˜ iteration๋งˆ๋‹ค DD๋ฅผ kk๋ฒˆ, ๊ทธ๋ฆฌ๊ณ  GG๋ฅผ 1๋ฒˆ ํ•™์Šตํ•จ์œผ๋กœ์จ DD๊ฐ€ optimal solution๊ณผ ๊ฐ€๊นŒ์ด ์œ ์ง€๋˜๋ฉด์„œ ์ฒœ์ฒœํžˆ GG๋ฅผ ๊ฐœ์„ ํ•˜๋Š” ๋ฐฉ์‹์„ ๊ถŒํ•œ๋‹ค. (๊ทธ ์ด์œ ๋Š” Optimizing D to completion in the inner loop of training is computationally prohibitive, and on finite datasets would result in overfitting์ด๋ผ๊ณ  ์“ฐ์—ฌ ์ž‡๋Š”๋ฐ ๋ฌด์Šจ ์†Œ๋ฆฌ์ธ์ง€ ์ž˜ ๋ชจ๋ฅด๊ฒ ๋‹ค. ์ฐจ๋ผ๋ฆฌ ๋ฐ‘์˜ theoretical results๋ฅผ ๋ณด๊ณ  ๋‚˜๋ฉด ์ดํ•ด๊ฐ€ ๋œ๋‹ค.) ๋˜ GG์˜ ๊ฒฝ์šฐ discriminator๊ฐ€ ๋›ฐ์–ด๋‚œ ์„ฑ๋Šฅ์„ ๋ณด์ผ ๊ฒฝ์šฐ logโก(1โˆ’D(G(z)))\log{( 1 - D(G(z)) )}๊ฐ€ 0์— ๋งค์šฐ ๊ฐ€๊นŒ์šธ ๊ฒƒ์ด๊ธฐ ๋•Œ๋ฌธ์—(๊ทธ๋ฆฌํ•˜์—ฌ ํ•™์Šต์ด ์ž˜ ์ด๋ฃจ์–ด์ง€์ง€ ์•Š์„ ๊ฒƒ์ด๊ธฐ ๋•Œ๋ฌธ์—) logโก(1โˆ’D(G(z)))\log{(1 - D(G(z)))}๋ฅผ ์ตœ์†Œํ™”ํ•˜๋Š” ๊ฒƒ๋ณด๋‹ค๋Š” logโกD(G(z))\log{D(G(z))}๋ฅผ ์ตœ๋Œ€ํ™”ํ•˜๋Š” ์‹์œผ๋กœ ํ•™์Šตํ•˜๋Š” ๊ฒƒ์„ ๊ถŒํ•œ๋‹ค.
์œ„์˜ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ pseudo code๋กœ ํ‘œํ˜„ํ•˜๋ฉด ๋‹ค์Œ๊ณผ ๊ฐ™๊ฒ ๋‹ค. Mini(Generator)Max(Discriminator) game์ด๊ธฐ ๋•Œ๋ฌธ์— DD๋ฅผ ์—…๋ฐ์ดํŠธํ•  ๋•Œ๋Š” gradient ascending, GG๋ฅผ ์—…๋ฐ์ดํŠธํ•  ๋•Œ๋Š” gradient descending์ด ํ•„์š”ํ•˜๋‹ค๋Š” ์ ์— ์ฃผ์˜ํ•˜์ž.
for _ in range(num_iters): for _ in range(k): z = sample m noise samples from p_z x = sample m real data samples from p_d update discriminator by _ascending_ gradient dV(D, G)_d z = sample m noise samples from p_z update generator by _descending_ gradient dV(D, G)_g
Python

Theoretical Results

ํ•œํŽธ ์ด ๋…ผ๋ฌธ์—์„œ๋Š” ๊ต‰์žฅํžˆ ํ›Œ๋ฅญํ•˜๊ฒŒ๋„ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ์†Œ๊ฐœํ•˜๋Š” ๋ฐ์— ๊ทธ์น˜์ง€ ์•Š๊ณ , ์ด๋Ÿฌํ•œ ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ์ด๋ก ์  ํ† ๋Œ€๋ฅผ ์ œ๊ณตํ•œ๋‹ค. ์š”๋Š” "ํ‘œํ˜„๋ ฅ์˜ ์ œํ•œ์ด ์—†๋‹ค๋Š” ๊ฐ€์ • ํ•˜์—, ์†Œ๊ฐœ๋œ minimax game์€ pg=pdp_g = p_d์—์„œ global optimum์„ ๊ฐ€์ง€๊ณ , ์†Œ๊ฐœ๋œ ์•Œ๊ณ ๋ฆฌ์ฆ˜์„ ํ†ตํ•ด ์ด๋Ÿฌํ•œ global optimum์— ๋„๋‹ฌํ•  ์ˆ˜ ์žˆ๋‹ค"๋Š” ๊ฒƒ์ด๋‹ค.
์œ„์˜ statement๋ฅผ ์ฆ๋ช…ํ•˜๊ธฐ ์œ„ํ•ด ๋…ผ๋ฌธ์—์„œ๋Š” ๋จผ์ € 1) ์ฃผ์–ด์ง„ ์ž„์˜์˜ GG์— ๋Œ€ํ•œ optimal discriminator DGโˆ—(x)=pd(x)pd(x)+pg(x)D_{G}^{*}(x) = \frac{p_d(x)}{p_d(x) + p_g(x)}๋ผ๋Š” ๊ฒƒ์„ ๋ณด์ด๋ฉฐ, 2) ์ด๋Ÿฌํ•œ optimal discriminator DGโˆ—(x)D_{G}^{*}(x)์— ๋Œ€ํ•ด GG์˜ training criterion์˜ global minimum์ด pg=pdp_g = p_d์ผ ๋•Œ๋งŒ ๋‹ฌ์„ฑ๋œ๋‹ค๋Š” ๊ฒƒ์„ ๋ณด์ธ๋‹ค. ๊ทธ๋ฆฌ๊ณ  ๋งˆ์ง€๋ง‰์œผ๋กœ 3) ํ‘œํ˜„๋ ฅ์˜ ์ œํ•œ์ด ์—†๋Š” ์ƒํ™ฉ์—์„œ ์†Œ๊ฐœ๋œ ์•Œ๊ณ ๋ฆฌ์ฆ˜์˜ ๋งค iteration์—์„œ DD๊ฐ€ optimum์— ๋„๋‹ฌํ•  ์ˆ˜ ์žˆ๊ณ , pgp_g๊ฐ€ ์—…๋ฐ์ดํŠธ๋˜์–ด pdp_d๋กœ ์ˆ˜๋ ดํ•  ์ˆ˜ ์žˆ์Œ์„ ๋ณด์ธ๋‹ค.
์ฒซ ๋‘ ์ฆ๋ช…์€ ๋”ฐ๋ผ๊ฐˆ ๋งŒ ํ•œ๋ฐ, ๋งˆ์ง€๋ง‰ ๋ถ€๋ถ„์˜ ์ฆ๋ช…์ด ๋งค์šฐ ๊นŒ๋‹ค๋กญ๋‹ค. ์™œ pgp_g์— ๋Œ€ํ•œ criterion function V(G,D)=U(pg,D)V(G, D) = U(p_g, D)๊ฐ€ convex function์ธ์ง€ ๋“ฑ... ์–ด์จŒ๊ฑฐ๋‚˜ ํ•˜๊ณ ์ž ํ•˜๋Š” ๋ง์€ ์ž˜ ์•Œ๊ฒ ๋‹ค! ๋ฌผ๋ก  neural networks์— ํ‘œํ˜„๋ ฅ์˜ ์ œํ•œ์ด ์—†๋Š” ๊ฒƒ์€ ์•„๋‹ˆ์ง€๋งŒ, neural networks๊ฐ€ ๋ชจ๋“  ํ•จ์ˆ˜์— ๋Œ€ํ•œ approximation์„ ํ•  ์ˆ˜ ์žˆ๋‹ค๋Š” ์ธก๋ฉด์—์„œ ์œ„์˜ ์ด๋ก ์  ํ† ๋Œ€๋ฅผ ๋’ท๋ฐ›์นจํ•  ์ˆ˜ ์žˆ๋Š” (ํ˜„์žฌ๊นŒ์ง€๋Š”) ๊ฐ€์žฅ ๋“ ๋“ ํ•œ ๋ชจ๋ธ์ผ ๊ฒƒ์ด๋‹ค.
์˜ˆ์‹œ ์ฝ”๋“œ๋Š” ๋งจ ์œ„์—์„œ ์–ธ๊ธ‰๋œ ๋‚˜์˜ colab notebook์—์„œ ๋ณผ ์ˆ˜ ์žˆ๋‹ค. MNIST ๋ฐ์ดํ„ฐ๋ฅผ ๋ชจ์‚ฌํ•˜๋Š” ๋งค์šฐ ๊ฐ„๋‹จํ•œ ๋ฌธ์ œ์ด์ง€๋งŒ, ์ด๋Ÿฌํ•œ ์ง๊ด€์ ์ธ ์•Œ๊ณ ๋ฆฌ์ฆ˜์ด ๊ดœ์ฐฎ๊ฒŒ ์ž‘๋™ํ•œ๋‹ค๋Š” ์ธก๋ฉด์—์„œ ๊ต‰์žฅํžˆ ์‹ ๊ธฐํ•˜๋‹ค.
E.O.D.