README.md

Generative Adversial Networks (GAN)

GANs are synthetic data generation tools. Synthetic data generation became a must-have skill for new data scientists (see Synthetic data generation — a must-have skill for new data scientists).

MinMax Loss

The value function for GAN is [Google]

$$\min_{G} \max_{D} V(G, D) = \mathbb{E}_x [ \log{\big( D(x) \big)} ] + \mathbb{E}_z[ \log{\big( 1-D(G(z)) \big)} ],$$

where $D(x)$ is the discriminator's estimate of the probability that real data instance $x$ is real, whereas $G(z)$ is the generator's output when given noise $z$, so $D(G(z))$ is the discriminator's estimate of the probability that a fake instance is real.

$\mathbb{E}_x$ is the expected value over all real data instances, whereas $\mathbb{E}_z$ is the expected value over all random inputs to the generator, i.e. the expected value over all generated fake instances $G(z)$.

The generator tries to minimize this function while the discriminator tries to maximize it. Looking at it as a min-max game, this formulation of the loss seemed effective.

In practice, it saturates for the generator, meaning that the generator quite frequently stops training if it doesn’t catch up with the discriminator.

Reference

• [Google] Google ML Developer - Loss Functions

• [neptune.ai] Understanding GAN Loss Functions

Convergence and Evaluate GAN

• [Konstantin Shmelkov et al.] How good is my GAN?

• [Jonathan Hui] GAN — Why it is so hard to train Generative Adversarial Networks!

GANs for Time-Series Data

• TimeGAN: [Jinsung Yoon et al.]

• Probabilistic autoregressive model (PAR): [Open source - SDV]

• DoppelGANger: [Kendrick Boyd], [Alex Watson]

Reference

• [Jinsung Yoon et al.] Time-series Generative Adversarial Networks

• [Parul Yadav et al.] Qualitative and Quantitative Evaluation of Multivariate Time-Series Synthetic Data Generated Using MTS-TGAN: A Novel Approach

• [Open source - SDV] PAR Model

• [Shuntaro Takahashi et al.] Modeling financial time-series with generative adversarial networks

• [Kendrick Boyd] Generate Synthetic Time-series Data with Open-source Tools

• [Alex Watson] Creating Synthetic Time Series Data for Global Financial Institutions – a POC Deep Dive

GANs for Tabluar data