Inverse Autoregressive Flows

Sampler fitting an Inverse Autoregressive Flow in the multimodal variational autoencoder’s latent space. This class is simply adapted from Pythae’s iaf_sampler for the multimodal case.

class multivae.samplers.IAFSamplerConfig(n_made_blocks=2, n_hidden_in_made=3, hidden_size=128, include_batch_norm=False)

This is the IAF sampler model configuration instance.

Parameters:

• n_made_blocks (int) – The number of MADE model to consider in the IAF. Default: 2.

• n_hidden_in_made (int) – The number of hidden layers in the MADE models. Default: 3.

• hidden_size (list) – The number of unit in each hidder layer. The same number of units is used across the n_hidden_in_made and n_made_blocks. Default: 3.

• include_batch_norm (bool) – Whether to include batch normalization after each MADE layers. Default: False.

class multivae.samplers.IAFSampler(model, sampler_config=None)

Fits an Inverse Autoregressive Flow in the multimodal autoencoder’s latent space. If the model has multiple latent spaces, we fit one flow per latent space.

Parameters:

• model (BaseMultiVAE) – The model to sample from

• sampler_config (IAFSamplerConfig) – A IAFSamplerConfig instance containing the main parameters of the sampler. If None, a pre-defined configuration is used. Default: None

Note

The method fit must be called to fit the sampler before sampling.

fit(train_data, eval_data=None, training_config=None)

Method to fit the sampler from the training data.

Parameters:

• train_data (MultimodalBaseDataset) – The train data needed to retreive the training embeddings and fit the mixture in the latent space. Must be of shape n_imgs x im_channels x … and in range [0-1]

• eval_data (MultimodalBaseDataset) – The train data needed to retreive the evaluation embeddings and fit the mixture in the latent space. Must be of shape n_imgs x im_channels x … and in range [0-1]

• training_config (BaseTrainerConfig) – the training config to use to fit the flow.

load_flows_from_folder(dir_path)

Instead of calling fit, you can reload weights from a previous training.

>>> sampler.save(dir_path)
>>> new_sampler = IAFSampler(model, sampler_config) # must be the same model and config
>>> new_sampler.load_flows_from_folder(dir_path)

sample(n_samples=1, batch_size=500, **kwargs)

Main sampling function of the sampler.

Parameters:

• num_samples (int) – The number of samples to generate

• batch_size (int) – The batch size to use during sampling

• output_dir (str) – The directory where the images will be saved. If does not exist the folder is created. If None: the images are not saved. Defaults: None.

• return_gen (bool) – Whether the sampler should directly return a tensor of generated data. Default: True.

• save_sampler_config (bool) – Whether to save the sampler config. It is saved in output_dir

Returns:

The generated images

Return type:

Tensor

save(dir_path)

Save the config and trained models.