Translated PolyMNIST Dataset

class multivae.data.datasets.TranslatedMMNIST(path, scale, translate, n_modalities, background_path=None, split='train', transform=ToTensor(), target_transform=None)[source]

Translated version of the PolyMNIST dataset. The data is built from background images that need to be downloaded beforehand.

The original PolyMNIST (5 modalities) background images can be downloaded from : https://mybox.inria.fr/d/78e581ee5b07402983fa/.

To use the ExtendedPolyMNIST dataset (10 modalities) introduced in “Score-Based Multimodal Autoencoder” (Wesego 2024), download the background images from https://github.com/rooshenasgroup/sbmae/tree/main/poly_background.

>>> from multivae.data.datasets import TranslatedMMNIST
>>> dataset = TranslatedMMNIST(
...   path = 'your_data_path',
...   scale = 0.75, # downscale 75%
...   translate = True, # random translation
...   n_modalities = 5,
...   background_path = 'path_to_background_image'
...)
Parameters:

  • path (str) – parent path where to save the dataset

  • scale (float) – The scale factor to downsample the MNIST images

  • translate (bool) – Wether to translate the MNIST images

  • n_modalities (int) – The number of modalities. It must match the number of background images.

  • background_path (str, optional) – Path to the background images. Defaults to None.

  • split (str, optional) – train or test. Defaults to ‘train’.

  • transform (torchvision.transforms, optional) – The transform to apply to images. Defaults to ToTensor().

  • target_transform (torchvision.transform, optional) – The transform to apply to labels. Defaults to None.

check_or_create_dataset(unimodal_paths, background_path, split)[source]

Check if the dataset exists at the provided path and if not creates the dataset from the background images