Likelihoods

This module computes likelihood for any multivae model.

from multivae.metrics import LikelihoodsEvaluator, LikelihoodsEvaluatorConfig

eval_config = LikelihoodsEvaluatorConfig(batch_size=128,
                            wandb_path='your_wandb_path',
                            num_samples=1000)

eval_module = LikelihoodsEvaluator(
    model = your_model,
    test_dataset=test_set,
    output='./metrics',# where to save metrics
    eval_config=eval_config
)

# Compute joint negative log likelihood
eval_module.eval()

eval_module.finish() # finishes wandb run
class multivae.metrics.LikelihoodsEvaluatorConfig(batch_size=512, wandb_path=None, num_samples=1000, batch_size_k=100, unified_implementation=True)[source]

Config class for the evaluation of the coherences module.

Parameters:

  • batch_size (int) – The batch size to use in the evaluation. Default to 512

  • wandb_path (str) – The user can provide the path of the wandb run with a format β€˜entity/projet_name/run_id’ where the metrics should be logged. See Where to find the WandB path for a trained model ? for more information. If None is provided, the metrics are not logged on wandb. Default to None.

  • num_samples (int) – How many samples to use for likelihoods estimates. Default to 1000.

  • batch_size_k (int) – How to batch the K samples for likelihoods estimates. Default to 100.

  • unified_implementation (bool) – When the paper implementation of the likelihood differ from the unified implementation, specify which to use. Default to True.

class multivae.metrics.LikelihoodsEvaluator(model, test_dataset, output=None, eval_config=LikelihoodsEvaluatorConfig(name='LikelihoodsEvaluatorConfig', batch_size=512, wandb_path=None, num_samples=1000, batch_size_k=100, unified_implementation=True))[source]

Class for computing likelihood metrics.

Parameters:

  • model (BaseMultiVAE) – The model to evaluate.

  • test_dataset (MultimodalBaseDataset) – The dataset to use for computing the metrics.

  • output (str) – The folder path to save metrics. The metrics will be saved in a metrics.txt file.

  • eval_config (EvaluatorConfig) – The configuration class to specify parameters for the evaluation.

joint_nll_from_subset(subset)[source]

Only available for the MoPoE model for now. Use a subset posterior instead of the joint posterior as the importance sampling distribution.