deeprob.torch package

Submodules

deeprob.torch.base module

class deeprob.torch.base.ProbabilisticModel(*args, **kwargs)[source]

Bases: ABC, Module

Initializes internal Module state, shared by both nn.Module and ScriptModule.

has_rsample = False
log_prob(x)[source]

Compute the log-likelihood of a batched sample. Note that the nn.Module.forward method of sub-classes must implement log-likelihood evaluation.

Parameters

x (Tensor) – The batched sample.

Returns

The batched log-likelihoods.

Return type

Tensor

abstract sample(n_samples, y=None)[source]

Sample some values from the modeled distribution.

Parameters

  • n_samples (int) – The number of samples.

  • y (Optional[Tensor]) – The samples labels. It can be None.

Returns

The samples.

Return type

Tensor

abstract loss(x, y=None)[source]

Compute the loss of the model.

Parameters

  • x (Tensor) – The outputs of the model.

  • y (Optional[Tensor]) – The ground-truth. It can be None.

Returns

The loss.

Return type

Tensor

apply_constraints()[source]

Apply the constraints specified by the model.

training: bool
deeprob.torch.base.DensityEstimator

A density estimator is either a DeeProb-kit probabilistic model or a Torch distribution.

alias of Union[ProbabilisticModel, Distribution]

deeprob.torch.callbacks module

class deeprob.torch.callbacks.EarlyStopping(model, patience=1, filepath='checkpoint.pt', delta=0.001)[source]

Bases: object

Early stops the training if validation loss doesn’t improve after a given number of consecutive epochs.

Parameters

  • model (Module) – The model to monitor.

  • patience (int) – The number of consecutive epochs to wait.

  • filepath (Union[PathLike, str]) – The checkpoint filepath where to save the model state dictionary.

  • delta (float) – The minimum change of the monitored quantity.

Raises

ValueError – If the patience or delta values are out of domain.

property should_stop: bool

Check if the training process should stop.

get_best_state()[source]

Get the best model’s state dictionary.

Return type

OrderedDict

__call__(loss, epoch)[source]

Update the state of early stopping.

Parameters

  • loss (float) – The validation loss measured.

  • epoch (int) – The current epoch.

deeprob.torch.constraints module

class deeprob.torch.constraints.ScaleClipper(eps=1e-05)[source]

Bases: Module

Constraints the scale to be positive.

Parameters

eps (float) – The epsilon minimum value threshold.

Raises

ValueError – If the epsilon value is out of domain.

forward(module)[source]

Call the constraint.

Parameters

module (Module) – The module.

training: bool

deeprob.torch.datasets module

class deeprob.torch.datasets.UnsupervisedDataset(dataset, transform=None)[source]

Bases: Dataset

Initialize an unsupervised dataset.

Parameters

  • dataset – The dataset.

  • transform – An optional transformation to apply.

property features_shape: Union[int, tuple]

Get the dataset features shape.

class deeprob.torch.datasets.SupervisedDataset(dataset, targets, transform=None)[source]

Bases: Dataset

Initialize a supervised dataset.

Parameters

  • dataset – The dataset.

  • targets – The targets.

  • transform – An optional transformation to apply.

property features_shape: Union[int, tuple]

Get the dataset features shape.

property num_classes: int

Get the number of classes.

class deeprob.torch.datasets.WrappedDataset(dataset, unsupervised=True, classes=None, transform=None)[source]

Bases: Dataset

Initialize a wrapped dataset (either unsupervised or supervised).

Parameters

  • dataset – The dataset (assumed to be supervised).

  • unsupervised – Whether to treat the dataset as unsupervised.

  • classes – The class domain. It can be None if unsupervised is True.

  • transform – An optional transformation to apply.

property features_shape: Union[int, tuple]

Get the dataset features shape.

property num_classes: int

Get the number of classes.

deeprob.torch.initializers module

deeprob.torch.initializers.dirichlet_(tensor, alpha=1.0, log_space=True, dim=- 1)[source]

Initialize a tensor using the symmetric Dirichlet distribution.

Parameters

  • tensor (Tensor) – The tensor to initialize.

  • alpha (float) – The concentration parameter.

  • log_space (bool) – Whether to initialize the tensor in the logarithmic space.

  • dim (int) – The dimension over which to sample.

deeprob.torch.metrics module

class deeprob.torch.metrics.RunningAverageMetric[source]

Bases: object

Initialize a running average metric object.

__call__(metric, num_samples)[source]

Accumulate a metric value.

Parameters

  • metric (float) – The metric value.

  • num_samples (int) – The number of samples from which the metric is estimated.

Raises

ValueError – If the number of samples is not positive.

reset()[source]

Reset the running average metric accumulator.

average()[source]

Get the metric average.

Returns

The metric average.

Return type

float

deeprob.torch.metrics.fid_score(dataset1, dataset2, model=None, transform=None, batch_size=100, num_workers=0, device=None, verbose=True)[source]

Compute the Frechet Inception Distance (FID) between two data samples. This implementation has been readapted from https://github.com/mseitzer/pytorch-fid. IMPORTANT NOTE: The computed FID score is not comparable with other FID scores based on Tensorflow’s InceptionV3.

Parameters

  • dataset1 (Union[Dataset, Tensor]) – The first samples data set.

  • dataset2 (Union[Dataset, Tensor]) – The second samples data set.

  • model (Optional[Module]) – The model to use to extract the features. If None the Torchvision’s InceptionV3 model pretrained on ImageNet will be used.

  • transform (Optional[Any]) – An optional transformation to apply to every sample. If transform and model are both None, then the transformation resizes to 3x299x299 and normalizes values from (0, 1) to (-1, 1).

  • batch_size (int) – The batch size to use when extracting features.

  • num_workers (int) – The number of workers used for the data loaders.

  • device (Optional[device]) – The device used to run the model. If it’s None ‘cuda’ will be used, if available.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The FID score.

Return type

float

deeprob.torch.metrics.extract_features(model, dataset, transform=None, device=None, verbose=True, **kwargs)[source]

Extract the features produced by a model using a data set.

Parameters

  • model (Module) – The model to use to extract the features.

  • dataset (Union[Dataset, Tensor]) – The data set.

  • transform (Optional[Any]) – An optional transformation to apply to every sample.

  • device (Optional[device]) – The device used to run the model. If it’s None ‘cuda’ will be used, if available.

  • verbose (bool) – Whether to enable verbose mode.

  • kwargs – Additional parameters to pass to the data loader.

Returns

The extracted features for each data sample.

Return type

Tensor

deeprob.torch.routines module

deeprob.torch.routines.train_model(model, data_train, data_valid, setting, lr=0.001, batch_size=100, epochs=1000, optimizer='adam', optimizer_kwargs=None, patience=20, checkpoint='checkpoint.pt', train_base=True, drop_last=True, num_workers=0, device=None, verbose=True)[source]

Train a Torch model.

Parameters

  • model (ProbabilisticModel) – The model to train.

  • data_train (Union[ndarray, Dataset]) – The train dataset.

  • data_valid (Union[ndarray, Dataset]) – The validation dataset.

  • setting (str) – The train setting. It can be either ‘generative’ or ‘discriminative’.

  • lr (float) – The learning rate to use.

  • batch_size (int) – The batch size for both train and validation.

  • epochs (int) – The number of epochs.

  • optimizer (str) – The optimizer to use.

  • optimizer_kwargs (Optional[dict]) – A dictionary containing additional optimizer parameters.

  • patience (int) – The epochs patience for early stopping.

  • checkpoint (Union[PathLike, str]) – The checkpoint filepath used for early stopping.

  • train_base (bool) – Whether to train the input base module. Only applicable for normalizing flows.

  • drop_last (bool) – Whether to drop the last train data batch having size less than the specified batch size.

  • num_workers (int) – The number of workers for data loading.

  • device (Optional[device]) – The device used for training. If it’s None ‘cuda’ will be used, if available.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The train history.

Raises

ValueError – If a parameter is out of domain.

Return type

Union[Dict[str, list], Dict[str, Dict[str, list]]]

deeprob.torch.routines.train_generative(model, train_loader, valid_loader, optimizer, device, early_stopping, epochs=1000, train_base=True, verbose=True)[source]

Train a Torch model in generative setting.

Parameters

  • model (ProbabilisticModel) – The model.

  • train_loader (DataLoader) – The train data loader.

  • valid_loader (DataLoader) – The validation data loader.

  • optimizer (Optimizer) – The optimize to use.

  • device (device) – The device to use for training.

  • epochs (int) – The number of epochs.

  • early_stopping (EarlyStopping) – The early stopping callback object.

  • train_base (bool) – Whether to train the input base module. Only applicable for normalizing flows.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The train history with keys ‘train’ and ‘validation’.

Raises

ValueError – If a parameter is out of domain.

Return type

Dict[str, list]

deeprob.torch.routines.train_discriminative(model, train_loader, valid_loader, optimizer, device, early_stopping, epochs=1000, train_base=True, verbose=True)[source]

Train a Torch model in discriminative setting.

Parameters

  • model (ProbabilisticModel) – The model.

  • train_loader (DataLoader) – The train data loader.

  • valid_loader (DataLoader) – The validation data loader.

  • optimizer (Optimizer) – The optimize to use.

  • device (device) – The device to use for training.

  • early_stopping (EarlyStopping) – The early stopping callback object.

  • epochs (int) – The number of epochs.

  • train_base (bool) – Whether to train the input base module. Only applicable for normalizing flows.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The train history with keys ‘train’ and ‘validation’ and for both keys ‘loss’ and ‘accuracy’.

Raises

ValueError – If a parameter is out of domain.

Return type

Dict[str, Dict[str, list]]

deeprob.torch.routines.test_model(model, data_test, setting, batch_size=100, num_workers=0, device=None, verbose=True)[source]

Test a Torch model.

Parameters

  • model (ProbabilisticModel) – The model to test.

  • data_test (Union[ndarray, Dataset]) – The test dataset.

  • setting (str) – The test setting. It can be either ‘generative’ or ‘discriminative’.

  • batch_size (int) – The batch size for testing.

  • num_workers (int) – The number of workers for data loading.

  • device (Optional[device]) – The device used for training. If it’s None ‘cuda’ will be used, if available.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The mean log-likelihood and two standard deviations if setting=’generative’. The negative log-likelihood and classification metrics if setting=’discriminative’.

Raises

ValueError – If a parameter is out of domain.

Return type

Union[Tuple[float, float], Tuple[float, dict]]

deeprob.torch.routines.test_generative(model, test_loader, device, verbose=True)[source]

Test a Torch model in generative setting.

Parameters

  • model (ProbabilisticModel) – The model to test.

  • test_loader (DataLoader) – The test data loader.

  • device (device) – The device used for testing.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The mean log-likelihood and two standard deviations.

Return type

Tuple[float, float]

deeprob.torch.routines.test_discriminative(model, test_loader, device, verbose=True)[source]

Test a Torch model in discriminative setting.

Parameters

  • model (ProbabilisticModel) – The model to test.

  • test_loader (DataLoader) – The test data loader.

  • device (device) – The device used for testing.

  • verbose (bool) – Whether to enable verbose mode.

Returns

The negative log-likelihood and classification report dictionary.

Return type

Tuple[float, dict]

deeprob.torch.transforms module

class deeprob.torch.transforms.Transform[source]

Bases: ABC

Generic data transform function.

__call__(x)[source]

Evaluate in forward mode the transformation. Equivalent to forward(x).

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

abstract forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

abstract backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.TransformList(iterable=(), /)[source]

Bases: Transform, list

A list of transformations.

forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.Normalize(mean, std, eps=1e-07)[source]

Bases: Transform

Initialize a normalization transformation. This transformation computes the following equations:

y = (x - mean) / (std + eps)
x = y * (std + eps) + mean
Parameters

  • mean (Union[float, Tensor]) – The mean values. One for each channel.

  • std (Union[float, Tensor]) – The standard deviation values.

  • eps (float) – The epsilon value (used to avoid divisions by zero).

Raises

ValueError – If the epsilon value is out of domain.

forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.Quantize(n_bits=8)[source]

Bases: Transform

Initialize a quantization transformation. This transformation computes the following equations:

y = clamp(floor(x * 2 ** n_bits), 0, 2 ** n_bits - 1) / (2 ** n_bits - 1)
x = ((x * (2 ** n_bits - 1)) + u) / (2 ** n_bits)
with u ~ Uniform(0, 1)
Parameters

n_bits (int) – The number of bits.

Raises

ValueError – If the number of bits is not positive.

forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.Flatten(shape=None)[source]

Bases: Transform

Initialize a flatten transformation.

Parameters

shape (Optional[Union[Size, List[int], Tuple[int, ...]]]) – The original tensor shape. It can be None to enable only forward transformation.

forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.Reshape(target_shape, shape=None)[source]

Bases: Transform

Initialize a reshape transformation.

Parameters
forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

class deeprob.torch.transforms.RandomHorizontalFlip(p=0.5)[source]

Bases: Transform

Initialize a random horizontal flip transformation.

Parameters

p (float) – The probability of flipping.

Raises

ValueError – If the probability of flipping is out of domain.

forward(x)[source]

Evaluate in forward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

backward(x)[source]

Evaluate in backward mode the transformation.

Parameters

x (Tensor) – The inputs.

Returns

The outputs.

Return type

Tensor

deeprob.torch.utils module

deeprob.torch.utils.get_activation_class(name)[source]

Get the activation function class by its name.

Parameters

name (str) – The activation function’s name. It can be one of: ‘relu’, ‘leaky-relu’, ‘softplus’, ‘tanh’, ‘sigmoid’.

Returns

The activation function class.

Raises

ValueError – If the activation function’s name is not known.

deeprob.torch.utils.get_optimizer_class(name)[source]

Get the optimizer class by its name.

Parameters

name (str) – The optimizer’s name. It can be ‘sgd’, ‘rmsprop’, ‘adagrad’, ‘adam’.

Returns

The optimizer class.

Raises

ValueError – If the optimizer’s name is not known.

class deeprob.torch.utils.ScaledTanh(weight_size=1)[source]

Bases: Module

Build the module.

Parameters

weight_size (Union[int, tuple, list]) – The size of the weight parameter.

forward(x)[source]

Apply the scaled tanh function.

Parameters

x (Tensor) – The inputs.

Returns

The outputs of the module.

Return type

Tensor

training: bool
class deeprob.torch.utils.MaskedLinear(in_features, out_features, mask)[source]

Bases: Linear

Build a masked linear layer.

Parameters

  • in_features (int) – The number of input features.

  • out_features (int) – The number of output features.

  • mask (ndarray) – The mask to apply to the weights of the layer.

Raises

ValueError – If the mask parameter is not consistent with the number of input and output features.

forward(x)[source]

Evaluate the layer given some inputs.

Parameters

x (Tensor) – The inputs.

Returns

The outputs of the module.

Return type

Tensor

in_features: int
out_features: int
weight: Tensor
class deeprob.torch.utils.WeightNormConv2d(in_channels, out_channels, kernel_size, stride=1, padding=0, bias=True)[source]

Bases: Module

Initialize a Conv2d layer with weight normalization.

Parameters

  • in_channels (int) – The number of input channels.

  • out_channels (int) – The number of output channels.

  • kernel_size (Union[int, Tuple[int, int]]) – The convolving kernel size.

  • stride (Union[int, Tuple[int, int]]) – The stride of convolution.

  • padding (Union[int, Tuple[int, int]]) – The padding to apply.

  • bias (bool) – Whether to use bias parameters.

forward(x)[source]

Evaluate the weight-normalized convolutional layer.

Parameters

x (Tensor) – The inputs.

Returns

The outputs of the module.

Return type

Tensor

training: bool

Module contents