mwptoolkit.loss.smoothed_cross_entropy_loss

class mwptoolkit.loss.smoothed_cross_entropy_loss.SmoothCrossEntropyLoss(weight=None, mask=None, size_average=True)[source]

Bases: AbstractLoss

Computes cross entropy loss with uniformly smoothed targets.

Cross entropy loss with uniformly smoothed targets.

Parameters

  • smoothing (float) – Label smoothing factor, between 0 and 1 (exclusive; default is 0.1)

  • ignore_index (int) – Index to be ignored. (PAD_ID by default)

  • reduction (str) – Style of reduction to be done. One of ‘batchmean’(default), ‘none’, or ‘sum’.

eval_batch(outputs, target)[source]

calculate loss

Parameters

  • outputs (Tensor) – output distribution of model.

  • target (Tensor) – target classes.

get_loss()[source]

return loss

Returns

loss (float)

class mwptoolkit.loss.smoothed_cross_entropy_loss.SmoothedCrossEntropyLoss(smoothing: float = 0.1, ignore_index: int = -1, reduction: str = 'batchmean')[source]

Bases: Module

Computes cross entropy loss with uniformly smoothed targets.

Cross entropy loss with uniformly smoothed targets.

Parameters

  • smoothing (float) – Label smoothing factor, between 0 and 1 (exclusive; default is 0.1)

  • ignore_index (int) – Index to be ignored. (PAD_ID by default)

  • reduction (str) – Style of reduction to be done. One of ‘batchmean’(default), ‘none’, or ‘sum’.

forward(input: Tensor, target: LongTensor) → Tensor[source]

Computes cross entropy loss with uniformly smoothed targets. Since the entropy of smoothed target distribution is always same, we can compute this with KL-divergence.

Parameters

  • input (torch.Tensor) – Log probability for each class. This is a Tensor with shape [B, C]

  • target (torch.LongTensor) – List of target classes. This is a LongTensor with shape [B]

Return type

torch.Tensor

Returns

Computed loss

training: bool