mwptoolkit.data.dataloader.template_dataloader

class mwptoolkit.data.dataloader.template_dataloader.TemplateDataLoader(config, dataset)

Bases: AbstractDataLoader

template dataloader.

you need implement:

TemplateDataLoader.__init_batches()

We replace abstract method TemplateDataLoader.load_batch() with TemplateDataLoader.__init_batches() after version 0.0.5 . Their functions are similar.

Parameters

• config –

• dataset –

expected that config includes these parameters below:

model (str): model name.

equation_fix (str): [infix | postfix | prefix], convert equation to specified format.

train_batch_size (int): the training batch size.

test_batch_size (int): the testing batch size.

symbol_for_tree (bool): build output symbols for tree or not.

share_vocab (bool): encoder and decoder of the model share the same vocabulary, often seen in Seq2Seq models.

max_len (int|None): max input length.

max_equ_len (int|None): max output length.

add_sos (bool): add sos token at the head of input sequence.

add_eos (bool): add eos token at the tail of input sequence.

device (torch.device):

init_batches()

Initialize batches of trainset, validset and testset. :return: None

load_data(type: str)

Load batches, return every batch data in a generator object.

Parameters

type – [train | valid | test], data type.

Returns

Generator[dict], batches

load_next_batch(type: str)

Return next batch data :param type: [train | valid | test], data type. :return: batch data