mwptoolkit.data.dataloader.multi_equation_dataloader

class mwptoolkit.data.dataloader.multi_equation_dataloader.MultiEquationDataLoader(config: Config, dataset: MultiEquationDataset)

Bases: AbstractDataLoader

multiple-equation dataloader

Parameters

• config –

• dataset –

expected that config includes these parameters below:

model (str): model name.

equation_fix (str): [infix | postfix | prefix], convert equation to specified format.

train_batch_size (int): the training batch size.

test_batch_size (int): the testing batch size.

symbol_for_tree (bool): build output symbols for tree or not.

share_vocab (bool): encoder and decoder of the model share the same vocabulary, often seen in Seq2Seq models.

max_len (int|None): max input length.

max_equ_len (int|None): max output length.

add_sos (bool): add sos token at the head of input sequence.

add_eos (bool): add eos token at the tail of input sequence.

device (torch.device):

build_batch_for_predict(batch_data: List[dict])

init_batches()

Initialize batches of trainset, validset and testset. :return: None

load_data(type: str)

Load batches, return every batch data in a generator object.

Parameters

type – [train | valid | test], data type.

Returns

Generator[dict], batches

load_next_batch(type: str) → dict

Return next batch data :param type: [train | valid | test], data type. :return: batch data

mwptoolkit.data.dataloader.multi_equation_dataloader.get_num_mask(num_size_batch, generate_nums)