mwptoolkit.data.dataloader.abstract_dataloader

class mwptoolkit.data.dataloader.abstract_dataloader.AbstractDataLoader(config, dataset)

Bases: object

abstract dataloader

the base class of dataloader class

Parameters

• config –

• dataset –

expected that config includes these parameters below:

model (str): model name.

equation_fix (str): [infix | postfix | prefix], convert equation to specified format.

train_batch_size (int): the training batch size.

test_batch_size (int): the testing batch size.

symbol_for_tree (bool): build output symbols for tree or not.

share_vocab (bool): encoder and decoder of the model share the same vocabulary, often seen in Seq2Seq models.

max_len (int|None): max input length.

max_equ_len (int|None): max output length.

add_sos (bool): add sos token at the head of input sequence.

add_eos (bool): add eos token at the tail of input sequence.

device (torch.device):

convert_idx_2_symbol(equation_idx: List[int])

convert symbol index of equation to symbol. :param equation_idx: :return:

convert_idx_2_word(sentence_idx: List[int])

convert token index of input sequence to token. :param sentence_idx: :return:

convert_symbol_2_idx(equation: List[str])

convert symbol of equation to index. :param equation: :return:

convert_word_2_idx(sentence: List[str])

convert token of input sequence to index. :param sentence: List[str] :return:

init_batches()

initialize batches.

load_data()

load data.

load_next_batch()

load data.