Corpus

struct Corpus {

Dictionary dict;

int[][string] dataset;

size_t batchSize;

void register(string path, string name);

auto batchfy(string name);

auto vocabSize();

}

Members

Functions

batchfy auto batchfy(string name): returns word-id 2d slice shaped (seqlen, batchsize)
register void register(string path, string name): Undocumented in source. Be warned that the author may not have intended to support it.
vocabSize auto vocabSize(): Undocumented in source. Be warned that the author may not have intended to support it.

Variables

batchSize size_t batchSize;: Undocumented in source.
dataset int[][string] dataset;: Undocumented in source.
dict Dictionary dict;: Undocumented in source.

Meta

Source

See Implementation

grain dataset ptb functions

prepareDataset

structs