Relative Content

Tag Archive for pytorchlarge-language-modelseq2seqopennmtbeam-search

Issue with tracking beam and word id in beam search class for seq2seq language model in openNMT file

I have a question regarding the implementation of beam search in the openNMT library, more specifically in the file translate/beam.py. i am using the beam class for a custom seq2seq LM. In this file, for reconstructing which beam and word the respective log probability came from, the authors divide the ids of the topk output on the flattened representation of bos token probs by the num_words variable, which is, according to the comment, the second dim in a tensor of size (K x words), where I can only guess that K shall be the beam size. At the start, when no previous nodes were computed, (line 125), the tensor is indexed, resulting in a tensor of size [words], which results in indeces being divided by the vocab size in line 136 for computing prev_k such that floats are the outcome, which may not serve as indeces in line 138.