I’m working on Dynamic Bernoulli Embeddings using Python, but I’m encountering an “IndexError: index out of range in self” when I attempt to train the model. I’m a beginner in NLP, so I’m struggling to understand and fix this problem.

Here’s the relevant portion of my code:

import pickle
import re
import numpy as np
import pandas as pd
from dynamic_bernoulli_embeddings.analysis import DynamicEmbeddingAnalysis
from dynamic_bernoulli_embeddings.training import train_model
from nltk import word_tokenize as nltk_word_tokenize
from gensim.corpora import Dictionary
from tqdm.notebook import tqdm
tqdm.pandas()

# bow contains tokenized words
dataset = df2[['bow', 'time', 'year']]

# Create a Gensim dictionary
dictionary = Dictionary(dataset['bow'])
dictionary.filter_extremes(no_below=2, no_above=1.0)
dictionary.compactify()

# Training the Dynamic Bernoulli Embeddings model
model, loss_history = train_model(
    dataset, dictionary.token2id, validation=.1, num_epochs=5, k=50)

The following error occurs during the model training:

IndexError: index out of range in self

Traceback:
  File "<ipython-input-44-1d8d4321a42a>", in <cell line: 2>
    model, loss_history = train_model(
      dataset, dictionary.token2id, validation=.1, num_epochs=5, k=50)

7 frames
  File "/usr/local/lib/python3.10/dist-packages/torch/nn/functional.py", in embedding
    return torch.embedding(weight, input, padding_idx, scale_grad_by_freq, sparse)

• I’m using the Dynamic Bernoulli Embeddings library from this repository: [Dynamic Bernoulli Embeddings].(https://www.kaggle.com/code/llefebure/dynamic-bernoulli-embeddings#Dynamic-Bernoulli-Embeddings)

What steps can I take to resolve it?