Relative Content

Tag Archive for huggingface-transformerslarge-language-modelhuggingface-tokenizers

Inconsistent Sum of Log Probabilities in PyTorch Causal Language Model

I’m working on a project using the transformers library from Hugging Face and PyTorch to compute log probabilities from a causal language model. I have two inputs: a single sentence and a batch of two sentences. However, I’m encountering an issue where the sum of log probabilities for the single sentence differs when computed in a batch versus individually.