Mixtral-8x7B-DPO hallucinates empty spaces until reach max_tokens while generating markdown tables
I’m using the Nous-Hermes-2-Mixtral-8x7B-DPO model via the Hugging Face Transformers library within a RAG (Retrieval-Augmented Generation) application that I’m deploying with LangChain. Most of my documents are web pages scraped and converted to markdown for indexing. However, I’m encountering an issue where a significant number of my queries that request the generation of tables are hitting the max tokens limit, even with high values such as 1K or 2K tokens.