When using the VertexAI API, it works 80% of the time, but every fifth file being processed errors out with Error message: “Internal error encountered.” Status: 500 Error code: 13. It does not seem to be an intermittent failure, rather, specific files fail on the generate_content method.

      response = model.generate_content(
      contents=user_prompt_content,
      generation_config=GenerationConfig(temperature=0, response_mime_type="application/json"))

I’m assuming there is some special character in the file or some attribute of these files which drive these failures, but the error responses are not descriptive enough to understand the problem. How would one figure out what the issue is?

A 1/5 error rate is so frequent that a multi-LLM approach ends up being required here unless there is more guidance on what drives the error…