I’m using Gemini 1.5 Pro on Vertex AI to tag attributes about people. I’m using the forced function calling feature to force the model to output JSON in the provided schema. One of the attributes is a biography string. The model fails to generate quotation symbols in this string—where a quote symbol should appear, the string attribute simply ends. It does not cause the JSON output to be invalid, merely that string attribute to stop.

Here is a reproducible example:

"""Helper functions for interacting with Vertex AI's LLMs"""

import os

import vertexai
from dotenv import load_dotenv
from google.auth import default
from vertexai.generative_models import GenerativeModel
from vertexai.preview.generative_models import (FunctionDeclaration, Tool,
                                                ToolConfig)

load_dotenv()


SCHEMA = {
    "type": "object",
    "properties": {
        "summary": {
            "type": "string",
            "description": "Summary about the person, using full sentences.",
        },
    },
    "required": ["bullet_points"],
}

PROMPT = "The person's summary should be exactly this: The person's name is Jack."
SYSTEM = "You are a person summarizer."

credentials, _ = default()
vertexai.init(project=os.getenv("GCP_PROJECT_NAME"), location="us-east4", credentials=credentials)

tool = Tool(
    function_declarations=[
        FunctionDeclaration(
            name="generate_content",
            parameters=SCHEMA,
        )
    ]
)

tool_config = ToolConfig(
    function_calling_config=ToolConfig.FunctionCallingConfig(
        mode=ToolConfig.FunctionCallingConfig.Mode.ANY,
        allowed_function_names=["generate_content"],
    )
)

model = GenerativeModel("gemini-1.5-pro", tools=[tool], tool_config=tool_config, system_instruction=SYSTEM)

response = model.generate_content(
    PROMPT,
    generation_config={
        "max_output_tokens": 200,
        "temperature": 0,
    },
    tools=[tool],
    tool_config=tool_config,
)

fc = response.candidates[0].content.parts[0].function_call

print(type(fc).to_dict(fc)["args"])

Am I missing something about stop sequences or other parameters? Or is this a bug with Vertex AI?