I am currently using Vertex AI’s Multimodal Embedding Model (https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings)

I was able to get the Image and Text examples running just fine using the Python SDK and POST, however, when I try to run the following Video example (from the above link), I get the following error

import vertexai

from vertexai.vision_models import MultiModalEmbeddingModel, Video

project_id = 'multimodal-project'
location = 'us-central1'

vertexai.init(project=project_id, location=location)

# Document metadata
video_path = 'gs://multi-assets/videos/dog_jumping_short.mp4' # small 1 MB 7 second video file
description = 'dogs jumping'

model = MultiModalEmbeddingModel.from_pretrained("multimodalembedding@001")

video = Video.load_from_file(video_path)

embeddings = model.get_embeddings(
    video=video
)

# Video Embeddings are segmented based on the video_segment_config.
print("Video Embeddings:")
for video_embedding in embeddings.video_embeddings:
    print(
        f"Video Segment: {video_embedding.start_offset_sec} - {video_embedding.end_offset_sec}"
    )
    print(f"Embedding: {video_embedding.embedding}")

print(f"Text Embedding: {embeddings.text_embedding}")

The error I get when running this sample code is related to Deadline

Traceback (most recent call last):
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/google/api_core/grpc_helpers.py", line 76, in error_remapped_callable
    return callable_(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/grpc/_channel.py", line 1181, in __call__
    return _end_unary_response_blocking(state, call, False, None)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/Users/me/Code/multi-modal-test/multi-modal/lib/python3.12/site-packages/grpc/_channel.py", line 1006, in _end_unary_response_blocking
    raise _InactiveRpcError(state)  # pytype: disable=not-instantiable
    ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
grpc._channel._InactiveRpcError: <_InactiveRpcError of RPC that terminated with:
        status = StatusCode.UNAUTHENTICATED
        details = "Video embedding failed with the following error: Deadline"
        debug_error_string = "UNKNOWN:Error received from peer ipv4:142.250.81.234:443 {grpc_message:"Video embedding failed with the following error: Deadline", grpc_status:16, created_time:"2024-05-23T15:53:15.429704-04:00"}"

Note that I have been able to embed contextual_text and image just fine with the same authentication, so I am fairly certain this has nothing to do with authentication even though the error says so.

I have also tried to POST using the following cURL, but I get the same error response Deadline

curl -X POST  
 -H "Authorization: Bearer $(gcloud auth print-access-token)" 
 -H "Content-Type: application/json; charset=utf-8" 
 -d @request.json 
 "https://us-central1-aiplatform.googleapis.com/v1/projects/multi/locations/us-central1/publishers/google/models/multimodalembedding@001:predict"

Anyone know what I can do here?