I am using Spring AI 1.0.0-M1 trying to submit an image to GPT-4o for description.
Below is the code:
<code>@Autowired
final ChatClient chatClient;
@Override
public String getDescription(MultipartFile file) throws IOException {
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
.withModel(OpenAiApi.ChatModel.GPT_4_O.getValue())
.build();
Resource resource = new ByteArrayResource(file.getBytes());
var userMessage = new UserMessage(
"Explain what do you see in this picture?",
List.of(new Media(MimeTypeUtils.IMAGE_JPEG, resource)));
var response = chatClient.prompt().user(userMessage.getContent()).call().chatResponse();
return response.getResult().getOutput().getContent();
</code>
<code>@Autowired
final ChatClient chatClient;
@Override
public String getDescription(MultipartFile file) throws IOException {
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
.withModel(OpenAiApi.ChatModel.GPT_4_O.getValue())
.build();
Resource resource = new ByteArrayResource(file.getBytes());
var userMessage = new UserMessage(
"Explain what do you see in this picture?",
List.of(new Media(MimeTypeUtils.IMAGE_JPEG, resource)));
var response = chatClient.prompt().user(userMessage.getContent()).call().chatResponse();
return response.getResult().getOutput().getContent();
</code>
@Autowired
final ChatClient chatClient;
@Override
public String getDescription(MultipartFile file) throws IOException {
OpenAiChatOptions chatOptions = OpenAiChatOptions.builder()
.withModel(OpenAiApi.ChatModel.GPT_4_O.getValue())
.build();
Resource resource = new ByteArrayResource(file.getBytes());
var userMessage = new UserMessage(
"Explain what do you see in this picture?",
List.of(new Media(MimeTypeUtils.IMAGE_JPEG, resource)));
var response = chatClient.prompt().user(userMessage.getContent()).call().chatResponse();
return response.getResult().getOutput().getContent();
}
I keep getting this message:
<code>"I'm sorry, but I can't see pictures or images. If you describe the picture to me, I'll do my best to help you understand or analyze it!"
</code>
<code>"I'm sorry, but I can't see pictures or images. If you describe the picture to me, I'll do my best to help you understand or analyze it!"
</code>
"I'm sorry, but I can't see pictures or images. If you describe the picture to me, I'll do my best to help you understand or analyze it!"
Am I doing something wrong?