I have an created a queue to store images from ImageReader in cpp.
while copying the queued items to the Image object from MediaCodec.getInputImage(index) method I’m facing an issue where the byte buffers are successfully copied but the timestamp is not set.
there are no errors thrown and so I’m very confused.
The queue holds objects of the following class.
class YUV420 {
public:
int width;
int height;
**long long timestampUs;**
std::vector<YUVImagePlane> planes;
YUV420(int width, int height, long long timestampUs)
: width(width), height(height), timestampUs(timestampUs) {
// Pre-allocate planes for Y, U, and V
planes.emplace_back(width * height, 1, width); // Y plane
planes.emplace_back(width * height / 2, 2, width); // U plane
planes.emplace_back(width * height / 2, 2, width); // V plane
}
....
The relevant code for the copy function is:
extern "C"
JNIEXPORT void JNICALL
Java_com_example_YuvUtils_copyToImage2(
JNIEnv *env,
jobject /* this */,
jobject image // The MediaCodec Image object from Kotlin)
) {
...
YUV420 &yuvFrame = yuvQueue.dequeue();
jclass imageClass = env->GetObjectClass(image);
// Get the setTimestamp method ID
jmethodID setTimestampMethodID = env->GetMethodID(imageClass, "setTimestamp", "(J)V");
if (setTimestampMethodID == nullptr) {
LOGE("setTimestamp method not found in Image class.");
env->DeleteLocalRef(imageClass);
return;
}
// Call the setTimestamp method
env->CallVoidMethod(image, setTimestampMethodID, static_cast<jlong>(yuvFrame.timestampUs));
// Check for exceptions
if (env->ExceptionCheck()) {
env->ExceptionDescribe(); // Print the exception to the log
env->ExceptionClear(); // Clear the exception so the JNI call can proceed
LOGE("Exception occurred while setting timestamp.");
} else {
LOGI("Timestamp set to %lld", yuvFrame.timestampUs);
}
// Verify the value by getting the current timestamp from the Image object
jmethodID getTimestampMethodID = env->GetMethodID(imageClass, "getTimestamp", "()J");
if (getTimestampMethodID == nullptr) {
LOGE("getTimestamp method not found in Image class.");
} else {
jlong retrievedTimestamp = env->CallLongMethod(image, getTimestampMethodID);
if (retrievedTimestamp == static_cast<jlong>(yuvFrame.timestampUs)) {
LOGI("Timestamp verification succeeded.");
} else {
LOGE("Timestamp verification failed. Expected: %lld, but got: %lld",
yuvFrame.timestampUs, retrievedTimestamp);
}
}
// planes will be copied below.... that part is working
}
a sample output for this code’s log is as follows:
Timestamp set to 608659736453232
Timestamp verification failed. Expected: 608659736453232, but got: 0
Can you tell me what I’m doing wrong here?
Thanks!
5
getInputBuffer(index) returns a writable buffer that we use to write the frame data into.
And when ready, we need to queue it to the Encoder MediaCodec for encoding using queueInputBuffer which would need buffer metadata, timestamp and frame flags.
Here is the complete (Java) version for your info:
Obtain the input buffer:
Call getInputBuffer(index) to obtain a writable buffer from the codec. You will write the raw frame data (e.g., YUV data) into this buffer for encoding.
ByteBuffer inputBuffer = codec.getInputBuffer(index);
if (inputBuffer != null) {
// Clear the buffer
inputBuffer.clear();
// Write frame data into the input buffer
inputBuffer.put(frameData);
}
Queue the input buffer:
After writing the frame data, you need to queue the buffer using queueInputBuffer. You’ll also provide metadata such as the buffer index, offset, size of the frame data, timestamp, and flags.
codec.queueInputBuffer(index, 0, frameData.length, presentationTimeUs, flags);
- index: The index of the input buffer
- 0: Offset in the buffer where
data begins (usually 0). - frameData.length: Size of the frame data in
the buffer. - presentationTimeUs: The presentation timestamp in
microseconds. It helps in ordering frames during playback. - flags: Frame-specific flags like BUFFER_FLAG_END_OF_STREAM or
BUFFER_FLAG_KEY_FRAME
End of Stream flag
You’ll also need to send an End of stream frame.
codec.queueInputBuffer(index, 0, frameData.length, presentationTimeUs, MediaCodec.BUFFER_FLAG_END_OF_STREAM);
2