How can I render frames received from the MediaFrameReader in Unity?
I gain access to the bitmap frame through the buffer:
uint32_t pixelBufferDataLength;
uint8_t* pixelBufferData;
MediaFrameReference frame = MediaFrameReader.TryAcquireLatestFrame()
winrt::Windows::Graphics::Imaging::BitmapBuffer bitmapBuffer = frame.VideoMediaFrame().SoftwareBitmap().LockBuffer(winrt::Windows::Graphics::Imaging::BitmapBufferAccessMode::Read);
auto spMemoryBufferByteAccess{ bitmapBuffer.CreateReference().as<::Windows::Foundation::IMemoryBufferByteAccess>() };
winrt::check_hresult(spMemoryBufferByteAccess->GetBuffer(&pixelBufferData, &pixelBufferDataLength));
I render the buffer on a texture in Unity:
texPv = new Texture2D(1080, 720, TextureFormat.BGRA32, false);
pixel32Pv = texPv.GetPixels32();
pixelHandlePv = GCHandle.Alloc(pixel32Pv, GCHandleType.Pinned);
pixelPtrPv = pixelHandlePv.AddrOfPinnedObject();
[...]
getCptr(pixelPtrPv);
texPv.SetPixels32(pixel32Pv);
texPv.Apply();
This approach results in a texture output with the wrong format (see image).
PV view capture and texture rendered in Hololens
I tried converting the raw buffer data into a different format but did not succeed.