I have a ffmpeg-autogen
video decoder, im trying to use a D3D11
as HW accelerator, but the frame i get in result(with NV12
format) is broken. If im doing the same with Vulkan
as accelerator, everything works fine, but vulkan
uses more graphic card resources..
For rendering i use OpenTK, received frames i convert to RGB with textures and shaders.
Below my decoder initialization.
public VideoStreamDecoder(AVCodecParameters* parameters, AVCodec* codec)
{
AVCodec* _codec = codec;
_pCodecContext = ffmpeg.avcodec_alloc_context3(_codec);
ffmpeg.av_hwdevice_ctx_create(&_pCodecContext->hw_device_ctx, AVHWDeviceType.AV_HWDEVICE_TYPE_D3D11VA, null, null, 0).ThrowExceptionIfError();
ffmpeg.avcodec_parameters_to_context(_pCodecContext, parameters).ThrowExceptionIfError();
ffmpeg.avcodec_open2(_pCodecContext, _codec, null).ThrowExceptionIfError();
CodecName = ffmpeg.avcodec_get_name(_codec->id);
FrameSize = new Size(_pCodecContext->width, _pCodecContext->height);
PixelFormat = _pCodecContext->pix_fmt;
_pFrame = ffmpeg.av_frame_alloc();
_receivedFrame = ffmpeg.av_frame_alloc();
}
And frames reading function
public bool TryReadNextFrame(out AVFrame frame, AVPacket packet)
{
int error;
do
{
ffmpeg.avcodec_send_packet(_pCodecContext, &packet).ThrowExceptionIfError();
error = ffmpeg.avcodec_receive_frame(_pCodecContext, _pFrame);
} while (error == ffmpeg.AVERROR(ffmpeg.EAGAIN));
error.ThrowExceptionIfError();
ffmpeg.av_hwframe_transfer_data(_receivedFrame, _pFrame, 0);
//here i get NV12 frame
var clonedFrame = ffmpeg.av_frame_clone(_receivedFrame);
frame = *clonedFrame;
return true;
}
And here is result i get with both accelerators:
Vulkan
normally working
The same camera, but D3D11
broken
One more intresting moment, is that cameras, that dont work have yuv420p
format, the only one working with D3D11
has yuvj420
.
I tried changing format using sws_scale
, and it worked, but it uses too much CPU.
Also tried changin format before av_hwframe_transfer_data function
, it sort of works with Vulkan
, but with D3D11
it doesnt.
dima2012 terminator is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.