I’ve got an iOS app that is using MetalKit to display raw video frames coming in from a network source. I read the pixel data in the packets into a single MTLTexture rows at a time, which is drawn into an MTKView each time a frame has been completely sent over the network. The app works, but only for several seconds (a seemingly random duration), before the MTKView seemingly freezes (while packets are still being received).

Watching the debugger while my app was running revealed that the freezing of the display happened when there was a large spike in memory. Seeing the memory profile in Instruments revealed that the spike was related to a rapid creation of many IOSurfaces and IOAccelerators.

Instruments View
Stack trace

Being a complete newbie to iOS programming as a whole, I wonder if this issue comes from a misuse of the MetalKit library. Below is the code that I’m using to render the video frames themselves:

class MTKViewController: UIViewController, MTKViewDelegate {
    
    /// Metal texture to be drawn whenever the view controller is asked to render its view.
    
    private var metalView: MTKView!
    private var device = MTLCreateSystemDefaultDevice()
    private var commandQueue: MTLCommandQueue?
    private var renderPipelineState: MTLRenderPipelineState?
    private var texture: MTLTexture?

    private var networkListener: NetworkListener!
    private var textureGenerator: TextureGenerator!

    override public func loadView() {
        super.loadView()
        assert(device != nil, "Failed creating a default system Metal device. Please, make sure Metal is available on your hardware.")
        
        initializeMetalView()
        initializeRenderPipelineState()
        
        networkListener = NetworkListener()
        textureGenerator = TextureGenerator(width: streamWidth, height: streamHeight, bytesPerPixel: 4, rowsPerPacket: 8, device: device!)
        
        networkListener.start(port: NWEndpoint.Port(8080))
        networkListener.dataRecievedCallback = { data in
            self.textureGenerator.process(data: data)
        }
        
        textureGenerator.onTextureBuiltCallback = { texture in
            self.texture = texture
            self.draw(in: self.metalView)
        }
        
        commandQueue = device?.makeCommandQueue()
    }
    
    public func mtkView(_ view: MTKView, drawableSizeWillChange size: CGSize) {
        /// need implement?
    }
    
    public func draw(in view: MTKView) {
        guard
            let texture = texture,
            let _ = device
        else { return }
    
        let commandBuffer = commandQueue!.makeCommandBuffer()!
        guard
            let currentRenderPassDescriptor = metalView.currentRenderPassDescriptor,
            let currentDrawable = metalView.currentDrawable,
            let renderPipelineState = renderPipelineState
        else { return }
        
        currentRenderPassDescriptor.renderTargetWidth = streamWidth
        currentRenderPassDescriptor.renderTargetHeight = streamHeight
        
        let encoder = commandBuffer.makeRenderCommandEncoder(descriptor: currentRenderPassDescriptor)!
        encoder.pushDebugGroup("RenderFrame")
        encoder.setRenderPipelineState(renderPipelineState)
        encoder.setFragmentTexture(texture, index: 0)
        encoder.drawPrimitives(type: .triangleStrip, vertexStart: 0, vertexCount: 4, instanceCount: 1)
        encoder.popDebugGroup()
        encoder.endEncoding()
        
        commandBuffer.present(currentDrawable)
        commandBuffer.commit()
    }

    private func initializeMetalView() {
        metalView = MTKView(frame: CGRect(x: 0, y: 0, width: streamWidth, height: streamWidth), device: device)
        metalView.delegate = self
        metalView.framebufferOnly = true
        metalView.colorPixelFormat = .bgra8Unorm
        metalView.contentScaleFactor = UIScreen.main.scale
        metalView.autoresizingMask = [.flexibleWidth, .flexibleHeight]
        view.insertSubview(metalView, at: 0)
    }
    
    /// initializes render pipeline state with a default vertex function mapping texture to the view's frame and a simple fragment function returning texture pixel's value.
    private func initializeRenderPipelineState() {
        guard let device = device, let library = device.makeDefaultLibrary() else {
            return
        }
        
        let pipelineDescriptor = MTLRenderPipelineDescriptor()
        pipelineDescriptor.rasterSampleCount = 1
        pipelineDescriptor.colorAttachments[0].pixelFormat = .bgra8Unorm
        pipelineDescriptor.depthAttachmentPixelFormat = .invalid
        
        /// Vertex function to map the texture to the view controller's view
        pipelineDescriptor.vertexFunction = library.makeFunction(name: "mapTexture")
        /// Fragment function to display texture's pixels in the area bounded by vertices of `mapTexture` shader
        pipelineDescriptor.fragmentFunction = library.makeFunction(name: "displayTexture")
        
        do {
            renderPipelineState = try device.makeRenderPipelineState(descriptor: pipelineDescriptor)
        }
        catch {
            assertionFailure("Failed creating a render state pipeline. Can't render the texture without one.")
            return
        }
    }
}

My question is simply: what gives?