I’m developing a Flutter app using the Google MLkit Pose Detection plugin
It works really well. Except for one weird thing that I just can’t seem to solve.

The first time I call poseDetector.processImage(someInputImage) It takes really long. About 20-30 seconds and at the end the app freezes for a bit. After this initial process image call everything is smooth sailing.

I have a workaround now where I create a simple dummy image InputImage.frombytes when the app starts. A provider keeps track of this operation and flips a bool when it’s done waiting for the processImage.

I have no idea why this is. My guess is that the first time the app starts fresh, the model is downloaded from google (I’m guessing here). And after that is done it will load this model much quicker. I’m just a bit surprised that I can’t seem to find a way to bundle the pose model with my assets. Or find a way to properly preload the model.

The UX in my app is terrible when opening the camera the first time the app is started.

Does anyone know why this is, and what i could do to make it smoother?

The pose detector is defined like this:

  final PoseDetector _poseDetector = PoseDetector(
    options: PoseDetectorOptions(
      mode: PoseDetectionMode.stream,
      model: PoseDetectionModel.base,
    ),
  );

And no matter how I create the inputImage for the first call to the processImage function this long delay happens.

I “warm up” the pose detector like this now:

  Future<void> _warmupPoseDetector() async {
    // Use a dummy image with dimensions and format that match what the detector expects
    const int width = 640;
    const int height = 480;
    const int bytesPerPixel = 4; 
    const int totalBytes = width * height * bytesPerPixel;
    
    final Uint8List bytes = Uint8List(totalBytes);

    final dummyImage = InputImage.fromBytes(
      bytes: bytes,
      metadata: InputImageMetadata(
        size: Size(width.toDouble(), height.toDouble()),
        rotation: InputImageRotation.rotation0deg,
        format: InputImageFormat.bgra8888,
        bytesPerRow: width * bytesPerPixel,
      ),
    );

    try {
      await _poseDetector.processImage(dummyImage);
      LogService.log('Pose detector model warmed up');
    } catch (e) {
      LogService.log('Error during pose detector warmup: $e', level: LogLevel.error);
    }
  }

But like I mentioned this takes just as long as if I would send a fram from the camera stream. The first call , after installing the app fresh, is always extremely slow and makes the app freeze a little bit.