I am creating an Android application where I have a need to isolate just the parts of the detected image that include the bars of the barcode and nothing else. I am struggling to get just the bars without any extra data in the image such as inlayed numbers or things in the image that are outside the barcode itself. I am using Google ML Kit to detect and decode the barcode.

Here is my current approach:

1. Google ML Kit detects and decodes the image.
2. When there is a successful decode, using OpenCV, I determine if the image is sharp enough to capture, as I’ve noticed the image can be blurry at times.
3. I take the image proxy, rotate it so the barcode is upright, and convert it to a bitmap.
4. I take the corner points provided from ML Kit and add a padding of 5 to ensure that I have the entire barcode in the frame, and crop the image to these dimension. (I have a need for this image as well, so this is why I don’t immediately crop to the just the bars of the barcode immediately.)

Now is where I am trying to get the bars of the barcode isolated, but I am unable to do so. Using just the corner points from ML Kit/barcode decoding does not yield the proper results as the barcode can be cut off. Using OpenCV seems to be the most promising, but it’s still not ideal.

Here is the code that I have tried with a lot of debugging and tweaking involved:

private fun cropBarcodeUsingOpenCV(bitmap: Bitmap, context: Context): Bitmap? {
    // Convert Bitmap to OpenCV Mat
    val src = Mat()
    Utils.bitmapToMat(bitmap, src)

    // Convert to grayscale
    val gray = Mat()
    Imgproc.cvtColor(src, gray, Imgproc.COLOR_BGR2GRAY)

    // Apply Gaussian blur to reduce noise
    val blurred = Mat()
    Imgproc.GaussianBlur(gray, blurred, Size(5.0, 5.0), 0.0)

    // Apply Canny edge detection with adjusted thresholds
    val edges = Mat()
    Imgproc.Canny(blurred, edges, 30.0, 100.0)

    // Apply morphological transformations to close gaps
    val kernel = Imgproc.getStructuringElement(Imgproc.MORPH_RECT, Size(7.0, 7.0))
    val morphedEdges = Mat()
    Imgproc.morphologyEx(edges, morphedEdges, Imgproc.MORPH_CLOSE, kernel)

    // Find contours
    val contours = ArrayList<MatOfPoint>()
    val hierarchy = Mat()
    Imgproc.findContours(
        morphedEdges, contours, hierarchy, Imgproc.RETR_EXTERNAL, Imgproc.CHAIN_APPROX_SIMPLE
    )

    // DEBUG: Log the number of contours found
    Log.d("BarcodeProcessor", "Number of contours found: ${contours.size}")

    // Create a copy of the source image to draw contours
    val contourImage = Mat()
    src.copyTo(contourImage)

    // Iterate over contours and draw them
    for (contour in contours) {
        val rect = Imgproc.boundingRect(contour)
        Imgproc.rectangle(contourImage, rect.tl(), rect.br(), Scalar(0.0, 255.0, 0.0), 2)
        Log.d(
            "BarcodeProcessor",
            "Contour - Width: ${rect.width}, Height: ${rect.height}, Aspect Ratio: ${rect.width.toDouble() / rect.height.toDouble()}, Area: ${
                Imgproc.contourArea(contour)
            }"
        )
    }

    // Save the contour image for inspection
    saveMatAsImage(contourImage, "detected_contours.png", context)

    // Now, select the best contour based on stricter criteria
    var barcodeContour: MatOfPoint? = null
    var maxArea = 0.0

    for (contour in contours) {
        val area = Imgproc.contourArea(contour)
        val rect = Imgproc.boundingRect(contour)
        val aspectRatio = rect.width.toDouble() / rect.height.toDouble()

        // Stricter filtering criteria, ensuring the contour represents the barcode
        if (aspectRatio > 2.0 && aspectRatio < 5.0 && area > 1500 && rect.width > 100 && rect.height > 20) {
            // Ensure the contour isn't located in the top part where numbers typically are
            if (rect.y > bitmap.height / 4) {
                maxArea = area
                barcodeContour = contour
            }
        }
    }

    // If no valid barcode contour is found, return null
    if (barcodeContour == null) {
        Log.e("BarcodeProcessor", "No valid barcode contour found.")
        return null
    }

    // Get bounding rectangle around the selected contour
    var boundingRect = Imgproc.boundingRect(barcodeContour)

    // Manually adjust the bounding box to exclude the numbers above and below the barcode
    val topPadding = boundingRect.height / 6  // Slightly increase top padding
    boundingRect.y += topPadding
    boundingRect.height -= topPadding

    // Manually set a limit for the bottom of the barcode, ignoring the numbers
    val bottomLimit = boundingRect.y + boundingRect.height - (boundingRect.height / 3)
    boundingRect.height = bottomLimit - boundingRect.y

    // Ensure the bounding box is still within the image bounds
    boundingRect.y = boundingRect.y.coerceAtLeast(0)
    boundingRect.height =
        (boundingRect.height + boundingRect.y).coerceAtMost(bitmap.height) - boundingRect.y

    // Crop the bitmap using the adjusted bounding rectangle
    val croppedBitmap = Bitmap.createBitmap(
        bitmap, boundingRect.x, boundingRect.y, boundingRect.width, boundingRect.height
    )

    return croppedBitmap
}

This image is the closet I have been able to come to getting just the bars of the barcode, but even then it will still encompass some numbers outside of the barcode and I even need to remove the numbers superimposed in the barcode. If anyone can help me with this function or give me an alternative approach that would be greatly appreciated!