I’m trying to pull images from a pdf and allow users to select an image from a grid. Essentially I’m working on a WYSIWYG SOP editor. A user creates an SOP (Standard Operating Procedure) using some reference material and can insert images into the SOP from the source material.

I can extract the images in C# (backend) or js (front end Nuxt3) no problem; however, some pdfs have images that are essentially composite images. Let’s say I have an image of a car. I extract the image and in reality it’s 5 sub images stitched together. I’ve tried stitching the images together by detecting overlapping lines but sometimes they are broken up diagonally or in some weird shape which makes things super complicated.

I’ve attached my initial attempt to read images from a pdf on the front end using tato30/vue-pdf. As you can see from the sandbox below, it pulls the initial 2 images fine but then there are tons of partials that are potentially overlays? What is strange is that I also provided a url in the sandbox to a pdf of just the single page rather than the entire pdf and if you sub that into the VuePDF component, you only get two images…

https://codesandbox.io/p/devbox/ptmy44

EDIT
In case you can’t view the code sandbox, here is the code.

<script setup lang="ts">
  import { ref } from 'vue';
  import { VuePDF, usePDF } from '@tato30/vue-pdf';
  import * as PDFJS from 'pdfjs-dist';

  const singlePagePdf = ref('https://stackoverflowrandomfile.blob.core.windows.net/randomfile/split-image-example.pdf');
  const multiPagePdf = ref('https://stackoverflowrandomfile.blob.core.windows.net/randomfile/Walmart Supply Chain Standards 2023.pdf');
  const currentPage = ref(48);
  
  let { pdf, pages } = usePDF(multiPagePdf.value);
  const loaded = ref(false);
  
  const switchPdf = () => {
    loaded.value = false;
    images.value = [];
    let returnObj;
    if (currentPage.value === 48) {
      returnObj = usePDF(singlePagePdf.value);
      currentPage.value = 1
    }
     else {
      returnObj = usePDF(multiPagePdf.value);
      currentPage.value = 48
    }
    pdf = returnObj.pdf;
  }

  const images = ref<Array<{
    url: string;
    transform: any | null; 
    page: number;
    width: number;
    height: number;
  }>>([]);
  
  const getPageImages = (page: number, documentUrl: string) => {
    loaded.value = true;
    pdf.value.promise.then(async (document: any) => { // Replace 'any' with appropriate type if available
      const pageProxy = await document.getPage(page);
      const ops = await pageProxy.getOperatorList();
      const objs: Array<{ type: string; transform: any | null; imageId: string }> = [];
      
      for (let i = 0; i < ops.fnArray.length; i++) {
        if (
          ops.fnArray[i] === PDFJS.OPS.paintImageXObject ||
          ops.fnArray[i] === PDFJS.OPS.transform
        ) {
          const argsVals = ops.argsArray[i];
          objs.push({
            type: ops.fnArray[i] === PDFJS.OPS.paintImageXObject ? 'image' : 'transform',
            transform: ops.fnArray[i] === PDFJS.OPS.transform ? argsVals : null,
            imageId: argsVals[0],
          });
        }
      }

      objs.map(async (val, i) => {
        if (val.type === 'image') {
          const imageKey = val.imageId;
          pageProxy.objs.get(imageKey, async (obj: any) => { // Replace 'any' with appropriate type if available
            const bitmap = await createImageBitmap(obj.bitmap);
            const ocanvas = new OffscreenCanvas(bitmap.width, bitmap.height);

            const ctx = ocanvas.getContext("bitmaprenderer");
            if (ctx) {
              ctx.transferFromImageBitmap(bitmap);
              const blob = await ocanvas.convertToBlob({ type: "image/png" });
              const blobUrl = URL.createObjectURL(blob);

              const transform = objs.length > 1 && i > 0 ? objs[i - 1] : null;
              images.value.push({
                url: blobUrl,
                transform: transform ? transform.transform : null,
                page,
                width: ocanvas.width,
                height: ocanvas.height,
              });
            }
          });
        }
      });
    })
  };
</script>

<template>
  <button v-if="loaded" @click="switchPdf">Switch Pdf</button>
  <VuePDF :key="currentPage" @loaded="getPageImages(currentPage)" :pdf="pdf" :page="currentPage"></VuePDF>
  <ul v-if="loaded">
    <li v-for="image in images">
        <img :src="image.url"/>
    </li>
  </ul>
  <div v-else>Loading large pdf...</div>
 </template>

<style scoped>
.read-the-docs {
  color: #888;
}
</style>

Are there any known algorithms out there that can get join together images using shared sides?