Assingment

• Task: Segment images based on texture without using supervised learning methods.

• Test Dataset: 10 images from the Prague dataset, generated from the website https://mosaic.utia.cas.cz/.

• My Algorithm:
  The algorithm is based on the concept of Principle Representative Patterns (PRP), inspired by the method described in the article A DATA-CENTRIC APPROACH TO UNSUPERVISED TEXTURE SEGMENTATION USING PRINCIPLE REPRESENTATIVE PATTERNS (pages 3–4).

• GitHub Repository:
  You can find the full code for my implementation here: Texture-Segmentation-App.

Questions:

1. Is the Gaussian convergence formula correctly implemented in the compute_texture_features method?

   • The article explains: how to compute the texture features ( T(i) ) using Principle Representative Patches (PRPs). To assess the similarity between the image patch ( P_i ) and the PRP, a Gaussian weighting function is used, inspired by the famous non-local means denoising algorithm. The similarity between patches is calculated using the Gaussian function with parameters ( sigma ) (the standard deviation) and ( h ) (the smoothing factor). The value ( T(i) ) is interpreted as a probability mass function or an energy spectrum, where each value of ( T(i) ) describes the probability that patch ( P_i ) matches the main texture pattern or the energy that ( P_i ) projects onto the corresponding representative texture pattern. Higher values indicate higher probability and more intense energy concentration, while lower values provide complementary evidence to better describe ( P_i ).

   • Formulas:

     1. The formula for the similarity weight function is:

     w(P_i, P_j) = 1 / Z(i) * exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )
     

     where:

     • σ > 0 is the standard deviation of the Gaussian kernel,
     • h is the smoothing factor,
     • Z(i) is the normalization constant.
     2. The formula for the normalization constant Z(i) is:

     Z(i) = Σ_j exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )
     

   • Screen: Gaussian Similarity Formula.

   • My implementation: I’m not sure if this is the correct implementation. I got some help from ChatGPT here. One issue is that the article used a parameter of 0.1 for Gaussian convergence on my dataset. However, it only works consistently with 1.0 for me. So, I think I may have made a mistake somewhere in implementing this formula.

    @staticmethod
     def compute_texture_features(
        image: np.ndarray,
        prp_centroids: np.ndarray,
        *,
        patch_size: Tuple[int, int] = (7, 7),
        patch_step: int = 1,
        gamma: float = 1.0
     ) -> np.ndarray:
        # Get image dimensions
        height, width = image.shape
   
        # Initialize array to store texture features
        texture_features = np.zeros((height, width))
   
        # Extract patches from the image
        patches = TextureSegmentation.extract_patches(image, patch_size, patch_step)
        print(patches.shape)
   
        patch_vectors = patches.reshape(patches.shape[0], -1)
   
        # Reshape PRP centroids
        prp_centroids = prp_centroids.reshape(prp_centroids.shape[0], -1)
   
        # Calculate RBF kernel similarities
        similarities = rbf_kernel(patch_vectors, prp_centroids, gamma=gamma)
        # Normalize similarities
        sum_similarities = similarities.sum(axis=1, keepdims=True)
        probabilities = np.divide(similarities, sum_similarities,
                                  where=sum_similarities > 0,
                                  out=np.full_like(similarities, 1.0 / similarities.shape[1]))
   
        # Average probabilities across all patches for each PRP
        mean_probabilities = np.mean(probabilities, axis=0)
        contrast_weights = np.abs(probabilities - mean_probabilities)  # Difference from global mean
   
        # Update probabilities with contrast weights
        enhanced_probabilities = probabilities * contrast_weights
   
        # Get cluster assignments
        cluster_assignments = np.argmax(enhanced_probabilities, axis=1)
   
        # Map the results back to the image dimensions
        n_patches_h = (height - patch_size[0]) // patch_step + 1
        n_patches_w = (width - patch_size[1]) // patch_step + 1
        offset_h = patch_size[0] // 2
        offset_w = patch_size[1] // 2
        for idx, cluster in enumerate(cluster_assignments):
          # Find the coordinates of the starting point of the patch
          row_start = (idx // n_patches_w) * patch_step
          col_start = (idx % n_patches_w) * patch_step
   
          # Update the area corresponding to the patch
          texture_features[row_start:row_start + patch_size[0], col_start:col_start + patch_size[1]] = cluster
   
        return texture_features