Assingment

• Task: Segment images based on texture without using supervised learning methods.

• Test Dataset: 10 images from the Prague dataset, generated from the website https://mosaic.utia.cas.cz/.

• My Algorithm:
  The algorithm is based on the concept of Principle Representative Patterns (PRP), inspired by the method described in the article A DATA-CENTRIC APPROACH TO UNSUPERVISED TEXTURE SEGMENTATION USING PRINCIPLE REPRESENTATIVE PATTERNS (pages 3–4).

• GitHub Repository:
  You can find the full code for my implementation here: Texture-Segmentation-App.

Questions:

1. Is the Gaussian convergence formula correctly implemented in the compute_texture_features method?

   • The article explains: how to compute the texture features ( T(i) ) using Principle Representative Patches (PRPs). To assess the similarity between the image patch ( P_i ) and the PRP, a Gaussian weighting function is used, inspired by the famous non-local means denoising algorithm. The similarity between patches is calculated using the Gaussian function with parameters ( sigma ) (the standard deviation) and ( h ) (the smoothing factor). The value ( T(i) ) is interpreted as a probability mass function or an energy spectrum, where each value of ( T(i) ) describes the probability that patch ( P_i ) matches the main texture pattern or the energy that ( P_i ) projects onto the corresponding representative texture pattern. Higher values indicate higher probability and more intense energy concentration, while lower values provide complementary evidence to better describe ( P_i ).

   • Formulas:

     1. The formula for the similarity weight function is:
     Plain text

     Copy to clipboard

     Open code in new window

     EnlighterJS 3 Syntax Highlighter

     <code>w(P_i, P_j) = 1 / Z(i) * exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )

     </code>

     <code>w(P_i, P_j) = 1 / Z(i) * exp( - || P_i - P_j ||^2 / (2 * σ_h^2) ) </code>

     w(P_i, P_j) = 1 / Z(i) * exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )
     

     where:

     • σ > 0 is the standard deviation of the Gaussian kernel,
     • h is the smoothing factor,
     • Z(i) is the normalization constant.
     2. The formula for the normalization constant Z(i) is:
     Plain text

     Copy to clipboard

     Open code in new window

     EnlighterJS 3 Syntax Highlighter

     <code>Z(i) = Σ_j exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )

     </code>

     <code>Z(i) = Σ_j exp( - || P_i - P_j ||^2 / (2 * σ_h^2) ) </code>

     Z(i) = Σ_j exp( - || P_i - P_j ||^2 / (2 * σ_h^2) )
     

   • Screen: Gaussian Similarity Formula.

   • My implementation: I’m not sure if this is the correct implementation. I got some help from ChatGPT here. One issue is that the article used a parameter of 0.1 for Gaussian convergence on my dataset. However, it only works consistently with 1.0 for me. So, I think I may have made a mistake somewhere in implementing this formula.

   Plain text

   Copy to clipboard

   Open code in new window

   EnlighterJS 3 Syntax Highlighter

   <code> @staticmethod

   def compute_texture_features(

   image: np.ndarray,

   prp_centroids: np.ndarray,

   *,

   patch_size: Tuple[int, int] = (7, 7),

   patch_step: int = 1,

   gamma: float = 1.0

   ) -> np.ndarray:

   # Get image dimensions

   height, width = image.shape

   # Initialize array to store texture features

   texture_features = np.zeros((height, width))

   # Extract patches from the image

   patches = TextureSegmentation.extract_patches(image, patch_size, patch_step)

   print(patches.shape)

   patch_vectors = patches.reshape(patches.shape[0], -1)

   # Reshape PRP centroids

   prp_centroids = prp_centroids.reshape(prp_centroids.shape[0], -1)

   # Calculate RBF kernel similarities

   similarities = rbf_kernel(patch_vectors, prp_centroids, gamma=gamma)

   # Normalize similarities

   sum_similarities = similarities.sum(axis=1, keepdims=True)

   probabilities = np.divide(similarities, sum_similarities,

   where=sum_similarities > 0,

   out=np.full_like(similarities, 1.0 / similarities.shape[1]))

   # Average probabilities across all patches for each PRP

   mean_probabilities = np.mean(probabilities, axis=0)

   contrast_weights = np.abs(probabilities - mean_probabilities) # Difference from global mean

   # Update probabilities with contrast weights

   enhanced_probabilities = probabilities * contrast_weights

   # Get cluster assignments

   cluster_assignments = np.argmax(enhanced_probabilities, axis=1)

   # Map the results back to the image dimensions

   n_patches_h = (height - patch_size[0]) // patch_step + 1

   n_patches_w = (width - patch_size[1]) // patch_step + 1

   offset_h = patch_size[0] // 2

   offset_w = patch_size[1] // 2

   for idx, cluster in enumerate(cluster_assignments):

   # Find the coordinates of the starting point of the patch

   row_start = (idx // n_patches_w) * patch_step

   col_start = (idx % n_patches_w) * patch_step

   # Update the area corresponding to the patch

   texture_features[row_start:row_start + patch_size[0], col_start:col_start + patch_size[1]] = cluster

   return texture_features

   </code>

   <code> @staticmethod def compute_texture_features( image: np.ndarray, prp_centroids: np.ndarray, *, patch_size: Tuple[int, int] = (7, 7), patch_step: int = 1, gamma: float = 1.0 ) -> np.ndarray: # Get image dimensions height, width = image.shape # Initialize array to store texture features texture_features = np.zeros((height, width)) # Extract patches from the image patches = TextureSegmentation.extract_patches(image, patch_size, patch_step) print(patches.shape) patch_vectors = patches.reshape(patches.shape[0], -1) # Reshape PRP centroids prp_centroids = prp_centroids.reshape(prp_centroids.shape[0], -1) # Calculate RBF kernel similarities similarities = rbf_kernel(patch_vectors, prp_centroids, gamma=gamma) # Normalize similarities sum_similarities = similarities.sum(axis=1, keepdims=True) probabilities = np.divide(similarities, sum_similarities, where=sum_similarities > 0, out=np.full_like(similarities, 1.0 / similarities.shape[1])) # Average probabilities across all patches for each PRP mean_probabilities = np.mean(probabilities, axis=0) contrast_weights = np.abs(probabilities - mean_probabilities) # Difference from global mean # Update probabilities with contrast weights enhanced_probabilities = probabilities * contrast_weights # Get cluster assignments cluster_assignments = np.argmax(enhanced_probabilities, axis=1) # Map the results back to the image dimensions n_patches_h = (height - patch_size[0]) // patch_step + 1 n_patches_w = (width - patch_size[1]) // patch_step + 1 offset_h = patch_size[0] // 2 offset_w = patch_size[1] // 2 for idx, cluster in enumerate(cluster_assignments): # Find the coordinates of the starting point of the patch row_start = (idx // n_patches_w) * patch_step col_start = (idx % n_patches_w) * patch_step # Update the area corresponding to the patch texture_features[row_start:row_start + patch_size[0], col_start:col_start + patch_size[1]] = cluster return texture_features </code>

    @staticmethod
     def compute_texture_features(
        image: np.ndarray,
        prp_centroids: np.ndarray,
        *,
        patch_size: Tuple[int, int] = (7, 7),
        patch_step: int = 1,
        gamma: float = 1.0
     ) -> np.ndarray:
        # Get image dimensions
        height, width = image.shape
   
        # Initialize array to store texture features
        texture_features = np.zeros((height, width))
   
        # Extract patches from the image
        patches = TextureSegmentation.extract_patches(image, patch_size, patch_step)
        print(patches.shape)
   
        patch_vectors = patches.reshape(patches.shape[0], -1)
   
        # Reshape PRP centroids
        prp_centroids = prp_centroids.reshape(prp_centroids.shape[0], -1)
   
        # Calculate RBF kernel similarities
        similarities = rbf_kernel(patch_vectors, prp_centroids, gamma=gamma)
        # Normalize similarities
        sum_similarities = similarities.sum(axis=1, keepdims=True)
        probabilities = np.divide(similarities, sum_similarities,
                                  where=sum_similarities > 0,
                                  out=np.full_like(similarities, 1.0 / similarities.shape[1]))
   
        # Average probabilities across all patches for each PRP
        mean_probabilities = np.mean(probabilities, axis=0)
        contrast_weights = np.abs(probabilities - mean_probabilities)  # Difference from global mean
   
        # Update probabilities with contrast weights
        enhanced_probabilities = probabilities * contrast_weights
   
        # Get cluster assignments
        cluster_assignments = np.argmax(enhanced_probabilities, axis=1)
   
        # Map the results back to the image dimensions
        n_patches_h = (height - patch_size[0]) // patch_step + 1
        n_patches_w = (width - patch_size[1]) // patch_step + 1
        offset_h = patch_size[0] // 2
        offset_w = patch_size[1] // 2
        for idx, cluster in enumerate(cluster_assignments):
          # Find the coordinates of the starting point of the patch
          row_start = (idx // n_patches_w) * patch_step
          col_start = (idx % n_patches_w) * patch_step
   
          # Update the area corresponding to the patch
          texture_features[row_start:row_start + patch_size[0], col_start:col_start + patch_size[1]] = cluster
   
        return texture_features