Details:

I am using PCA for dimensionality reduction on a time series dataset of hand motion capture data and am encountering unexpected clustering behaviour. Below are the details of my process and the issue I am facing.

Context:

I have a dataset of hand motion recordings captured using a smart glove with 4 sensors on each finger. Each sensor provides:

• Positional values: X, Y, Z
• Rotational values: X, Y, Z, W

The data was recorded in separate files, each containing 10 repetitions of a gesture. These repetitions were later segmented into individual files (1 repetition per file), labelled, and combined into a single large dataset.

After label encoding and scaling the data, I applied PCA for dimensionality reduction with the following code:

# PCA for dimensionality reduction
pca_components = 3
pca = PCA(n_components=pca_components)
x_train_pca = pca.fit_transform(x_train_scaled)

To visualise the results, I created a PCA summary plot where each segment of data is represented as a single point. The goal is to see clusters of 10 points for each gesture. Here’s the code to summarise the PCA results:

# Convert PCA results into a DataFrame to associate labels
x_train_pca_df = pd.DataFrame(x_train_pca, columns=['PC1', 'PC2', 'PC3'])
x_train_pca_df['label'] = y_train_data
x_train_pca_df['segment'] = train_data['segment'].values.ravel()

# Calculate the mean of PC1 and PC2 for each dataset (to summarise each dataset as a point)
summary_train_points = x_train_pca_df.groupby(['label', 'segment']).mean().reset_index()

And here’s the method I use to plot the summarised PCA data:

def plot_3d_pca_matplotlib(summary_points, title='3D PCA Plot'):
    fig = plt.figure(figsize=(10, 8))
    ax = fig.add_subplot(111, projection='3d')

    # For each unique dataset (label), scatter the points and assign a legend entry
    unique_labels = summary_points['label'].unique()

    for label in unique_labels:
        # Filter the data for the current label
        filtered_data = summary_points[summary_points['label'] == label]

        # Scatter plot for the current label's points
        ax.scatter(filtered_data['PC1'],
                   filtered_data['PC2'],
                   filtered_data['PC3'],
                   label=label)

    ax.set_xlabel('PCA Component 1')
    ax.set_ylabel('PCA Component 2')
    ax.set_zlabel('PCA Component 3')
    ax.set_title(title)
    ax.legend(title="Labels", loc="center left", bbox_to_anchor=(1.05, 0.7))

    plt.subplots_adjust(left=0.05, right=0.75)
    plt.show()

The Issue:

When I record a new set of data for the same gestures and plot the PCA results, I expect to see clusters of 20 points (10 points from the original data + 10 points from the new data) for each gesture.

Instead, the PCA plot shows two separate clusters of 10 points for each gesture. This suggests that the new data isn’t aligning with the original data in the PCA space, even though the gestures are the same.

Question:

What could be causing this separation into distinct clusters for the same gestures?

Could it be related to:

1. The scaling process?
2. Inconsistent sensor calibration?
3. A need for alignment or additional preprocessing of the data before applying PCA?

Any guidance or suggestions for debugging this issue would be greatly appreciated. If you require more information please let me know.