I have a question about loading data in Python for a CNN model. I created two 3D matrices in MATLAB with dimensions 509x985x3779 (each being a collection of 3779 2D matrices that I combined into a 3D matrix).
I want to load only 100 slices of this 3D matrix along the third dimension into Python. However, when I load the matrix in Python and check the size, it appears as 100x985x509 instead of the expected 100x509x985.
I cannot figure out why the dimensions are reversed. Could it be related to the difference in how MATLAB and Python handle array dimensions? How can I ensure the matrix has the correct dimensions in Python?
this is my code:
# Define the paths
data_path_surface = 'E:\New Data\Data\Experiment 4\CNN\preparing data\surface'
file_name_surface = 'Exp4surface_without_Nan.mat'
data_path_velocity = 'E:\New Data\Data\Experiment 4\CNN\preparing data\velocity\final'
file_name_velocity = 'Exp4velocities_without_NaNs.mat'
# Construct the full path to the .mat files
full_path_surface = os.path.join(data_path_surface, file_name_surface)
full_path_velocity = os.path.join(data_path_velocity, file_name_velocity)
# Function to load only the first 100 samples from .mat files
def load_partial_mat_file(file_path, variable_name, num_samples):
try:
data = h5py.File(file_path, ‘r’)
data_partial = np.array(data[variable_name][:num_samples])
except OSError:
data = scipy.io.loadmat(file_path)
data_partial = np.array(data[variable_name][:num_samples])
return data_partial
# Load the first 100 samples from the .mat files
num_samples = 100
surface = load_partial_mat_file(full_path_surface, 'surface_without_Nans', num_samples)
u = load_partial_mat_file(full_path_velocity, 'u', num_samples)
w = load_partial_mat_file(full_path_velocity, 'w', num_samples)
# Print the shapes of the loaded data
print(f"Data in 'surface_without_Nans': {surface.shape}")
print(f"Data in 'u': {u.shape}")
print(f"Data in 'w': {w.shape}")
# Ensure the dimensions match your expectations
assert surface.shape == (100, 985), "Shape of 'surface' does not match expected dimensions (100, 985)"
assert u.shape == (100, 985, 509), "Shape of 'u' does not match expected dimensions (100, 985, 509)"
assert w.shape == (100, 985, 509), "Shape of 'w' does not match expected dimensions (100, 985,509)"
# Prepare input data
input_data = np.zeros((num_samples, 985, 2))
wave_age = 0.1221
for i in range(num_samples):
input_data[i, :, 0] = np.ones(985) * wave_age
input_data[i, :, 1] = surface[i, :]
# Prepare output data
output_data = np.zeros((num_samples, 985, 509, 2))
for i in range(num_samples):
output_data[i, :, :, 0] = u[i, :, :]
output_data[i, :, :, 1] = w[i, :, :]
print("Input and output data prepared successfully.")
# Train the model with the actual data
model.fit(input_data, output_data, epochs=10, batch_size=32)
Data in ‘surface_without_Nans’: (100, 985)
Data in ‘u’: (100, 985, 509)
Data in ‘w’: (100, 985, 509)
I tried to swap the rows and columns, but I was not successful.
Ali Bizhanpour is a new contributor to this site. Take care in asking for clarification, commenting, and answering.
Check out our Code of Conduct.