I am working on a project to estimate the weight of pears using their RGB and Depth images in MATLAB. Initially, I used only front-facing images for training, but now I want to improve the accuracy by including images taken from approximately 90 degrees to the left or right of the pear, in addition to the Depth images.

I tried modifying the code to accommodate these new images, but I’m facing some issues. Below is the code that I’m currently using:

cd 'RGB front file path'
folder_name = pwd;
XImageTrainFront = imageDatastore(folder_name);

cd 'RGB side file path'
folder_name = pwd;
XImageTrainSide = imageDatastore(folder_name);

cd 'Depth file path'
folder_name = pwd;
XDispTrain = imageDatastore(folder_name);

YTrain = [ data ]; % weight labels
YTrain = arrayDatastore(YTrain);

% Combine datasets
ds = combine(XImageTrainFront, XImageTrainSide, XDispTrain, YTrain);
dsrand = shuffle(ds);
dsVali = partition(dsrand, 10, 1);
dsTest = partition(dsrand, 10, 2);
dsTrain = partition(dsrand, 10, 3:10);

YTest = readall(dsTest);
YTest = YTest(:,4); % Assuming YTrain is the fourth column
YTest = cell2mat(YTest);

cd 'path to googlenet300400_multiple.mat'
load googlenet300400_multiple.mat

miniBatchSize = 16;
options = trainingOptions('sgdm', ...
    'MiniBatchSize', miniBatchSize, ...
    'MaxEpochs', 300, ...
    'InitialLearnRate', 1e-7, ...
    'LearnRateSchedule', 'piecewise', ...
    'LearnRateDropFactor', 0.1, ...
    'LearnRateDropPeriod', 30, ...
    'Shuffle', 'every-epoch', ...
    'ValidationData', dsVali, ...
    'ValidationFrequency', 50, ...
    'Plots', 'training-progress', ...
    'Verbose', true);

net = trainNetwork(dsTrain, lgraph_2, options);

YPredicted = predict(net, dsTest);
predictionError = YTest - YPredicted;

squares = predictionError.^2;
rmse = sqrt(mean(squares));
figure
scatter(YPredicted, YTest, '+');
xlabel("Predicted Value");
ylabel("True Value");
hold on;
plot([100 550], [100 550], 'r--');