good day. Thank you for your time.

I’m having trouble converting image and mask data into Dataset that can be used in keras.Model.

I have a list of 100 images with a shape of 128*128*3 (Yes, type(images) = (100, 128, 128, 3))
and I have a list of 100 segmenation_masks with a shape of 128*128*1 (type(segmenation_masks) = (100, 128, 128, 1))

Please look at the code below.

import cv2
from pycocotools import mask as maskUtils
from keras.callbacks import ModelCheckpoint
from pycocotools.coco import COCO
import numpy as np

from unet import *

coco_images_dir = '../dataset/images/'
coco_json_path = '../dataset/COCO_Football Pixel.json'
checkpoint_path = "unet3plus_ckpt.weights.h5"
export_path = "unet3plus_export.weights.h5"
input_size = [128, 128]
batch_size = 30

# Json 파일 로드
coco = COCO(coco_json_path)
img_ids = coco.getImgIds()
cat_ids = coco.getCatIds()
images = []
segmentation_masks = []
for img_id in img_ids:
    img = coco.loadImgs(img_id)[0]
    file_name = coco_images_dir + img['file_name']
    image = cv2.cvtColor(cv2.imread(file_name), cv2.COLOR_BGR2RGB)
    image = image.astype(np.float32) / 255.0
    image = tf.convert_to_tensor(image)
    image = tf.image.resize(image, input_size)
    images.append(image)
    annIds = coco.getAnnIds(imgIds=img_id, iscrowd=None)
    anns = coco.loadAnns(annIds)
    segmentation_mask = np.zeros((input_size[0], input_size[1], 1))
    for ann in anns:
        if 'segmentation' in ann:
            rle = coco.annToRLE(ann)
            mask = maskUtils.decode(rle)
            mask = tf.expand_dims(mask, axis=-1)
            mask = tf.convert_to_tensor(mask)
            mask = tf.image.convert_image_dtype(mask, tf.uint8)
            mask = tf.image.resize(mask, input_size)
            mask *= (cat_ids.index(ann['category_id']) + 1)
            segmentation_mask += mask
    segmentation_masks.append(segmentation_mask)

##################################################################
# What should I do with images and segmentation_masks?
##################################################################
dataset = ...

model = unet((input_size[0], input_size[1], 3), len(coco.getCatIds()))
model.compile(optimizer='adam',
              loss='sparse_categorical_crossentropy',
              metrics=['accuracy'])
model.fit(dataset, epochs=100,
          callbacks=ModelCheckpoint(filepath=checkpoint_path, save_weights_only=True, save_best_only=False))
model.save(export_path)

I searched and tried several methods like slice or TFRecord but to no avail.
Using the above method, the loss was ‘nan’ and the accuracy was less than 1%.
On the other hand, if I loaded the data through tensorflow-dataset’s load function, it worked fine.

What should I do with images and segmentation_masks?

Thank you for your time again.
Have a nice day!