I have a pytorch Dataset subclass and I create a pytorch DataLoader out of it. It works when I return two tensors from DataSet’s __getitem__() method. I tried to create minimal (but not working, more on this later) code as below:

import torch
from torch.utils.data import Dataset
import random

device = torch.device("cuda" if torch.cuda.is_available() else "cpu")

class DummyDataset(Dataset):
    def __init__(self, num_samples=3908, window=10): # same default values as in the original code
        self.window = window
        # Create dummy data
        self.x = torch.randn(num_samples, 10, dtype=torch.float32, device='cpu')  
        self.y = torch.randn(num_samples, 3, dtype=torch.float32, device='cpu')
        self.t = {i: random.choice([True, False]) for i in range(num_samples)}

    def __len__(self):
        return len(self.x) - self.window + 1

    def __getitem__(self, i):
        return self.x[i: i + self.window], self.y[i + self.window - 1] #, self.t[i]

ds = DummyDataset()
dl = torch.utils.data.DataLoader(ds, batch_size=10, shuffle=False, generator=torch.Generator(device='cuda'), num_workers=4, prefetch_factor=16)

for data in dl:
    x = data[0]
    y = data[1]
    # t = data[2]
    print(f"x: {x.shape}, y: {y.shape}") # , t: {t}
    break   

Above code gives following error:

RuntimeError: Expected a 'cpu' device type for generator but found 'cuda'

on line for data in dl:.

But my original code is exactly like above: dataset contains tensors created on cpu and dataloader’s generator’s device set to cuda and it works (I mean above minimal code does not work, but same lines in my original code does indeed work!).

When I try to return a boolean value from it by un-commenting , self.t[i] from __get_item__() method, it gives me following error:

Traceback (most recent call last):
  File "/my_project/src/train.py", line 66, in <module>
    trainer.train_validate()
  File "/my_project/src/trainer_cpu.py", line 146, in train_validate
    self.train()
  File "/my_project/src/trainer_cpu.py", line 296, in train
    for train_data in tqdm(self.train_dataloader, desc=">> train", mininterval=5):
  File "/usr/local/lib/python3.9/site-packages/tqdm/std.py", line 1181, in __iter__
    for obj in iterable:
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 630, in __next__
    data = self._next_data()
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1344, in _next_data
    return self._process_data(data)
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/dataloader.py", line 1370, in _process_data
    data.reraise()
  File "/usr/local/lib/python3.9/site-packages/torch/_utils.py", line 706, in reraise
    raise exception
RuntimeError: Caught RuntimeError in DataLoader worker process 0.
Original Traceback (most recent call last):
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/worker.py", line 309, in _worker_loop
    data = fetcher.fetch(index)  # type: ignore[possibly-undefined]
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/fetch.py", line 55, in fetch
    return self.collate_fn(data)
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 317, in default_collate
    return collate(batch, collate_fn_map=default_collate_fn_map)
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 174, in collate
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 174, in <listcomp>
    return [collate(samples, collate_fn_map=collate_fn_map) for samples in transposed]  # Backwards compatibility.
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 146, in collate
    return collate_fn_map[collate_type](batch, collate_fn_map=collate_fn_map)
  File "/usr/local/lib/python3.9/site-packages/torch/utils/data/_utils/collate.py", line 235, in collate_int_fn
    return torch.tensor(batch)
  File "/usr/local/lib/python3.9/site-packages/torch/utils/_device.py", line 79, in __torch_function__
    return func(*args, **kwargs)
  File "/usr/local/lib/python3.9/site-packages/torch/cuda/__init__.py", line 300, in _lazy_init
    raise RuntimeError(
RuntimeError: Cannot re-initialize CUDA in forked subprocess. To use CUDA with multiprocessing, you must use the 'spawn' start method

Why is it so? Why it does not allow me to return extra boolean value from __get_item__?

PS:

Above is main question. However, I noticed some weird observations: above code (with or without , self.t[i] commented) starts working if I replace DalaLoader‘s generator’s device from cuda to cpu ! That is, if I replace generator=torch.Generator(device='cuda') with generator=torch.Generator(device='cpu'), it outputs:

x: torch.Size([10, 10, 10]), y: torch.Size([10, 3])

And if I do the same in my original code, it gives me following error:

RuntimeError: Expected a 'cuda' device type for generator but found 'cpu'

on line for data in dl:.

Update

It started working as soon as I changed type of self.t from python dict to torch tensor of type bool and moved it to cpu:

self.t = torch.tensor([random.choice([True, False]) for _ in range(num_samples)], dtype=torch.bool).to('cpu')

Please explain why.