I’m doing a hyperparameter tuning using sklearn’s GridSearchCV.

    if cfg.tuning is True:
        print("extractingY...n")
        y = extract_Y(test_dataloader)
        print("finished extractionn")
        ht = hyp_tuning(model=model, optimizer=optimizer)
        param_grid = {
            'lr': [1e-2, 1.5e-2]
        }
        search = GridSearchCV(ht, param_grid, cv=5, scoring='precision')
        #x is a "dummy" matrix because i use dataloader in fit and predict
        x = [0 for _ in range(len(y))]
        print(len(y))
        result = search.fit(x, y)
        print('Best: %f using %s' % (result.best_score_, result.best_params_))

However, the scoring always fails because of this error:

C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmodel_selection_validation.py:982: UserWarning: Scoring failed. The score on this train-test partition for these parameters will be set to nan. Details:
Traceback (most recent call last):
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmodel_selection_validation.py", line 971, in _score
    scores = scorer(estimator, X_test, y_test, **score_params)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_scorer.py", line 279, in __call__
    return self._score(partial(_cached_call, None), estimator, X, y_true, **_kwargs)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_scorer.py", line 376, in _score
    return self._sign * self._score_func(y_true, y_pred, **scoring_kwargs)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnutils_param_validation.py", line 213, in wrapper
    return func(*args, **kwargs)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_classification.py", line 2190, in precision_score
    p, _, _, _ = precision_recall_fscore_support(
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnutils_param_validation.py", line 186, in wrapper
    return func(*args, **kwargs)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_classification.py", line 1775, in precision_recall_fscore_support
    labels = _check_set_wise_labels(y_true, y_pred, average, labels, pos_label)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_classification.py", line 1547, in _check_set_wise_labels
    y_type, y_true, y_pred = _check_targets(y_true, y_pred)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnmetrics_classification.py", line 99, in _check_targets
    check_consistent_length(y_true, y_pred)
  File "C:UsersParluAppDataLocalProgramsPythonPython39libsite-packagessklearnutilsvalidation.py", line 460, in check_consistent_length
    raise ValueError(
ValueError: Found input variables with inconsistent numbers of samples: [2403, 12014]

I tried to print the length of y, but the terminal says that len(y) is 12014, not 2403 as the error says. I really have no idea of what the problem is.

I’m also attaching the function extract_Y

def extract_Y(dataloader):
    all_labels = []
    for data, lengths, targets in tqdm(dataloader):
        targets = targets.view(-1)#.cuda()
        mask = (targets != -1)
        all_labels.extend(targets[mask].cpu().numpy())
    return all_labels