I encountered some confusion when using ‘sklearn. metrics. ndcg_score’ to calculate NDCG. Specifically, suppose I have the following situations:

-Related documents: a, b. C (Their correlation score is 1)
-Search result: a, d, e, f, g, h, i, j, k

I want to know how to calculate NDCG and how to use the ‘ndcg_score’ function of ‘sklearn’ to calculate this value. Meanwhile, I would like to know how to handle my ‘predicted_scores’ if it contains positive and negative scores.

import pytrec_eval
from sklearn.metrics import ndcg_score
import numpy as np

# Create simulated data
qrels_dict = {
    'q1': {'6289186#3': 1, '77756#0': 1, '3155460#0': 1, 
           '2866270#0': 0, '57869#0': 0, '2852023#0': 0,
           '1515255#0': 0, '3639#12': 0, '77756#15': 0, '681279#0': 0}

}

results_dict = {
    'q1': {'6289186#3': 0.4140625, '77756#18': 0.400390625, '2866270#0': -0.03076171875, '3559599#0': -0.197265625, 
           '254448#0': -0.470703125, '3967750#0': -0.486328125, '6188311#0': -0.5234375, '6434653#30': -0.6171875, 
           '7008190#0': -0.6953125, '7840592#0': -0.7109375, }

}


# Calculate metrics using pytrec_eval
evaluator = pytrec_eval.RelevanceEvaluator(qrels_dict, {'ndcg_cut.10'})
pytrec_results = evaluator.evaluate(results_dict)

# Prepare data for sklearn
specific_qid = 'q1'
specific_relevances = [
    qrels_dict[specific_qid].get(docid, 0)
    for docid in results_dict[specific_qid].keys()
]
print(specific_relevances)
specific_scores = [
    results_dict[specific_qid][docid] for docid in results_dict[specific_qid].keys()
]
print(specific_scores)

sklearn_specific_ndcg_10 = ndcg_score(
    [specific_relevances], [specific_scores], k=10, ignore_ties=True
)
pytrec_specific_ndcg_10 = pytrec_results[specific_qid]['ndcg_cut_10']

# Print results
print(f"qrels_dict['{specific_qid}']:", qrels_dict[specific_qid])
print(f"results_dict['{specific_qid}']:", results_dict[specific_qid])
print(
    f"Specific QID: {specific_qid} | pytrec_eval NDCG@10: {pytrec_specific_ndcg_10} | sklearn NDCG@10: {sklearn_specific_ndcg_10}"
)

Specific QID: q1 | pytrec_eval NDCG@10: 0.46927872602275644 | sklearn NDCG@10: 1.0

Actually i want to get 0.4692