I am trying to create a version of Array.groupBy() that handles more cases.
Specifically, I want to be able to store a single object for a given key (instead of an array of objects with Array.groupBy()) or pluck a single property from each object.

The execution of the function works well, however the types are not always inferred properly.

The type definition is the following:

declare global {
  interface Array<T> {
    /**
     * Groups the elements of an array by a specified field name.
     * 
     * @param fieldName - The field name to group by.
     * @param valueFieldName - Optional. The field name to use as the value in the grouped result.
     *                         If not provided, the entire object will be used as the value.
     *                         If set to '[]', an array of objects with the same field name will be used as the value.
     * @returns An object with the grouped elements.
     */
    groupBy<Key extends keyof T>(
      fieldName: Key,
      valueFieldName?: T[Key]|'[]',
    ): { [key: string]: typeof valueFieldName extends '[]' ? T[] : typeof valueFieldName extends string ? T[Key] : T };
  }
}

And the following calls, despite sending back correct results, are not properly typed:

// returns { John: [{ id: 1, name: 'John' }, { id: 3, name: 'John' }], Jane: [{ id: 2, name: 'Jane' }] }
// FIXME when hovering over second the inferred type is wrong:
// { [key: string]: { id: number; name: string } }
// instead of
// { [key: string]: { id: number; name: string }[] }
const second = [{ id: 1, name: 'John' }, { id: 2, name: 'Jane' }, { id: 3, name: 'John' }]
      .groupBy('name', '[]');
console.log(second)
 
 // returns { John: 28, Jane: 30 }
// FIXME when hovering over second the inferred type is wrong:
// { [key: string]: { id: number; name: string; age: number } }
// instead of
// { [key: string]: number }
const third = [{ id: 1, name: 'John', age: 25 }, { id: 2, name: 'Jane', age: 30 }, { id: 3, name: 'John', age: 28 }]
      .groupBy('name', 'age');
console.log(third)

What more can I tell Typescript so that the types are inferred properly? I feel like the parts with typeof valueFieldName extends are not optimal, but they should still work in my opinion.
Or could it be that Typescript can’t infer these types precisely enough? If so, what workarounds do I have?

Playground link