I am trying to remove the duplicates from a list of tuples that contains matrices. The tuples are composed of two items: a number and a corresponding matrix. To be considered a ‘duplicate’, both the number and the matrix must be identical to another tuple in the list.

For instance, a shortened version of my initial list is as follows (notice the last two tuples are identical, and thus the goal is to remove one of them):

[  ( 5.42,
      c1     c2    c3     c4     
r1    1      1      1      1    
r2    1      0      0      0   
r3    0      1      0      1     
r4    0      0      0      0 ),

  ( 4.31,
      c1    c2     c3     c4     
r1    0     1      0      1    
r2    0     1      1      1    
r3    0     0      0      0   
r4    1     0      0      1 ),

  ( 5.42,
      c1     c2    c3    c4     
r1    0      0      0     0    
r2    0      0      1     0     
r3    0      0      0     1   
r4    1      1      1     1 ),

  ( 5.42,
      c1    c2    c3    c4     
r1    0     0      0     0    
r2    0     0      1     0     
r3    0     0      0     1   
r4    1     1      1     1 )  ]

The desired output will be a list of 3 tuples (instead of the initial 4), all tuples now being unique

[  ( 5.42,
      c1    c2    c3    c4     
r1    1     1     1     1    
r2    1     0     0     0   
r3    0     1     0     1     
r4    0     0     0     0),

  ( 4.31,
      c1    c2    c3    c4     
r1    0     1     0     1    
r2    0     1     1     1    
r3    0     0     0     0   
r4    1     0     0     1 ),

  ( 5.42,
      c1    c2    c3    c4     
r1    0     0     0     0    
r2    0     0     1     0     
r3    0     0     0     1   
r4    1     1     1     1 )  ]

All of my attempts result in either one of two errors:

ValueError: The truth value of the DataFrame is ambiguous. Use a.empty() , a.bool(), a.item() or a.all()

-or-

TypeError: DataFrame objects are mutable, thus cannot be hashed

When I check the ‘type’ of my incoming tuple-list, it returns <Class ‘list’>. So, I am confused that my error messages reference DataFrames.

Just a sample of my many attempts include:

(Note, the incoming list is ‘tup_list’ below)

attempt example 1:

from itertools import groupby

deduped = [next (g) for k, g in groupby (sorted (tup_list), key = lambda tup: tup [:2] ) ]

attempt example 2:

new_tup_list = [ tuple (item) for item in tup_list ]

deduped = list (set (new_tup_list) )

attempt example 3:

deduped = list (set (map(tuple, tup_list) ) )

 attempt example 4:

from collections import OrderedDict

`for t in tup_list:

d.setdefault ( t[1], t )

deduped = list( d.values() )

attempt example 5:

deduped = set()

temp = [deduped.add((a,b))  for (a,b) in tup_list if (a,b) not in deduped]

If you have any advice as to how to successfully remove duplicates from a list of tuples that involves matrices, I would be most appreciative – Thank you.