I have lists of data I’m trying to organize where the list with the least variation/most precision/closest to the average and most amount of data is most highly rated.
Some of the lists are like 2000 numbers long and others only have 1 number. Obviously the arithmetic mean of a list with one number will be exactly the value of the one number. Any individual value from a list of 2000 numbers might not be as close to the arithmetic mean. When sorting these, all the smaller lists will be higher rated.
It seems really easy, but I can’t come up with a way to order the list more proportionally.
Examples of lists:
List 1
12, 20, 15, 17, 9, 19
Average of list:
12+20+15+17+9+19/6=15
Score: (15-12) + (20-15) + (15-15) + (17-15) + (15-9) + (19-15)= 20
List 2
30
Average of list: 30/1=30
Score: (30-30) = 0
List 3
10, 10, 10, 10, 10, 10, 10, 10, 10, 9
Score: (10-10)…etc = 1
Ranking
-
List 2: Score=0
-
List 3: Score=1
-
List 1: Score=20
This is what i have right now. The problem with this is that list 2 will be at the top because its small. I’d like list 3 to be at the top because the score is small and it has a lot of members. I’m not sure how to add weight to size though. List 1 is an example of a list with a lot of members, none of which are really close to the average. Thats two bad things so it should be at the bottom.
As for outliers, they arent treated any differently. If one super high number gives a list a really bad score thats okay.
5
A common approach to measuring what you’re describing is to use the standard deviation of the set of numbers. Others prefer to use the variance instead. Fortunately, calculating both is fairly simple once you understand the algorithm.
For the variance and standard deviation, follow these steps:
- Find the average of your set of numbers:
float avgValue;
float totalValue = 0;
for( var i in setOfNumbers ){
totalValue += i;
}
avgValue = totalValue / setOfNumbers.Count(); // or use Length() instead ...
- Sum the squares of the difference from the average value.
- Find the variance by find the average from step 2.
//Find sum of squares of the difference from average
float sumOfSquares = 0;
for( var i in setOfNumbers ){
float diffFromAvg = i - avgValue;
sumOfSquares += diffFromAvg^2; // use your language's square function
}
//find variance
float variance = 0;
variance = sumOfSquares / setOfNumbers.Count(); // or use Length() instead ...
- If you want the standard deviation, that’s the square root of the variance.
This link and this link also offer up some additional explanation for calculating either the variance or the standard deviation.
You said that the longest list you have is 2000 numbers, so you’re not likely to need alternative algorithms. That said there are a number of ways to calculate variance and standard deviation in one pass.
But as with all things in life, there are disputes over the best approach, so do your additional research if you get beyond the basic algorithms.
1