Here are some cases and examples of ways to describe and classify them so they might be analyzed as a whole.

1. A < B < C – This is a distinct ordered set
2. A = B < C – This is an indistinct ordered set
3. A > B < C – This is an unordered set
4. A < B = C – This is an indistinct ordered set
5. A = B = C – This is an indistinct ordered set
6. A > B = C – This is an indistinct ordered set
7. A < B > C – This is an unordered set
8. A = B > C – This is an indistinct ordered set
9. A > B > C – This is a distinct ordered set

Grab whatever other terminological definitions you can think of for sets and lay them over those 9 cases and you may find more identifiable groupings than what I found using just 2 filters (distinctness and orderliness).

I’m sure the math lexicon is brimming with terms you could put on these for describing sets.

From Set theory on Wikipedia (emphasis mine):

Equivalence and order relations are ubiquitous in mathematics, and the theory of mathematical relations can be described in set theory.

Considering the area you’re looking at, like I said there’s a lot in math for you to find here. Here’s a few things I would further suggest you read up on to give yourself some terminology and prior precedent to work from:

Transitive relationship:

For example, “is greater than,” “is at least as great as,” and “is equal to” (equality) are transitive relations:

whenever A > B and B > C, then also A > C

whenever A ≥ B and B ≥ C, then also A ≥ C

whenever A = B and B = C, then also A = C.

Commutative Property (in relation to your greaterthan/lessthan/equality operations, commutativity can help you derive transitivity):

The term “commutative” is used in several related senses.[7][8]

1. A binary operation * on a set S is called commutative if:

   x * y = y * xqquadmbox{for all }x,yin S

Associative Property (in relation to your operations, associativity can help you derive transitivity as well):

Within an expression containing two or more occurrences in a row of the same associative operator, the order in which the operations are performed does not matter as long as the sequence of the operands is not changed. That is, rearranging the parentheses in such an expression will not change its value. Consider, for instance, the following equations:

(5 + 2) + 1 = 5 + (2 + 1)=8

5 x (5 x 3) = (5 x 5) x 3=75

You have exactly 13 relationships.

6 of them are in the form A>B>C, for various orderings of A,B,C. Let us name them A_B_C and similar.

6 of them are in the form A>B=C or A=B>C, for various orderings of A,B,C. Let us name them A_BC and AB_C and similar.

The final 1 is A=B=C. Let us name that ABC.

These 13 names form an enumeration that identifies all the possible relationships. You can provide synonyms if you believe your users will be more comfortable with LT, LE, GE etc but I would not recommend it. Let them find the right member to describe what they need.

Use that enumeration as an index into a data structure to deliver the various flags you need to drive your code.

There are three possible relations between A and B:

1. A = B
2. A > B
3. A < B

Similarly, there are three possible relations between A and C, and three relations between B and C.

I feel that rather than storing one value representing the relationship between three values (an enum with 21 values after throwing out the degenerate cases), it would be better to store three relationships between each pair of values, each of which is an enum with three values.

public enum XCompY
{
    LessThan = -1,
    Equals = 0,
    GreaterThan = 1
}

public class MyData
{
    private XCompY aToB;
    private XCompY aToC;
    private XCompY bToC;

    ...

    public bool IsValid(MyObject a, MyObject b, MyObject c)
    {
        int aCompB = a.CompareTo(b); // MyObject.CompareTo returns exactly -1, 0, or 1
        int aCompC = a.CompareTo(c);
        int bCompC = b.CompareTo(c);

        return (aCompB == (int)aToB) && (aCompC == (int)aToC) && (bCompC == (int)bToC);
    }
}

We now have something easy to read, we don’t have a giant switch block, and we’ve added the ability to validate the relationship between fewer than 3 objects (or more than 3). (Yes, this approach is very close to what your initial idea was.)

It seems to me that you could build a look up table to dispatch handling.

You have three relationships which in turn have three values.

We can think of this as three indices:

1. Relation b/w A and B
2. Relation b/w A and C

3. Relation b/w B and C

   enum Relation { less, equal, greater };

   public delegate void Handle(int a, int b, int c);
   
   class Handlers
   {
       public void handleLessLessLess(int a, int b, int c) { }
       public void handleLessLessEqual(int a, int b, int c) { }
       public void handleLessLessMore(int a, int b, int c) { }
   
   };
   public class Dispatcher
   {
       public Dispatcher()
       {
           table[Relation.less][Relation.less][Relation.less] = new Handle(h.handleLessLessLess);
           table[Relation.less][Relation.less][Relation.equal] = new Handle(h.handleLessLessEqual);
           table[Relation.less][Relation.less][Relation.greater] = new Handle(h.handleLessLessEqual);
           table[Relation.less][Relation.greater][Relation.less] = null; // cannot have a < b, a > c, b < c
       }
   
   public void handle(Relation ab, Relation ac, Relation bc, int a, int b, int c)
   {
       Handle h = table[ab][ac][bc];
       if (h != null)
           h(a, b, c);
       else
           // exception of what have you.
           ;
   }
   
   Handlers h = new Handlers();
   Dictionary<Relation, Dictionary<Relation, Dictionary<Relation, Handle>>> table =
       new Dictionary<Relation, Dictionary<Relation, Dictionary<Relation, Handle>>>();
   

   };