scanf() will only return input when encountering 'n' or EOF (on Linux can generate an EOF on stdin by by pressing ctrl-d once or twice).

The format string %x will skip any leading white space and accept any number of digits so I don’t think you can get away with using it (you will need to use %n to figure out how many bytes where consumed; it is unclear if your program should consume invalid input).

If input must be strictly validated with scanf() then you you have to read a character at a time with either %c or %1[. They preserve white and can report incomplete input:

#include <stdio.h>

#define MAX_CHAR_LENGTH 5

int main() {
    printf("enter hex values: " ); 
    char s[3*MAX_CHAR_LENGTH+1] = {0};
    for(int i = 0; i < MAX_CHAR_LENGTH; i++) {
        int rv = scanf("%1[0-9a-fA-F]%1[0-9a-fA-F]%1[ n]", s+3*i, s+3*i+1, s+3*i+2);
        if(!i && !rv) { // "n"
            int rv2 = scanf("%1[n]", s);
            if(!rv2) goto fail;
            break; 
        }
        if(rv != 3) goto fail;
        if(s[3*i+2] == 'n') break;
    }
    printf(""%s"n", s);
    return 0;
fail:
    printf("invalid input "%s"n", s);
    return 1;
}

And example runs:

enter hex values: 
"
"

enter hex values:  // <space> <enter>
invalid input ""

enter hex values: zz 
invalid input ""

enter hex values: aa // aa <space> <enter>
invalid input "aa "

enter hex values: aa
"aa
"

enter hex values: aa aa aa aa aa aa
"aa aa aa aa aa "

A better approach is separate I/O and parsing. Use fgets() or getline() to read a line. Then parse upto those 5 hex digits then print the the valid digits xor tell user about any and all errors found.

You can read individual characters with %c or %[ but be aware that both the terminal line discipline and the stdin buffering scheme will prevent any input from reaching the program until a newline is entered when the user hits the Enter key.

Here is a modified version:

#include <stdio.h>

#define MAX_CHAR_LENGTH 5

int main(void) {
    unsigned int curr_hex;
    char c;
    
    printf("enter hex values: " ); // prompt user to enter hex values
    
    for (int i = 0; i < MAX_CHAR_LENGTH; i++) { 
        if (scanf("%x%c", &curr_hex, &c) != 2) {
            printf("invalid inputn");
            return 1;
        }
        printf("Read hex: 0x%02Xn", curr_hex);
        if (c == 'n')
            break;
        if (c != ' ') {
            printf("invalid separator: %cn", c);
            return 1;
        }
    }
    return 0;
}

The above program will accept hex values with more than 2 digits, including values above 255. You can avoid this with this alternate approach:

#include <stdio.h>

#define MAX_CHAR_LENGTH 5

int main(void) {
    unsigned int curr_hex;
    char sep[2];
    
    printf("enter hex values: " ); // prompt user to enter hex values
    
    for (int i = 0; i < MAX_CHAR_LENGTH; i++) { 
        // read at most 2 hex digits followed by either a space or a newline
        if (scanf("%2x%1[ n]", &curr_hex, sep) != 2) {
            printf("invalid inputn");
            return 1;
        }
        printf("Read hex: 0x%02Xn", curr_hex);
        if (*sep == 'n')
            break;
    }
    return 0;
}

The above program will still accept hex values with a single digit. To ensure exactly 2 digits were entered, you can use %n to get the number of characters read and check the length:

#include <stdio.h>

#define MAX_CHAR_LENGTH 5

int main(void) {
    unsigned int curr_hex;
    int n;
    char sep[2];
    
    printf("enter hex values: " ); // prompt user to enter hex values
    
    for (int i = 0; i < MAX_CHAR_LENGTH; i++) { 
        // read at most 2 hex digits followed by either a space or a newline
        if (scanf("%2x%n%1[ n]", &curr_hex, &n, sep) != 2 || n != 2) {
            printf("invalid inputn");
            return 1;
        }
        printf("Read hex: 0x%02Xn", curr_hex);
        if (*sep == 'n')
            break;
    }
    return 0;
}

The problem with scanf is error recovery: it is cumbersome to handle invalid input gracefully. A better solution is to read input one line at a time with fgets() and parse it with sscanf or other functions.

Err… reading a line oriented input with scanf is masochism (or sadism from your teacher if this is an assignment). The blessed way would be to first read the line with fgets and then scan it with strtol.

Something like:

char line[32];
if (NULL == fgets(line, sizeof(line), stdin)) {       // read a full line
    ...                                 // handle the read error
}
if (strlen(line) == sizeof(line) -1) {  // a too long line?
    ...                                 // handle the error condition
}
char *start = line;                     // search for a number starting at start
char *end;                              // define an end pointer
for (;;) {
    int val = strtol(start, &end, 16);  // read an hexa value
    if (end == start) {                 // nothing more to read?
        break;                          // exit the loop
    }
    printf("Read hex: 0x%02Xn", val);
    start = end;                        // continue past the got value
}
printf("n");

If you are required to use scanf, you will have to control whether the character following the hexa value is a 'n'.

Something like:

for(;;) {
    int val;
    char c;
    if (1 != scanf("%x", &val)) {             // try to read an hex value
        ...                                   // handle the read error
        break;
    }
    if (1 != scanf("%c", &c)) {
        ...                                   // handle the read error
        break;
    }
    if (c == 'n') break;                     // got the newline!
    printf("Read hex: 0x%02Xn", val);
}

Yet this is not immune to blank characters before the end of line, so this is a very fragile code.

The robust way would be to use fgetc and ungetc:

for(;;) {
    int val;
    int c;
    if (1 != scanf("%x", &val)) {             // try to read an hex value
        ...                                   // handle the read error
        break;
    }
    do {
        c = fgetc(stdin);                     // BEWARE: getc returns an int!
        if (c == EOF) {                       // because EOF is an int constant
            ...                               // handle the read error
            break;
    } while (isblank(c));                     // will skip any space or tab
    if ((c == 'n') || (c == EOF) break;      // got the newline or an error
    ungetc(c, stdin);                         // else put c back into the stream
    printf("Read hex: 0x%02Xn", val);
}
printf("n");

But beware: none of the above code has been tested: typos can be expected…