I am trying to implement a Boundary fill algorithm, for which I am using glReadPixels() function.

But when I use glReadPixels(x, y, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &pixel); , it always returns the background color, even when I have previously changed the color of the pixel.

So to try to figure out what may be wrong, I wrote this code which does not seem to work as intended:

typedef struct pixel {
    GLubyte r,g,b;
} pixel;

pixel p;

void ctest(){
    glReadPixels(400, 400, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &p);
    while (p.r!=255 || p.g!=0 || p.b!=0){
        glColor3ub(255, 0, 0);
        glBegin(GL_POINTS);
        glVertex2i(400, 400);
        glEnd();
        glFlush();
        glReadPixels(400, 400, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &p);

        cout<<(int)p.r<<' '<<(int)p.g<<' '<<(int)p.b<<endl;
    }
}

This code simply goes into infinite loop, continuously outputting ‘0 0 0’.

But when I just add a glPointSize(2);, the output is as I would normally expect: The loop exits after excuting once.

void ctest(){
    glReadPixels(400, 400, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &p);
    while (p.r!=255 || p.g!=0 || p.b!=0){
        glColor3ub(255, 0, 0);
        glPointSize(2);
        glBegin(GL_POINTS);
        glVertex2i(400, 400);
        glEnd();
        glFlush();
        glReadPixels(400, 400, 1, 1, GL_RGB, GL_UNSIGNED_BYTE, &p);

        cout<<(int)p.r<<' '<<(int)p.g<<' '<<(int)p.b<<endl;
    }
}

Output: ‘255 0 0’, and then the program finishes.

(Visually speaking, the pixel is colored red in both the cases, in different sizes)

The issue that arises out of this for my boundary fill algorithm is:

1. When using glLineWidth(1) for boundary polygon, glPointSize(1) for point size to fill the color, and glReadPixel(..., 1, 1, ...) to read the pixel, my algorithm simply goes into infinite loop at the last pixel that needs to be colored, suggesting that the second last pixel reads as ‘uncolored’, even though it is colored, as is coded and is visually apparent to me.

2. When using glLineWidth(2) for boundary polygon, glPointSize(2) for point size to fill the color, and glReadPixel(..., 2, 2, ...), my program terminates at the boundary pixel, throwing

*** stack smashing detected ***: terminated
Aborted (core dumped)

3. When using glLineWidth(2) for boundary polygon, glPointSize(2) for point size to fill the color, and glReadPixel(..., 1, 1, ...) to read the pixel, the output colors my screen in a manner that only the North-to-East (0° to 90°) from the seed pixel to boundary is colored, the rest 270° directions’ area remains uncolored.

There are certainly more combinations possible for line widths, point sizes, and pixel reading sizes, but the output is either one among these, or is what I would expect, so they might not be of interest here.

The code of my boundary fill algorithm is:

void boundary_fill(int x,int y){
    pixel c;
    glReadPixels(x,y,1,1,GL_RGB,GL_UNSIGNED_BYTE,&c);
    if((c.r!=b_color.r || c.g!=b_color.g || c.b!=b_color.b )&&(c.r!=f_color.r || c.g!=f_color.g || c.b!=f_color.b )){
        glColor3ub(f_color.r,f_color.g,f_color.b);//set fill color for pixel
        glBegin(GL_POINTS);
        glVertex2d(x,y);
        glEnd();
        glFlush();

        boundary_fill(x+1,y);//right pixel
        boundary_fill(x-1,y);//left pixel
        boundary_fill(x,y+1);//upper pixel
        boundary_fill(x,y-1);//lower pixel
    }
}

This is the part of my main function where I have initialized stuff:

glutInitDisplayMode(GLUT_SINGLE | GLUT_RGB);
glutInitWindowPosition(0,0);
glutInitWindowSize(500,500);
glutCreateWindow("Pattern");
glClearColor(1.0,1.0,1.0,0.0);
glClear(GL_COLOR_BUFFER_BIT);
glColor3f(0.0,0.0,0.0);
gluOrtho2D(0,500,0,500);
glFlush();

I am currently on Ubuntu 22.04.

What exactly is going on here?