Caution: A very big block of text

Overview

The question of How graphics works? has plagued me ever since I wanted to figure out How graphics library works?. Some might give me the answer and I thank you but the problem I am going to describe will make you rethink because it has caused me a lot of pain.

The Entire Question

It should be pretty obvious that each GPU comes with firmware which is a sort of the Operating System of the GPU. The GPU and the CPU and ultimately the OS, communicate through a common piece of memory. My guess of how this communication plays out is:

• Either the value for each pixel is written to the memory and the GPU displays it.
• Or certain values are placed into the memory which each represents some kind of function in the GPU.
  These are all speculations from my part while the working principle could different entirely.
  We now have the OS. The OS, obviously, puts a device driver that does the task of writing to the memory and communicate with the GPU. That should abstract a lot of the underlying complexity. Let us assume that our OS is Ubuntu running the Gnome window manager. The OS itself doesn’t really need the graphics other than for a simple terminal but for users a GUI is indispensable.

I don’t know where the window manager is in the abstraction soil but I believe it communicates directly with the GPU device driver so it is pretty low-level. Now, say you wrote a game engine and are using the OpenGL library.

How does that library work?

The question may sound simple but the answer? I haven’t found it yet. Websites say that it is just a specification defined by the GPU manufacturer but that doesn’t answer anything.

You obviously need a window for your game engine so you call the functions from the OpenGL library to create that window BUT how is the window created? Does your game engine communicate with the Gnome Window manager or the driver? But based on the details from the website Your program directly communicates with the GPU. But other than the first possibility, other answers doesn’t make sense.

Breakdown of the answers: If the game engine communicated with the driver than how does the driver know what kind of window to create because the window that pops up does look the same as when you open Firefox or File Manager so it can’t be that the driver knows what kind of window does Gnome make. You also notice that your window is movable, re-sizable, tileable, which the primitive driver doesn’t know how to do. So it can’t be that your program communicates with the driver.

The last answer is also not possible as the OS wouldn’t allow it. Then the first answer is correct?

Contradiction

According to ChatGPT, the window manager uses the same OpenGL and Vulcan libraries. The way this works is that the window manager will define the functions that these libraries expect to exist and then use them afterwards but then what about your game engine? You didn’t define the functions so is the library you are using communicating with the window manager? Then there are two kinds of the same library?

See the problems?

How rendering of text is done?

Everyone has certainly changed the system fonts at some point but how does that work as well? Just look at a bootloader which uses the BIOS to print something. The bootloader gives a number, the ASCII value, to the BIOS interrupt handler but the text gets printed. That tells you that the BIOS must have told the GPU or whatever how to display that specific letter in that format pixel by pixel because the component that displays the text certainly doesn’t know how to print every kind of letter that exists. I would believe if you said just the ASCII representations but the Chinese, Japanese, Hindi? That’s not possible. Thus, the only probable answer is that you have to tell, whatever the component is, how to display what you want to display pixel by pixel.

That also means that the GPU needs to know how to display which is possible; Just pass a structure for each possible letter and the GPU should handle the rest.

What I learnt

Don’t ever ask LLM for doubts.