I'd like to use /dev/fb0 or whichever other files are provided by the linux.
This in a system specific way as low as possible how to read/write/interpret those device files for GUI frame-buffer non-console based.
I'd like to do it for low-level fun, just like accessing keyboard events.
You open a file descriptor for the framebuffer device and use ioctl(), with FBIOGET_FSCREENINFO and FBIOGET_VSCREENINFO. From the returned structures, you can straightforwardly determine the X resolution, Y resolution, stride, and bits per pixel.
If the framebuffer is shared with the kernel's built-in terminal emulator, and used for displaying kernel virtual terminals, you need to negotiate access with that subsystem, telling it to not plot its characters into the framebuffer whilst your program is doing so and negotiating KVT switching with other processes.
Plotting pixels is then a matter of performing calculations with the pixel address, desired colours, and the aforementioned values; to obtain a byte offset and the byte value(s) to write. This can be done with the pwrite() system call; but is more efficiently done by memory mapping the framebuffer device into the process' address space and just poking the memory-mapped area.
There are entire books about how one builds on top of this.
Things become a lot more complex when instead of using the pre-supplied framebuffer, in the graphics mode that is already in use, you use ioctl() mode setting to create one or more frambuffers dynamically.
Sure, there are lots of documentation and articles on how to read/write/interpret linux devices. For the framebuffer e.g you can write to it even from your console using the cat command from a tty console to test the waters.
# cat /dev/urandom >/dev/fb0
Read this e.g to understand how it works and then use C/C++ language or whatever to make more sophisticated things.
Not certain if you can get mouse / keyboard input from fb0. It is great fun though and you'll learn a great deal of how things work in the Unix world.
What you want to begin with is to use the open system call to open the file descriptor and from there you will use ioctl to set the parameters that you need. Then you'll basically just read and write to the fb as a regular file.
You might want to look into mmaping the file though then you can just access it as an array!
So what ioctl can you will probably find in man pages and by reading files in /usr/include
For reading keyboard and mouse input you'd like to use input devices found in the dev filessystem.
All of this might seem mighty strange but the input system is quite powerful.
Happy hacking!
