On this page about os-prober I could find the following sentence: The program Josh Kwan and I developed for d-i to do this is quite flexible, new OSes and linux distros can be added by just dropping in a test file to check for them. It's fairly good at probing all the info needed to boot some installation of linux (the root, and /boot partitions, a kernel, an initrd, and kernel parameters). Since every version of linux does things slightly differently, this is no mean feat, techniques used include parsing /etc/fstab, rummaging around in /boot, parsing grub menu.lst files, etc.
I think that these applications first check information that is connected with the disk layout (for example entries from a MBR or EFI system partition or another example: it might recognise a NTFS-formatted partition as part of Windows) and then they mount relevant partitions and try to check for some other characteristics (for example CentOS includes a file called /etc/centos-release).