There are plenty of open source (and proprietary) monitoring tools designed to solve this problem. They rely on tools within Linux, and they in turn rely on system calls within the kernel.
Some tools focus on data gathering and monitoring, while others focus on alerting, which you pick depends on your primary need.
The most well known example of an alerting and monitoring tool would be Nagios. Other tools, more focussed on data gathering and graphing, with some alerting built in would be Cacti and Munin. If you have large clusters with lots of machines, then Ganglia might be your best bet.
These tools are often called Network Monitoring Systems, and Wikipedia has an extensive list.
I recommend you don't re-invent the wheel and look for / use a tool like this.
Depending on which Linux distribution you're using, one or more of these tools many already be available in the distribution repository, with default configurations that support the environment you have.