We are running Red Hat Enterprise Linux version 7, and all our machines are virtual machines.
Our memory resources are limited and physical RAM costs money so we are thinking of increasing swap instead of adding memory.
Is this a good idea?
Secondly, when / from which point does the OS start using swap?
No, it’s a bad idea.
You shouldn’t think of swap as a mechanism by which you can expand memory; it’s a storage area for parts of memory which don’t have to remain in physical memory, and whose contents don’t exist anywhere else. See Why does Linux need swap space in a VM? for details.
If the processes running inside your VMs are running out of memory, you need to determine what their real working set is, both in nominal operation and in the worst case. Then, assuming you can’t reduce their memory usage, you need to configure their memory setups to suit: RAM allocation, swap, and kernel configuration (swappiness etc.). The RAM allocation will have a direct impact on the number of VMs you can run per host, and that should really be your main adjustment variable if you can’t add more memory to your hosts. (That doesn’t help with the cost aspect of course...) Depending on what you need VMs for, another strategy could be to use containers instead since that will allow you to reduce the overhead.
Operating systems typically start using swap when they need to allocate memory and they’ve run out of available physical memory, and the least used memory pages currently in physical memory don’t have what’s called a backing store (or rather, their backing store is swap). When a program needs more memory, the kernel will first look for some free memory; then it will look through a hierarchical list of things it can get rid of — cache, buffers, mapped executables, etc. Note that swap can be used even in the absence of “visible” memory pressure: there are always pieces of data stored in memory which aren’t actually used, and are better stored in swap.
If you have memory shortage and you cannot afford to upgrade the RAM better you find ways to free up some memory on your server. Generally, there exist many processes/services that you may not actually need. For example on a server system "cups" printing service is not required. So identify all such services and disable them.
There is no harm in adding additional "swap", but keep in mind that it is not a replacement for RAM. Your applications experience serious performance degrade if they start using swap.
To add to the excellent Stephen Kitt answer: planned swap usage in physical machines can be a bad idea, in a virtual environment is a very bad idea due to all the I/O adding up on a (few) common central storage appliances.
Basically, to scale up the performance of your infra-structure, you want to devise strategies to minimize your I/O, not to make it worse.
