Find process source which executing python scripts

Question

From last month some process is executing python scripts on my server with centos 6 with LAMP stack which are nothing but spam bots and eating server resources. Please check image below,

How do I find what exactly executing these python scripts?

--UPDATE-- As these commands are executing as root I scanned the system with rootkit hunter(rkhunter) and found that my ssh is compromised (Possible rootkit: Trojaned SSH daemon). So I did following,

#removed ssh
chattr -aui /usr/sbin/sshd
rm -f /usr/sbin/sshd
chattr -aui /usr/bin/ssh
rm -f /usr.bin/ssh

Then,

yum erase openssh-server
yum erase openssh-clients
yum install openssh-server
yum install openssh-clients

Now I ran rkhunter again and all are good.

Thank you

Answer 1

Instead of the htop picture, please post the text output of pstree -p and/or ps axfuw in your question if you need help finding the source process. Text output is preferable to pictures.

These processes are running as root user, and they appear to be downloading and executing perl scripts through the python script x.py. There is also a bash script go running as root.

The following is not a specific answer to your question, but hopefully some helpful information about hacks with this type of privilege.

You could attempt to find all of those scripts and see if deleting them helps, making sure to also kill all the bad processes, but since this system is "rooted" you could spend a great deal of time investigating the entire server and may never clear out all of the possible hacks/backdoors. A big issue with root level hacks is that they would have access to alter anything they want, changing configurations, replacing any of your software binaries or editing/clearing log files. You should be able to check your package manager if they messed with any files installed from packages.

More important would be to determine how the hacks started in the first place, or you could find that your next server gets compromised in the same way. This could range from "obvious" to "extremely difficult to determine" depending on how well you understand your system. Since we can see the processes are running as the root user, some possibilities are: the root user is compromised (they know the password), a sudo user is compromised, an exposed service is allowing remote execution, or there is a possibility of a privilege escalation exploit (lower chance of this if you keep your system up-to-date).

One thing you could try is to find all of the malicious scripts (find / -name x.py) and determine the earliest creation time of those files. Then check if the creation time matches any logins to your system from locations you do not recognize, that would indicate they knew the password and were able to login to your system. Otherwise it is an exercise in determining what services you are running on your system with that kind of access and do they have any vulnerabilities that would allow running commands as root.

Once your are done investigating, I would advise you save all the data that is important to you and then reinstall the server with a fresh operating system and do not reuse any of the passwords that were used on the previous system. Using backups (hopefully those are elsewhere and did not get hacked) on the fresh system would be preferable to reloading the previous data.