0

I scheduled a one-time job with atd on an OracleLinux 7 server. However, it's still in the queue after the system clock has passed the scheduled time.

# atq
2       Mon Mar 14 00:00:00 2022 a root
# date
Mon Mar 14 09:38:15 NZDT 2022

The atd service is running:

# systemctl status atd -l
● atd.service - After the Deadline Spell Check Service
   Loaded: loaded (/etc/systemd/system/atd.service; enabled; vendor preset: enabled)
   Active: active (running) since Tue 2022-02-08 11:16:33 NZDT; 1 months 3 days ago
  Process: 1259 ExecStart=/opt/atd/startup.sh (code=exited, status=0/SUCCESS)
 Main PID: 1270 (sh)
   CGroup: /system.slice/atd.service
           ├─1270 /bin/sh ./run.sh
           └─1283 java -server -Datd.lowmem=true -Dsleep.pattern_cache_size=8192 -Dbind.interface=127.0.0.1 -Dserver.port=1049 -Xmx3840M -XX:ParallelGCThreads=8 -XX:+AggressiveHeap -XX:+UseParallelGC -Dsleep.classpath=./lib:./service/code -Dsleep.debug=24 -classpath ./lib/* httpd.Moconti atdconfig.sl

The job is simply altering some files with built-in commands, and it was successfully executed on other servers.

I also tried to check if the command was stuck, but I couldn't find any process which is running. So I can conclude, the atd job was even hasn't kicked off!

Under /var/spool/cron/root, there's a file called a0000201a2e594. Based on this thread: https://stackoverflow.com/questions/4304631/rescheduling-an-at-job we can get the time stamp by converting the last 8 digits to human readable time. So https://www.epochconverter.com/ tells me the job would be executed at 13 Mar 2022 20:51:41 in my timezone? Doesn't match the time I defined, but it's definitely passed the schedule.

Why the job couldn't happen on this box?

NeilWang
  • 272

1 Answers1

0

Weirdly enough, I found the atd service invoked /opt/atd/startup.sh script, and after taking a deep look into that script, it was just a java script.

I manually ran the /usr/sbin/atd and I noticed the queued job has been processed.

I also found the /var/run/atd.pid must exist to prove the atd daemon is running.

So, the issue was caused by somebody modified the systemctl service and "fake" the atd is running!

NeilWang
  • 272
  • If you do not know who or what did that at around 2022-02-08 11:16:33 NZDT, then I would regard the secrity of your system as suspect. – Bib Mar 14 '22 at 12:40