redirect systemd service logs to file

Question

I was trying to run flume on Ubuntu 16.04 as systemd service and have following in /etc/systemd/system/flume-ng.service

[Unit]
Description=Apache Flume

[Service]
ExecStart=/usr/bin/nohup /opt/flume/current/bin/flume-ng agent -c /etc/flume-ng/conf -f /etc/flume-ng/conf/flume.conf --name a1 &
ExecStop=/opt/flume/current/bin/flume-ng agent stop

[Install]
WantedBy=multi-user.target

I tried adding following lines

StandardOutput=/var/log/flume-ng/log1.log
StandardError=/var/log/flume-ng/log2.log

which didn't work for me. I did run systemctl daemon-reload and systemctl restart flume-ng

anyone know how this works ?

score 71 · Answer 1 · edited Aug 21 '23 at 06:57

71

Redirect

The following works only in systemd v236 and newer:

StandardOutput=file:/var/log/flume-ng/log1.log
StandardError=file:/var/log/flume-ng/log2.log

as documented here.

In case of systemd older than v236, you can use:

ExecStart=/bin/sh -c 'exec /usr/bin/my_binary [arguments] >/var/log/flume-ng/log1.log 2>/var/log/flume-ng/log2.log'

Note that this way the whole log files contents will be overwritten each time service restarts.

Append

If you want to maintain file log between service restarts and just append new logged lines to it:

# Works only in systemd v240 and newer!
# (but backported to e.g. RHEL8)
StandardOutput=append:/var/log/flume-ng/log1.log
StandardError=append:/var/log/flume-ng/log2.log

In case of systemd older than v240, you can use:

ExecStart=/bin/sh -c 'exec /usr/bin/my_binary [arguments] >>/var/log/flume-ng/log1.log 2>>/var/log/flume-ng/log2.log'

exec means that shell program will be substituted with my_binary program after setting up redirections without forking. So there will be no difference from running my_binary directly after ExecStart=.

edited Aug 21 '23 at 06:57

NodeUnderflow

3

answered Nov 04 '18 at 00:29

Piotr Jurkiewicz

1,960

4

Note: Writing to file was only added in systemd version 236. – Dave Dec 20 '18 at 05:54
@Dave Why log file name ends with a number ? – Ciasto piekarz Jan 03 '19 at 17:30
6

the file: option creates a new file the first time and then fails on subsequent restarts of the app for me, i tried append: and it didn't work at all – Ryan Burnette Jan 29 '19 at 17:07
Last solution works fine, although it has to be /bin/sh -c ... (at least on Ubuntu). – ToFi Oct 08 '19 at 21:32
1

This worked perfectly for me with Systemd v237. – DevOpsSauce Dec 13 '21 at 20:14
Note: It wasn't clear to me from the above answer but it looks like the StandardOutput and StandardError settings need to be added under the [Service] section – Seth Jul 14 '23 at 18:20

score 26 · Accepted Answer · answered Nov 07 '16 at 19:44

26

ExecStart=/usr/bin/nohup …

This is wrong. Remove it. This service is not running in an interactive login session. There is no controlling terminal, or session leader, to send a hangup signal to it in the first place.

ExecStart=… &

This is wrong. Remove it. This is not shell script. & has no special shell-like meaning, and in any case would be the wrong way to start a service.

StandardOutput=/var/log/flume-ng/log1.log
StandardError=/var/log/flume-ng/log2.log

These are wrong. Do not use these. systemd already sends the standard output and error of the service process(es) to its journal, without any such settings in the service unit. You can view it with

journalctl -e -u flume-ng.service

answered Nov 07 '16 at 19:44

JdeBP

68,745

73

What if I wanted to send logs to a different file - is it possible or not? if possible, what is the correct way of doing this? – AnthonyK Mar 31 '17 at 02:13
What is the location of the log file? I thought the question was about it. – ka3ak Aug 12 '20 at 13:56
@ka3ak assume the logfile is stored at $logfile – Jake Sep 26 '20 at 13:27
6

how is this selected as right answer? the question is how can we redirect service specific logs to a file that user want. not how to tail it. – Vignesh SP Jun 29 '21 at 06:40

score 16 · Answer 3 · edited Oct 09 '19 at 05:14

16

For logging to file without overwriting (system.d version 240+ required):

StandardOutput=append:/var/log/flume-ng/log1.log
StandardError=append:/var/log/flume-ng/log2.log

or

StandardOutput=append:/var/log/flume-ng/log.log
StandardError=inherit

edited Oct 09 '19 at 05:14

ToFi

103

answered Jan 29 '19 at 10:03

Billy McKee

161

9

This is unfortunately not working for me. The service still starts but nothing appears in the log file and if deleted it is not recreated. Ununtu 18.04, system.d 237 – SystemicPlural Apr 09 '19 at 11:08
2

@SystemicPlural you're right. This was only added in system.d 240 with a PullRequest for this issue: https://github.com/systemd/systemd/issues/10875 – ToFi Oct 08 '19 at 21:01

kubarik · Answer 4 · 2019-10-09T13:33:38.063

I hope this helps someone. for Ununtu

[Unit]
Description=name-service
After=syslog.target
After=mysql.service
Requires=mysql.service
[Service]
Type=idle
PIDFile=/path_to_PID/****.pid
WorkingDirectory=/path_to_directory
User=root
Group=root
ExecStart=****.py
Restart=always
TimeoutStartSec=1min
RestartSec=1min
KillMode=process
StandardOutput=append:/path_to_log/service.log
StandardError=append:/path_to_log/service_error.log
[Install]
WantedBy=multi-user.target

for CENTOS

[Unit]
Description=name-service
After=syslog.target
After=mysql.service
Requires=mysql.service
[Service]
Type=idle
PIDFile=/path_to_PID/****.pid
WorkingDirectory=/path_to_directory
User=root
Group=root
ExecStart=****.py  >>/path_to_log/log.log
Restart=always
TimeoutStartSec=1min
RestartSec=1min
KillMode=process
[Install]
WantedBy=multi-user.target

"After" and "Requires" of necessity. Attension "User" and "Group"

Another option: add logging inside executed scripts.

What's your question? You should start a new question instead of posting it as an answer here. — ewatt, Oct 09 '19 at 08:53
I'm sorry. I have written it wrong. "I hope someone will help"->"I hope this helps someone." — kubarik, Oct 09 '19 at 13:33

score 2 · Answer 5 · answered Sep 10 '20 at 08:34

2

If you don't get any output at all you should test your script first.

For example a Python script will buffer the output streams (but only when they are piped, not when you test on the console). So you will not see any log messages until the buffer is flushed.

Specify the -u switch ( python3 -u script) or set the PYTHONUNBUFFERED variable to avoid this:

Environment=PYTHONUNBUFFERED=1

answered Sep 10 '20 at 08:34

laktak

5,946

1

Wow, this is very helpful. In a perl script setting $|=1; also unbuffers. Fixed two of my units. – Bill McGonigle Apr 28 '22 at 19:08

lupguo · Answer 6 · 2019-11-13T10:58:33.257

[Unit]
Description=ngrokd server

[Service]
Type=forking
User=ngrok
Group=ngrok
ExecStart=/bin/sh -c '/usr/local/ngrok/bin/ngrokd \
    -tlsCrt /usr/local/ngrok/tls/srv-ngrok.cert \
    -tlsKey /usr/local/ngrok/tls/srv-ngrok.key \
    -domain="ioio.cool" \
    -httpAddr=:8480 \
    -httpsAddr=:8433 \
    -tunnelAddr=:8422 >> /var/log/ngrokd.log 2>&1 &'
Restart=always
KillMode=process

[Install]
WantedBy=multi-user.target

You can see man systemd.exec, search keywords StandardOutput, and man systemd.service, serach keywords COMMAND LINES

This syntax is intended to be very similar to shell syntax, but only the meta-characters and expansions described in the following paragraphs are understood. Specifically, redirection using "<", "<<", ">", and ">>", pipes using "|", running programs in the background using "&", and other elements of shell syntax are not supported.

You can remove the trailing ampersand & from the commandline because this is not an interactive shell. The startup will be forked anyway. — Ned64, Sep 05 '22 at 10:50

redirect systemd service logs to file

6 Answers6

Redirect

Append

Linked