2

I want to watch a file created and written in a directory by a process. I don't have access to inotifywait tool, so I'm using just the inotify library calls. I start with IN_CREATE to monitor that the file was created. If the file of interest was created, then I drop in a CLOSE_WRITE event to be monitored. This scheme kind of works, except in the case that the file was already written and closed by the time my handler processes the IN_CREATE event, and I miss that write event.

What is the best way to work around this situation? I was considering using the stat function to use atim, mtim, but wasn't sure if that would have any gotchas. For example, I was planning to check if the current timestamp is greater than st_ctim, and if st_ctim is greater than st_atim. These two conditions would indicate that the file status changed (accessed and changed) before the current time.

rookie
  • 135

1 Answers1

0

First of all, I am not sure what do you mean, when stating: "If the file of interest was created, then I drop in a CLOSE_WRITE event to be monitored."

You can monitor IN_CREATE and IN_CLOSE_WRITE in the same poll, there is no need to switch between them. Once the event happen, you read the inotify_event structure from the inotify handler (returned from inotify_init()) and the field mask of it will tell you which event actually happened.

Second, there is a chance to miss the inotify event. It is clearly stated in the documentation. There are whole list of possible problems listed under "Limitations and caveats" title in the inotify(7).

It looks like you encountered the "queue overflowing". Yes, it is a real thing, which happens quite often on the busy system. Unfortunately, it is not possible to rectify. Only work around it. There are several ways to solve it, but cannot say more specific without better understanding of what are your needs.

In general, if you want to see an updated file as soon as it is updated - just scandir() the directory with sorting and/or filtering files by they mtime. In this scenario, the inotify would serve as a wake up call "something happened in the directory", upon receiving, you go into long-running process of checking all the updates. Once they are done - sleep in the poll(). In this case, you will ignore the events which happened between end of one poll() and start of the next one. But you will get all the updates in bulk.

White Owl
  • 5,129
  • My file does not exist in the directory when I start my program, so I watch the directory. I think I ran into errors when I added IN_CLOSE_WRITE to my add_watch call, because the path I provided was of a directory not a file. – rookie Sep 12 '22 at 02:27
  • My need is to capture the first write_close event on this file. It is assumed that this file is only ever written once. – rookie Sep 12 '22 at 02:28
  • I believe you can ask inotify about a file and it can tell you if anything has the file open for write. So presumably, if you are notified of the file's creation (or, at least, the directory's modification), and you scandir and the file is there, and then you grab the file with inotify and find nothing has it open for write, you can assume the last write already occurred. I've used this, but I can't recall the exact details. – user10489 Sep 12 '22 at 02:35
  • @user10489 what API of inotify is that? – rookie Sep 12 '22 at 03:04
  • inotify is the API. It has a man page and everything. Maybe some day I'll reread it and give you a better answer. :-> But it's there. – user10489 Sep 12 '22 at 05:24
  • @rookie In this case, you do not need IN_CREATE at all. Just the IN_CLOSE_WRITE on the directory would be enough. If you need to process the incoming files, create a loop like: if dir is empty run a poll() on it, upon receiving IN_CLOSE_WRITE move all files from incoming dir to a secondary "processing" dir, process files, check is incoming dir empty, if not repeat to processing, if in-dir is empty - go to poll(). – White Owl Sep 12 '22 at 12:18
  • @WhiteOwl thanks! – rookie Sep 12 '22 at 19:17
  • @WhiteOwl I'm concerned of the case where I call my watcher at an instant where the file is open (being written by the other process) and just as I put in the CLOSE_WRITE, the file gets closed and I miss that event. I might still have to overcome that race – rookie Sep 12 '22 at 20:23
  • @rookie That is why I'm suggesting to not rely on inotify too much. Only to wake your thread from a sleeo, If monitored dir is empty - sleep inside a poll(), once it wakes up by inotify - process everything until the dir is empty again, and go to sleep in the poll(). – White Owl Sep 13 '22 at 13:19