[nurettin]

Yesterday I decided to write a simple log file watcher because customers especially requested for a simple script with no dependencies on the target machine. It had to detect any new logs that match the given pattern, too.

My plan was to detect new files and file modifications using two async tasks. Whenever a new file arrives, open a stream and read. Whenever a file is modified, find the previously opened stream and read. Every time I read, I push it onto an async Queue which will be consumed by another async task. That would emulate something similar to go channels. Multiple IO tasks, that sounds like a job for async/await. This was my mental model before I coded a single line.

After a few dozen minutes and a single confusing error (I forgot to pass the asyncio loop into the queue and the sleep functions) it was done. I even made it so that it would output the newly appended logs to the stdin of another process or another file. That way I could decouple my log watcher from the program that uses it.

I also opted for detecting file changes by using an async timed loop instead of inotify which would both simplify coding and buffer the new data to reduce read/write operations.

4-5 years ago, I would have experimented with code and tested different ways of accomplishing the same thing. Rather than favoring code experimentation, I had a product idea and implemented a minimal but complete example by reading docs .