However, there will
probably be a window in which both oldpath and newpath refer to
the file being renamed.
But that's not even the main point.
1. Move happens after email is sent, so there is a window where email is already being sent but file still exists.
2. Even if you do it before, there's still a window between os.listdir() and os.remove()
3. Complexity is O(N^2) due to listdir() + getctime() being called on every iteration.
If you just want to ensure order, it probably works fine at a small scale. But it would be unwise to run multiple consumers on a single instance, and impossible to run them on multiple instances.
I worked with a network filesystem that supported atomic renames and we based an entire large-scale production system on the idea that it would work (it did). The system supported Youtube and Google Play model training, regularly processing increments of hundreds of terabytes.
1. Move happens after email is sent, so there is a window where email is already being sent but file still exists. 2. Even if you do it before, there's still a window between os.listdir() and os.remove() 3. Complexity is O(N^2) due to listdir() + getctime() being called on every iteration.
If you just want to ensure order, it probably works fine at a small scale. But it would be unwise to run multiple consumers on a single instance, and impossible to run them on multiple instances.