[joyofdata]

okay - but what if frequency increases to three files and beyond within a year, the file size doubles and the expected latency of processing needs to be reduced?

Also compressing doesn't go well with ad hoc analysis.

Sounds like maybe a Hadoop setup is not the worst idea to be ready for the future.

[jononor]

If you cannot engineer to accommodate changing requirements, you are not ready for the future.