[ralphael]

..this right here. No error checking, inadequate testing, no reconciliation process.

Excel is used extensively in many industries. Any file could be cut off in processing by any number of reasons, one off errors for e.g.

So the solution is to "fix" the process by using the existing broken process and smaller files....