[jimbokun]

I worked on a project that had similar issues.

I think what saved us was the unlimited license we had negotiated for the database server we were using (which wasn’t bad technology), expired and then the company was going to charge us through the nose for any new servers.

That triggered a rearchitecture of our system, into micro services that made sense for the kinds of queries, volume, and load we were dealing with. We did have a lot of data, but many of the queries could be satisfied by a key value store, for example, which didn’t require the same amount of hardware resources.

The original project did lead us to products our customers wanted to buy. Then we just had to change the architecture to fit those actual products.