| With v3 we're bringing the v1 API into it so that people will be able to interact with it as if it were a v1 database. There will be data migration tools from both v1 and v2 into v3. We would have loved to bring the v2 API along, but ultimately the API and the Flux language surface area were far too large. We tried to do too much all at once with v2. Our goal with v3 is to bring the focus back onto the core of the database. I said a bit of this in our community Slack channel earlier today, but I think it's worth repeating here: I really would have loved for us to have more time developing Flux as a language, but we spent all of our time trying to optimize its performance. As a result, we weren't able to make changes to it that I think would have helped people understand and use it. We pushed on it for 4 years and were told again and again by new users and people that chose other tools that they didn't want to adopt Flux and found it difficult to use. I realize that this wasn't universal and we had some real fans. But we still weren't able to bring the performance that most of our users required.
At the end of the day it was just too much to try to do in addition to creating the core database and all the managed, hosted service around it. I was told once quite some time ago that while Flux made the impossible things possible (i.e. you could do things you actually couldn't do in other query languages), it made the easy things hard. I think this was a correct assessment and that it's what caused the getting started experience to be so poor. We knew this and wanted to improve it. I brainstormed with Nathaniel (primary Flux author & creator) around 3.5 years ago to make changes to the language. We did a lot of work and documented the whole thing: https://github.com/influxdata/flux/blob/master/docs/new_flux... But we couldn't ever get to that work. We shelved it because we were constantly fighting performance and reliability fires. With version 3 we built around an existing query engine (Apache DataFusion) and it's getting developed not only by us, but developers around the world. We think this will have huge benefits over time that lead to a faster query engine that has more features and is more reliable. |