Hacker News new | ask | show | jobs
by MJSplot_author 2273 days ago
Just my initial response.

Maybe fine a better first example? It is quite code dense:

    conv = c.aggregate({
       "a": c.reduce(c.ReduceFuncs.Array, c.item("a")),
       "a_sum": c.reduce(c.ReduceFuncs.Sum, c.item("a")),
       "b": c.reduce(c.ReduceFuncs.ArrayDistinct, c.item("b")),
    }).gen_converter()
    conv(input_data)
when compared a trivial native python equivalent:

    conv = lambda data:{ 'a': [el['a'] for el in data ],
                    'a_sum' : sum( [el['a'] for el in data ]),
                    'b': list(set( [el['b'] for el in data ])), }
    conv(input_data) 
which appears to have the same functionality. This is quite off putting and it took me a while to dig down to find why convtools can offer more than just an extra abstraction layer to learn. Perhaps pick an example that shows off the non trivial functions like joins or GroupBy?
2 comments

Just to add my previous answer: the trivial native python equivalent doesn't have the same functionality, because it consumes data iterator 3 times in your case, while convtools would consume it only once.
Thank you! I will add join and group_by examples shortly