Hacker News new | ask | show | jobs
by syntaxing 3396 days ago
I see from the earlier comments that this runs some sort of "optimized" ARIMA model. Is there anyway to output the statistical information of the fitted model though the API?
1 comments

Not really. We're directing this more towards a completely automated use-case of forecasting with no human interaction, so it's not on our plans to release internal parameterization of the forecast.
Hmm, I think at the minimum, the variance of the forecasted results should be obtainable. I'm not sure how many people would use a black box model without knowing some sort of performance/statistical confidence metric.
We could return a confidence interval for the forecasted values, you're right, it can help in cases where it would be used for simple anomaly detection (and it would give a greater sense of security/control over using the forecast).

I do believe we have some similar feature in the pipeline for development, I'll make sure to push it forward. Thanks for the feedback.

I think a confidence interval would be crucial for the forecasted values.
IMO this would be much more appealing if there were an option to get the model details. I can't see myself presenting results to a group and answering the question "How did you get this?" with "Well, I put my data into this box and what came out seemed pretty good."
Well, if we provide confidence intervals it should provide some kind of "safety"... but even so, with forecasting the only assurance you can get is by testing it against known historical values and checking it's accuracy (and even that way, historical results are no guarantee of future performance).

What kind of details would you say can be inspected to see if the model is reliable? AR or MA orders, inferred seasonalities? They can give me some notion of what kinds of assumptions were created about my data, but do not guarantee that it will perform :/

For instances where that kind of insight is needed, I don't think our way is the way to go, but rather the use of some forecasting package (R's Forecast or FB's Prophet) and a more exploratory work. But we're looking more at instances where what matters are the forecasted values and not so much the information underneath - automated anomaly detection systems, consumer-facing apps, and along those lines.

That's a fair answer. Your target audience is interested in prediction rather than inferring parameters.

I do think the confidence intervals/prediction intervals should be accessible and should probably be adjustable (e.g. 99%, 95%, 80%).

> I do think the confidence intervals/prediction intervals should be accessible and should probably be adjustable (e.g. 99%, 95%, 80%).

Definitely, this is arguably the most important feature.