Hacker News new | ask | show | jobs
by kqr 334 days ago
I'm going to assume this project is at best a few years old, so return over index is more likely to be an indicator of overbetting (taking on too much risk) than a performance indicator.

Backtesting would be more useful. Of course, LLMs cannot be backtested since they know the past.

This system is impossible to test. I would be hesitant to trust it.

2 comments

>> I'm going to assume this project is at best a few years old, so return over index is more likely to be an indicator of overbetting (taking on too much risk) than a performance indicator.

Best metric here would be a Sharpe Ratio and drawdown details. https://en.wikipedia.org/wiki/Sharpe_ratio

> This system is impossible to test. I would be hesitant to trust it.

Disagree. Best test would be a paper test going forward, audited by a common platform.

Any forward test depends on market assumptions. I'm not a practitioner, and as a layperson I wouldn't know whose assumptions to trust.
it looks like the code was committed a week ago. I do not see an iterative history on the code