[nkaz001]

No, that's why the project focuses more on cryptocurrency, which offers easier access to data and the market. I'm trying to understand trad-fi markets and implementing L3 market-by-order backtesting using Databento data, but creating a live bot without the market access would be impossible.

[RayVR]

Interactive brokers is probably the best non-professional setup I’m aware of. You won’t get hft level latencies or price feeds.

to get good historical data in “trad-fi” you would need to map all the trades by code to filter out bad ticks before aggregating into bars. This alone is enough of a barrier to keep most retail traders out of the space

[nkaz001]

databento provides high-quality market-by-order data at a reasonable price, so at least I can play with the data, though I'm unsure about actual trading. That's why people including me like crypto.

[RayVR]

from what I understand, databento is just aggregating exchange data. They must have some deal for processed data. Their MBO data is not cheap and they don't have much history. I would not trust their bars without a whitepaper documenting how they clean each market.

[someguydave]

“by code” what codes? Do you mean offered but cancelled bids/asks and things like that?

[RayVR]

No. The major concern is that bad ticks make it through your filters, which is more than likely going to make your models do something wonky.

It's been many years since I worked on cleaning data but essentially each event you receive is mapped to a code. Depending on your data provider, those codes may change at any time. Thomson Reuters is/was quite notorious for this. Maybe after Refinitiv spun out they cleaned up that mess but from what I have heard it is unlikely.