[driuha]

And its never going to work, at least in its current form. The problem is that some bricks will look the same from top while being different on the bottom. That produces wrong suggestions and you can only discover it when you checked all pieces that are needed for the suggested model. Perhaps it can work with basic collection of pieces, like Lego classic but it can't work with random heap I throw at it.

[Tempest1981]

How about 2 or 3 passes, where are you shuffle the pieces around and take another scan or two, to resolve ambiguity.

[driuha]

It will resolve some but it will probably add additional errors like confuse bricks from previous photo with each other. You need to separate pieces and take multiple shots like in that Lego-sorter video. You can probably get away with filming the bricks from different angles, preprocessing video and using that to feed into nn.