Casting the rays from the camera will be nice and coherent, plus you probably get much better and easier shadowing using ray casts compared to shadow maps.
How does it handle dynamic scenes? The work by the OP looked like it took an update to the VAO/VBO if the chunk changed. What's the cost of rebuilding the acceleration structure here if the scene is updated?