According to that, the Riot Act was an English law that was actually read to protestors ordering them to stop what they were doing, and threatening that if they did not stop, they would be executed.
So, metaphorically, reading someone the riot act is a threat that if the behavior continues, there will be severe consequences.