[koningrobot]

Yes, I've worked with this exact algorithm before, and others like it, and it should be much worse. Better than finite diffs or reinforce, but way worse than backprop's exact gradients, to the point of being unusable. Moreover the variance of the estimate grows with the number of parameters, so it gets worse still on bigger problems.