You can define a periodic function that goes from 0 to 1 as a straight line and then from 1 to 0 as another straight line, basically a triangle wave. It is not differentiable on an infinite (and countable) number of points:
I imagine they would have been talking about functions on the interval rather than our modern sense of functions on the reals. (Of course there are plenty of examples on the interval as well, but no obvious ones)
https://en.m.wikipedia.org/wiki/Triangle_wave