As far as I know, where I have lived most(Western Australia) it is not considered vandalism.
This is the third result on Google for "vandalism chalk"(though it seems to have some mixed messages):
https://itsartlaw.com/2013/09/02/california-court-rules-that...