[euyyn]

In the odd case that the data is clearly garbage, you'll just drop it and keep going as before (so you add a frame to the response time at that specific moment).

In the general case of data affected by noise, it shouldn't be a problem: These things run a simulation of the external world, and every new input frame is never fully trusted, just used probabilistically to update the simulation. It is expected to carry noise, so if your sensors would tell you something like "a pedestrian in the sidewalk just jumped 10m in the air", you'd deduce "the guy most probably has kept walking as he was doing before, he possibly jumped instead, or he might have changed directions instead".