[yaakov34]

This calculation amplifies any noise present in the values of the function, often to the point of the output being unusable. There are many methods that can be used to approximate derivatives, depending on the problem. Just as we shouldn't try to invent our cryptographic methods from scratch, we should take advantage of the extensive knowledge already in use for numeric methods.

I've seen naive numeric methods cause everything from jerky motion in video games to incorrect navigation data for cars.

[kqr]

So what would you suggest as a general calculation for finite differences, especially in those cases when only forward differentiation is possible, e.g. with respect to time?

[yaakov34]

The closest thing to a universal approach would be a Kalman filter. It's usually where you start when you have noisy measurements coming in, and you need to maintain state such as value and derivative.

Since the original question was about computing the velocity of a car, and since I work in the automotive field, let's take a real example: you want to know the approximate position, acceleration, and velocity (linear and angular) of your car. Your inputs are driven wheel speed (noisy, affected by wheelspin), non-driven wheel speed (noisy), accelerometer output (inaccurate, only present for some axes), GPS position (updated occasionally, has errors), and steering angle (pretty accurate, can be put into a chassis dynamics model). Almost certainly, you would use a Kalman filter to estimate the state of the car. Naive approaches such as subtracting two wheel speed values to obtain acceleration will not work well.

My point is that we should remember that numerical algorithms are a developed field with a lot of knowledge, and we should take advantage of the proven approaches. Sometimes, programmers who are not specifically from the physics or numerical fields, and who need to perform some computation, reach for a very simple approach such as the rectangle-rule integrals, and get bad results.

[kqr]

I see -- we are talking about two different things!

You are working on the problem of figuring out the hidden state based on noisy observations and a transition model.

I interpreted your statement much more broadly, so I was trying to discuss the problem of computing the explicit next state based on perfect knowledge of the transition model, in which case (y_1-y)/dt is a perfectly viable approach to estimate the derivative.

(You can do better by adding higher-order terms of course, but I haven't found that to be universally useful compared to making dt smaller.)