It may not need to be entirely retrained. The value spans and input are the same, and no extra weights are needed. You may be able to tune an existing model with this attention mechanism and get some of the benefits.
But overall... it's mainly a training change, so training is needed to make a difference.
But overall... it's mainly a training change, so training is needed to make a difference.