I imagine soil quality and time of harvest also play a decent part in taste. All other factors being equal picking a green tomato isn’t going to be as good as letting it ripen on the vine naturally.
Terroir is the term for this. It's variation in flavour isn't as great as varietal/strain based differences. But yes it does have some impact most commonly taken advantage of in wine production.