Is it just me or does the Vanilla-A* algorithms look too bad? I mean especailly in the second image [(d) A* (Adaptive Depth)], why does it search behind itself?
This behaviour exists because A*'s heuristic function assumes there are no obstacles when estimating the remaining distance to reach the goal. As the difference between the heuristic estimate to the goal and the actual distance increases, some nodes "behind" the start location will appear more promising than other nodes which are eventually proven to be on the optimal path.
It's a graph search. Imagine adding an out-of-plane edge that travels from behind the starting point to the goal. Since taming that path could be optimal it has to check - it doesn't attempt to reason that such a path would necessarily be obstructed by what's been explored thus far. It's just a graph search.
Suppose there was a wall or other obstacle between you and the destination. Then you might have to go "back" to find a way around. Vanilla A* can't "know" whether this is the case, so it searches outwards in all directions, using the heuristic to weight the search towards paths that are getting us closer to the destination.