FWIW you can also press the Search icon/key binding again to get out of it, or use the Back button on the toolbar. I agree that it's not at all obvious from a visual point view - I'll see what we can do.
I think what made it a little less intuitive is that the very first time I got to Search using the menu dropdown instead of the Search Icon (at that point I hadn't even noticed that there was a search icon). If I had switch to the Search view the first time using the icon, it might have been a little more obvious to me that I could back out by clicking it again.