Can you provide more details? Because i didn't encounter such problems. Was this 500 MB in top was RSS ?
What you describe should not happen, did you report a bug?
For me it seems like you probably leaked gorutines/not closed os resources or you encapsulated some data structures in themselves which then can't be garbage collected. It really looks like memory leak caused by app logic. But as i said you should report this if you are certain that is Go bug.