"Green card" literally refers to US permanent residency cards; it's called that because the physical cards issued by the US are/were green. "Other contexts" are riffing on actual green cards as a metaphor, and if speakers in other contexts want to talk about legal specificities, they should use an accurate term...
In other contexts, it literally is.