The extended attribute records that the file is quarantined, but it doesn't contain the detailed information about where the file came from. It just has an identifier that's used to look up in the database. For example, here's the quarantine info for a file in my Downloads folder: 0002;50e77ce1;Safari.app;BF047C49-2539-4536-A584-E69FDE328A2C. I don't know what the first two fields are, but the third is the app that downloaded it, and the forth is a UDID, which is presumably the identifier for the quarantine event.
Furthermore, even once a file is out of quarantine, the Get Info dialog can still tell you where it was downloaded from, precisely because of this database.
There is a reason why the marking of the file and the database containing the download information are separate. It is for personal privacy! If you copy or move the file or someone else runs whatever it is, the "quarantineness" is still there, but the personal information about where and when you downloaded it will not be accessible. The "taint" associated with the downloaded object will remain with the file, but the personal and potentially private information does not.
Furthermore, even once a file is out of quarantine, the Get Info dialog can still tell you where it was downloaded from, precisely because of this database.