Yes, but only because during your fork, you neglected to run the "white label" script that replaces the trademarked words and graphics. IOW, what you describe is not a problem in practice.
> So Ubuntu is open-source, but if I fork it, it comes with « Ubuntu » in package names, therefore I’m infringing on their trademark?
Yup. However, I suspect that Ubuntu gives a pretty permissive trademark license in their distribution.
This isn't a theoretical argument, the Android OS is open source, but the "Android" brand is a protected trademark. If you're a cell phone manufacturer, you can use the Android code freely, but if you want to put an Android sticker on your phone, you have to follow Google's rules.