Of course! Anyone can grab something from the ocean floor (~3000m deep), especially something as easy to find as a lump of glass. Glass is also totally easy to melt on your kitchen stove and adding radioactive glass to the bomb you have lying around anyway, turns the bomb from a harmless firecracker into a totally menacing doomsday machine.
Yes, that's the first thing a terrorist would think of.
Seriously, dirty bombs are a red herring. It's the "bomb" part you should be worried about, not the "dirty" part.