Meh. It doesn't require Javascript for your computer to run logic described by others. Browsers are such complex machines that it wouldn't surprise me if you could for example craft a malicious SVG that would bypass that, or a turing-complete CSS file that triggers a vulnerability...
By the way, does NoScript actually block in-SVG javascript?
Sure, but we all take risks every day. If you're worring about turning-complete CSS files exploiting Spectre and Meltdown then you probably don't leave the house much.
We know that attackers have reason to exploit literally all compute resources they can find a way to access. This is more like worrying about leaving the house during an epidemic of exploding ebola-infected pigeons — if you can do something about it, you should.
Attackers also have to consider cost/benefit analysis when evaluating methods of attack. Claims that "CSS is Turing complete" require a user to act as a "crank" [0], so there are lower-hanging fruit out there than trying to program complicated logic which can utilize the Meltdown / Spectre exploits in CSS.
By the way, does NoScript actually block in-SVG javascript?