My brother is quant physics PhD. Whenever I measure myself up to him, I realize it's good I didn't pursue academia because I'm not smart enough. It's insane how smart other people are.
Also quantum physics is about as gnarly as it gets. And not many people enter PhD's knowing all they need at the end. I'm doing a PhD at what I would consider a great school (Berkeley), and the quality still varies! People are smart yes, there is somewhat of a baseline to get in, but (most of) the smartest ones at the end have gotten there mostly through working hard, asking lots of questions, and being curious. I've seen this happen with my cohort over the past four years. Very few people came in where I thought they were geniuses, but they have all learned and grown so much in their specialty. Even if you're not the smartest one, a PhD can give you 4-7 (low) paid years of freedom to research whatever you're interested in (this depends on group), which is intellectually awesome. It's a process. You can always leave with a masters as well, but if you're interested in some sort of CS problem or engineering or science in general where CS is used for analysis (all of them), I'm sure there's a group that would be a great fit.
Shockingly, the quality of PhD students varies widely.
Just because you don't think you'd succeed in ${world_renowned_and_highly_pressured_university} doesn't mean you can't do amazing things at ${other_university}.
Seriously - if you have an original thought in your head, can write, and can justify your point of view, you can do a PhD.