[Cenk]

From personal experience: Yes, correct markup and h1 tags etc (“On-Page SEO“) all matter. Set everything up correctly, that should take your frontend dev a couple days at most for a small to medium site. But that will do almost nothing unless it’s coupled with loads of backlinks. Get a tool like ahrefs or semrush (they’re expensive because they’re good, but a one-week trial is enough to take a look around in the beginning). There you can then see how many backlinks your competitors have and track your own rankings. Concentrate on 1-2 keywords in the beginning and fill your site with well-written content for those keywords, interlink everything on your own site, and then go get backlinks. Yes, you can buy backlinks on Fiver, but those are terrible and will probably do more harm than good. Make something good and get people to write about it.

[buboard]

> ahrefs or semrush

OK I gotta ask: these 2 particular bots constantly scan my site once an hour. Why?

[throwaway77384]

Complete speculation: They make their money by assessing back-links. So they crawl as many websites as possible, as often as possible, to establish some kind of view of who is linking to what. From that, they extrapolate some kind of popularity score, which they can probably correlate with keywords and types of users and searches.

That is what they sell back to users, presumably. I am actually quite interested to see how clear their data / recommendations would be, but haven't had the time to do so yet.

[Cenk]

> I am actually quite interested to see how clear their data / recommendations would be, but haven't had the time to do so yet.

I have tried both, and while ahrefs probably has a better dataset, semrush does very specific recommendations when it comes to link building for example. They will suggest what sites to contact to ask for backlinks (something I have not yet tried, because other people email me all the time asking for backlinks or guest posts so I know how annoying it is).

[Cenk]

Basically what the other commenters said, they scan the entire web to find out who links to whom and how many backlinks each site has.

One thing you can do is stop them from scanning your site with robots.txt – some SEO people recommend this to prevent your competitors from finding out where you got your backlinks from, but I haven’t tried it.

[andygcook]

It's likely because other companies/people are tracking your site as a competitor in Ahrefs/SEMrush.

[alvah]

There's an arms race in the SEO tools market, to have the largest / most frequently updated / most accurate backlinks database. Ahrefs have used the claim that their bot is the most active (apart from Googlebot) in their marketing.