In a way I agree with those who claim that Google throttles your progress. Over the years I've seen it over and over again. As an example, my search console shows that whenever it starts going up, at some point, their algo brings it down. There's no real reason for it to go down, except it's all automated and done to throttle your progress.
If they make organic results varying and unpredictable, it is hard for a business to rely on them. That pushes some businesses to AdWords which makes Google money. AdWords tends to produce more consistent results and it also seems to reward higher spending with better results. But my experience is in very niche-y B2B technology solutions, so keep that in mind.
You're 100% right, it's all about their ever-insatiable bottom line, even though you could've had all sorts of traffic if a more reliable company ran the show.
Here's another interesting fact: I wrote code that masks my links, so bots can't access the parts of the site I don't want them to reach. Googlebot can't see them either, it scans the links but lands on a generic "Page not found" error page. After a couple of months, my Google rankings actually went up slightly. It turns out that all these years, Google and other bots were draining my server resources for nothing. All they really need is to index your homepage. You'll rank just as well without them crawling millions of pages (as they did in my case).
This is not the same, because your site is constantly changing, adding new content, etc., but I have a new, pretty static site that was getting clobbered by bots for weeks and weeks. I did not do anything at all. Eventually, they apparently got sick of it and just stopped. My traffic is back to normal (very low) and I do not see bots hitting it at all. Hasn't helped or hurt my search results (which are not great, anyway.)
Good for you! Keep in mind that most bots today don't look like obvious bots. Their user agents often mimic normal browsers. For example, I just copied this one from what appears to be an AI bot or automated visit: "Mozilla/5.0 (Macintosh; Intel Mac OS X 13_3_1) AppleWebKit/537.36 (KHTML, like Gecko) Chrome/131.0.6778.139 Safari/537.36" It looks exactly like a legitimate visit from a regular browser. Some days, they hit my site 100–200 times per second for a few minutes. Fun.
I've been using playwrite & headless chrome to make some api requests of a well-known social media site and they let me run for about a month and then shut me down. Apparently, even when you set it up perfectly, there are "tells". Look that up and see if any of those appear in your problem traffic.