Google's Crawling of Newly-Added Internal Links

Discussion in 'Google' started by Jim4767, Jul 17, 2023.

  1. #1
    Let’s say, for example, that I create some new internal links from my existing webpages A, B, and C to my home page.

    Usually I then submit those pages A, B, and C (with their newly-added internal links) to my Google Search Console. But a question occurred to me, and perhaps someone knows the precise answer (no guesses, please).

    If I only submit to my Google Search Console the home page that received those 3 new internal links, does the Google crawler have a way to discern just from the receiving home page the existence of those 3 new incoming links from the 3 other pages, and then go out to recrawl them too? In other words, can the Google crawler discern just from crawling the page receiving the link that there actually is a new link somewhere pointing to that page? Or can the crawler only discover that new internal link directly from crawling the sending page?
     
    Jim4767, Jul 17, 2023 IP
  2. sarahk

    sarahk iTamer Staff

    Messages:
    28,789
    Likes Received:
    4,528
    Best Answers:
    123
    Trophy Points:
    665
    #2
    We don't know just how much of browser traffic and analytics data gets used by google in search, or do we? I wouldn't rely on that, and would submit a full sitemap.
     
    sarahk, Jul 17, 2023 IP
  3. Oso Optimized

    Oso Optimized Active Member

    Messages:
    259
    Likes Received:
    36
    Best Answers:
    1
    Trophy Points:
    80
    #3
    I found the wording of your question a little confusing..

    But if I create new pages in a website, I make sure that I place a link to those pages somewhere in my site. Google absolutely finds them. I do nothing else.

    I believe this is sufficient for the sites I work with (most have only a few hundred pages). Seems like your site is thousands of pages, so submitting the sitemap might be good in that case.
     
    Oso Optimized, Jul 18, 2023 IP
  4. kjh-08

    kjh-08 Well-Known Member

    Messages:
    980
    Likes Received:
    92
    Best Answers:
    11
    Trophy Points:
    130
    #4
    There's two types of crawls that Google does, discovery crawls and refresh crawls. How often they occur depends on how often you publish new content and then how often you refresh older content.

    If those three links are not on the homepage but rather pointing to it, then the crawler wouldn't know they existed if it only crawled the homepage.

    So, it depends on how often you you publish new content and refresh. Submitting the pages in GSC would be the step to take if you didn't want to wait.

    A sitemap is for discovery and what you're looking for in this example is a refresh crawl.
     
    kjh-08, Jul 18, 2023 IP
    Jim4767 and sarahk like this.
  5. Jim4767

    Jim4767 Prominent Member

    Messages:
    4,738
    Likes Received:
    766
    Best Answers:
    10
    Trophy Points:
    305
    #5
    Thanks all. And sorry, Oso. I did sense that I was struggling with my wording to get my point across. kjh-08 dug through the clutter of my clumsy wording and nailed it with his second paragraph.

    P.S. I do submit a sitemap.xml every time I add a new page, but not when refreshing existing ones. For those I'll continue to use Google Search Console's submit-url function.
     
    Jim4767, Jul 19, 2023 IP
  6. bijutoha

    bijutoha Well-Known Member

    Messages:
    197
    Likes Received:
    5
    Best Answers:
    1
    Trophy Points:
    173
    #6
    Yes, Google's crawler can discern from the receiving home page the existence of those three new incoming links from the three other pages and then go out to recrawl them too. This is because Google crawlers follow all links they find on a page, including internal ones. When the crawler crawls the home page and sees the new internal links, it will follow those links to the pages they point to.
    You don't need to submit all pages with new internal links to Google Search Console. You can just submit the home page, and the crawler will eventually discover the new links and crawl the other pages.

    Caution
    However, it's important to note that the crawler may not discover the new links immediately. It may take some time for the crawler to crawl the home page and then follow the links to the other pages. So, you may not see the new links reflected in Google Search Console right away.
     
    bijutoha, Jul 21, 2023 IP