The Importance Of Being Indexed
In layman’s terms, indexing is the process of having Google add your webpages into Google, the search results [Search Engine Results Page aka SERPs].
Here are a few reasons for being indexed:
- Google has recognized your website as being a source of information on a particular subject
- Once people start searching for information, your site starts showing up
- Your website is on its way to ‘having a presence’ on the world wide web
One of the most common problems experienced when trying to rank in Google, is that your website is not currently being indexed or being indexed correctly. If this is the case, it means Google is failing to access your web pages to index your site’s content effectively.
What is Google Index Status and Why Does It Matter?
One of the common misconceptions people have of Google is that when they use Google, they are searching the web. They aren’t searching the web at all. They are searching Google’s index status of the web. One of the best ways to think about this is to think of how a library was once organized. Back in the day, OK ten years ago, when you went to find a book you went to the card catalog to find a book on a particular subject. If the card within the card catalog did not exist, you couldn’t find the book in the library.
Google works in a similar manner. Every hour of every day, Google and its robots/spiders is clicking on links to see where they go. They start on major websites like Wikipedia, the New York Times and their spiders go through the entire website and click on links that take them to other websites, where they repeat the process and go onto more and more until they reach presumably every website that has a link to other websites. As the spiders crawl over the web, they send information back to Google about what the website is about by looking at various keywords on the site. Google’s index stores this information. When people search Google, the Google algorithm will rank the best results from its index in the order of 1 – infinity.
Therefore, in order to have websites rank within Google, the first step is to make sure your site is within the Google index. No matter the work you do to make sure the the site is on the right content management system or has the right keywords in place, if you don’t have those pages within the index, they will not show up in Google search results.
According to searchengineland.com and its author Barry Schwartz Google has announced that Sites that follow the best practices for mobile-first indexing will be migrating over now, and Google will send notifications via Search Console.
Does or will this affect your website
Google has announced that it has begun the process of rolling out the mobile-first indexing to more sites. This rollout is only for sites that “follow the best practices for mobile-first indexing,” Google said.
This is the first time Google has confirmed it is moving a large number of sites to this mobile-first indexing process. Google did tell us last October that a limited number of sites had been moved over. But this Google announcement makes it sound like the process of mobile-first indexing on a larger scale has already begun.
Google did say it will notify webmasters/site owners that their sites are migrated to the mobile-first indexing process via messages in the Google Search Console. Here is a screen shot of a notification:
Google also said “site owners will see significantly increased crawl rate from the Smartphone Googlebot. Additionally, Google will show the mobile version of pages in Search results and Google cached pages.”
What is mobile-first indexing?
Google says it is about how Google crawls your site. Google will only have one index, but how Google crawls and creates the index will be based on a mobile-first experience going forward. Google wrote:
To recap, our crawling, indexing, and ranking systems have typically used the desktop version of a page’s content, which may cause issues for mobile searchers when that version is vastly different from the mobile version. Mobile-first indexing means that we’ll use the mobile version of the page for indexing and ranking, to better help our — primarily mobile — users find what they’re looking for.
FAQ: All about the Google mobile-first index
Google is rolling out a new mobile-first index. This means Google will create and rank its search listings based on the mobile version of content, even for listings that are shown to desktop users. Wondering how this will all work? We’ve compiled answers below.
Since the announcement, we have been tracking what Googlers have been saying about the change based on the industry’s questions. Below you will find a compilation of those questions and answers based on coverage from Jenny Halasz, Jennifer Slegg and me.
What is changing with the mobile-first index?
As more and more searches happen on mobile, Google wants its index and results to represent the majority of their users — who are mobile searchers.
Google has started to use the mobile version of the web as their primary search engine index. A search engine index is a collection of pages/documents that the search engine has discovered, primarily through crawling the web through links. Google has crawled the web from a desktop browser point of view, and now Google is changing that to crawl the web from a mobile browser view.
What if I don’t have a mobile website?
Google said not to worry. Although Google wants you to have a mobile site, it will crawl your desktop version instead. Google said, “If you only have a desktop site, we’ll continue to index your desktop site just fine, even if we’re using a mobile user agent to view your site.”
If you have a mobile site, then you need to make sure the content and links on the mobile site are similar enough to the desktop version so that Google can consume the proper content and rank your site as well as it did by crawling your desktop site.
My mobile site has less content than my desktop site. Should I be nervous?
Potentially, yes. Google has said that it will look at the mobile version of your site. If that has less content on page A than the desktop version of page A, then Google will probably just see the mobile version with less content.
This is why Google recommends you go with a responsive approach — the content is the same on a page-by-page basis from your desktop to your mobile site. You can do the same with other mobile implementations, but there is more room for error.
What about expandable content on mobile?
With desktop sites, Google said that content hidden in tabs, accordions, expandable boxes and other methods would not be weighted as high. But when it comes to mobile, Google’s Gary Illyes said content like this will be given full weight if done for user experience purposes. The idea is that expandable content makes sense on mobile and not so much on desktop.
Will this change the Google rankings in a big way?
Both Gary Illyes and Paul Haahr from Google said this should not change the overall rankings. In fact, they want there to be minimal change in rankings around this change. Of course, it is too early to tell, they said — but their goal is not to have this indexing change impact the current rankings too much.
When will this fully roll out?
Google said they have already begun testing this mobile-first index to some users. But it looks like we are still months away from this fully rolling out. Google won’t give us a date because they are still testing the rollout, and if things go well, they may push it sooner. If things do not go well, they may push it back.
Google did say they will push this out to more and more searchers over time as they become more confident with the mobile-first index.
Is this a mobile-friendly ranking boost?
Google has previously said that content that’s not deemed mobile-friendly will not rank as well. That remains the case with this new index.
In the current index, which most people will continue to get results from, desktop content is indexed and used for showing listings to both desktop and mobile users. A special mobile-friendly ranking system is then used to boost content for Google’s mobile listings. Content that’s not mobile-friendly doesn’t perform as well.
In the new mobile-first index, which some people will get results from as Google rolls it out, mobile content is indexed and used for showing listings to both desktop and mobile users. Then the mobile-friendly ranking boost is applied, as with the current system, to mobile-friendly pages.
How can I tell if Google sees my mobile pages?
The best way is to use the Fetch and Render tool in the Google Search Console. Specify the mobile:smartphone user-agent and look at the preview after the fetch and render is complete. What Google shows you in the rendered results is likely what Google can see and index from your mobile site. If content is missing, then you should look into how to fix that and run the tool again.
Ranking signals will come from your mobile, not desktop version
Google has ranked your mobile site based on many signals from your desktop site, as we covered before. That is going to flip, and Google will rank your mobile and desktop sites based on signals they get from crawling your site from a mobile view.
So the page speed of your mobile site will determine the rankings of your mobile site and desktop site in Google. Google will also likely look at your title, H1s, structured data and other tags and content generated from your mobile site, and use them over your desktop site.
Doesn’t this just flip the issue the other way to where Google is ranking its desktop results based on how it sees your mobile site? Yes, but Google knows that, and the trend is that mobile keeps growing and more and more searchers will use mobile over desktop to search.
Will Google have different indexes for mobile and desktop?
Eventually, Google plans to have only one index, one which is based on mobile content, to serve listings for both mobile and desktop users. During this rollout period, there will be two: desktop-first and mobile-first. A smaller group of users will get results out of the mobile-first index. It’s not something that anyone will be able to control. People will likely have no idea which index they’re actually using.
As Google grows confidence in the mobile-first index, eventually that will be the only index used. Or if the new index isn’t deemed useful, Google could go back to a desktop-first index. It has, after all, called the mobile-first index an “experiment.”
Google said in their blog post, “Our search index will continue to be a single index of websites and apps, our algorithms will eventually primarily use the mobile version of a site’s content to rank pages from that site.”
Paul Haahr from Google reiterated it by saying, “Index of mobile pages for mobile users and index of desktop pages for desktop users won’t happen.”
Will links and rankings change because of this?
There is a concern that mobile content tends to have fewer links than desktop content. This is a concern that is similar to the concern listed above around mobile content having less content than desktop content. Google’s search results are very dependent on links and content. So if both links and content are impacted, will the rankings be impacted?
Google said they are still testing, so it isn’t 100 percent clear. Gary Illyes said, “I don’t want to say anything definite about links yet. It’s too early for that ‘cos things are very much in motion.”