Website Structure. Do you know your website well?
Articles,  Blog

Website Structure. Do you know your website well?


Hi, everyone! My name is Serge Bezborodov Hi, everyone! My name is Serge Bezborodov I am CEO and co-founder of JetOctopus crawler Today I want to start a series of short but informative videos about FAQ in SEO Today I want to start a series of short but informative videos about FAQ in SEO Today I want to start a series of short but informative videos about FAQ in SEO I don’t want to record long webinars with a bunch of data, preparing for them for half a year I don’t want to record long webinars with a bunch of data, preparing for them for half a year In fact, I know that most of you have no time for exploring these webinars My idea is to create meaningful videos which you will be able to watch in the morning while drinking a cup of coffee or tea And in these videos I want to cover quite widespread themes in SEO And in these videos I want to cover quite widespread themes in SEO everyone talks about But are not as simple as it may seems at the first glance But are not as simple as it may seems at the first glance And the first topic of our Technical Tips videos is the Website Structure The Website Structure: theory and practice Website Structure is one of the most frequently- mentioned expression in the On-page SEO field Website Structure is one of the most frequently- mentioned expression in the On-page SEO field Website Structure is one of the most frequently- mentioned expression in the On-page SEO field And it seems clear and understandable enough And it seems clear and understandable enough But today I want to show you the Website Structure from the other side The most simple picture we see while talking about the website structure is that The most simple picture we see while talking about the website structure is that that we have an equal distribution of webpages From the main page, then there are pages with the first DFI, the second DFI, and so on.
* Learn more about DFI – link is below the video* From the main page, then there are pages with the first DFI, the second DFI, and so on.
* Learn more about DFI – link is below the video* And pages are growing gradually… This is a tree-like structure For instance the 0 page – is the main page, 1-st is the catalog, 2-nd is a subcategory For instance the 0 page – is the main page, 1-st is the catalog, 2-nd is a subcategory And pages with 3-rd DFI are product pages All is simple and understandable Everyone thinks his website looks like this Beautiful, logic both for users and Googlebot But the reality is different Long DFI is the problem from One in Ten websites unless for 99 in 100 websites Webmasters usually don’t consider that t structure thoroughly And that results in long tails of webpages And that results in long tails of webpages For instance, there can be tail of pages after 5-th, 6-th level that are away from the main page in 30-50 clicks For instance, there can be tail of pages after 5-th, 6-th level that are away from the main page in 30-50 clicks And that increase the DFI dramatically Why Distance From Index is so important? We will talk about the data from the log files in our next videos I mean our experience of Yandex, Google visits logs analyzes I mean our experience of Yandex, Google visits logs analyzes And there we see that DFI is the most important factor that influences on bot’s crawling And there we see that DFI is the most important factor that influences on bot’s crawling But now I just ask you to imagine how Google will discover your webpages when you launch a new website? Firstly, it will discover the main page Then it will process links on the main page and discover pages on the next level Further and further deep into the structure When it gets, for instance, on the 15th level There is a high risk that it won’t take these distant pages to the index There is a high risk that it won’t take these distant pages to the index There is a high risk that it won’t take these distant pages to the index There is a high risk that it won’t take these distant pages to the index In fact, it’s logical that if these pages are so deep in your website structure, user won’t probably find these pages And Googlebot thinks you don’t treat these pages as important Technical factors show Googlebot that these pages are very deep Furthermore, the structure becomes more complicated when we add pages from Noindex Let’s imagine that Bot goes to the 4th level Let’s imagine that Bot goes to the 4th level And there are webpages that are blocked for indexation For example, with robots.txt or nofollow And at the same time, we can see indexable pages that could be reached only through the links from non-indexable pages And, roughly speaking, using standard ways, I mean without sitemaps or external links, And, roughly speaking, using standard ways, I mean without sitemaps or external links, bot won’t ever discover these pages And that is the reason why your pages aren’t crawled and aren’t in the index And that is the reason why your pages aren’t crawled and aren’t in the index But we can always add the sitemap files And very often the sitemap is like the second part of the website And very often the sitemap is like the second part of the website So, Googlebot can discover your pages by processing the links from the main page And also it can analyze sitemap files with pages you want to show it The part of pages from sitemap will be also accessible in the website structure (in the left part) The other problem is that there are pages that are separated, and these pages are only in the sitemap. Googlebot will never find these pages while crawling from the main page We call these pages a “grey zone” And the grey zone in a number of cases has problems with bot’s crawl The same way non-indexable pages appear in a sitemap and other trash But where a website has a long history, the third part of the pages appears, so-called Orphan Pages, which neither in the structure and nor in the sitemap, but these pages exist Where you can find these pages? The answer is in log files A bunch of websites contains Orphan Pages We call them Grey or Pink zone So, these are pages which aren’t in the website structure they aren’t in the sitemap, there are neither internal links, nothing but in the past, maybe there were some internal links on these pages Maybe there are some backlinks links on them And bot visits these pages When, for instance, you crawl your website with Screaming Frog or with other crawlers, You think your website has a clear structure with a sitemap, everything is OK But there is the other reality – There are also some pages that appeared in the past and these pages should be checked very attentively Sometimes there are some trash pages, sometimes there are some profitable pages And you should clearly understand what to do with these pages in the future And you should clearly understand what to do with these pages in the future To sum up: the structure is the layout of the website To sum up: 1) the structure is the layout of the website And you should implement the technical tasks, ideas, experiments basing on what is the structure of your website basing on what is the structure of your website what internal factors are there and what to do with that 2) Monitor your website structure For instance, developers can break internal linking widget For instance, developers can break internal linking widget And one-third part of your pages can become orphaned And one-third part of your pages can become orphaned And that influence on Googlebot’s behavior dramatically How is your website crawled? How is it perceived by bots? How many pages are accessible for bots? The answers are in the website structure – сement your layout well In the next videos, we will talk about other wide-spread technical topics For instance, about load time, Why does it matter? What are the kinds of load time We will definitely talk a lot about the log files What information we can get from logs What decisions can be taken, what experience can be done we will also talk about cloudflare what is it, how to use it, and what cons it has and about other useful tips for SEO uplift So, subscribe on this channel. like and share this video Ask any questions in comments – I will be glad to chat with you Thank you! Bye-bye!

Leave a Reply

Your email address will not be published. Required fields are marked *