Having a website and not monitoring its performance in today’s competitive landscape is simply not smart. With tools like Google Analytics and Adobe Creative Cloud for tracking and reporting data, businesses can learn a lot from the behavior of those that visit their site. This type of analytical data can be the contributing factor between success and failure of your brand.
So what happens when your site isn’t performing well and you’re not sure why? There could be underlying issues plaguing your site that you can’t see. Things that analytical data simply doesn’t look for.
There are a number of tools that can provide insight into the health of your site, and most of them are free. The first tool that can help to identify issues with linking within your site is a crawler. A crawler is a “spider” or “robot” that literally spiders through every page of your site. It can return data like title, description, heading tags and more. But in this case, we would be using the crawler to identify broken links.
You can find a list of crawlers on Google’s Code site. Once you’ve chosen one, getting started is typically very straight forward. In most cases you simply enter your site’s home page URL to start the crawl. Most of the crawlers will look for pages that return status codes. A good status code is 200 OK, a bad code is 404 Not Found. There are other codes out there but in this case, it’s these two that we’d like to focus on.
So what happens if you find links that returned a 404 Not Found status? If the crawler you chose provides an export option, you should export the data so that it is easier to work with and then do some legwork to fix the problems. You may need to log into your CMS if you have one to update the links manually, or reach out to your webmaster to do them for you. Fixing the issues, in my opinion, is always better than having content on your website that leads nowhere.
The second and third tools are almost one in the same: Google Webmaster Tools and Bing Webmaster Tools. These two platforms are extremely important, and while they provide almost the same types of data, there are some subtle and major differences.
Both of these tools are standalone tools and do not require the use of Google Analytics. Once signed up, you’ll need to verify your site in one of multiple ways. If you have a CMS, it’s simple enough to verify using the meta tag process as most CMS’ provide a module for this. If you have a static site and require a webmaster to do this, you can download a verification file for them to upload to your server.
Once verified, of the many tools within these dashboards, the Crawl Errors report should be one of the reports you monitor on a regular basis. The reason for the scrutiny? While there are links that can be broken on your site and found with a crawler, there are links that may exist in search engines that were not generated by you.
A visitor of your site may have talked about one of your products on a social media site like Facebook or Twitter. They may have shared that link because they loved your product. If you inadvertently deleted this page from your website without knowing the viral nature of page, this could result in many people arriving to this page only to find a hopefully custom 404 Page Not Found page. Now that you have the tools to monitor the health of your site, you can identify broken links both with the manual crawler as well as with a Webmaster Tools account. Once you have this information, you can decide to provide a 301 Redirect to a similar page or topic ensuring your visitors have good user experience when they visit, or choose to leave it as 404 Page Not Found if the content is simply gone for good.