Screaming Frog: Custom Search Feature

We introduced the Screaming Frog application and its basic functionality in our Screaming Frog Simplified blog post. This time, we want to show you how to enhance your search capabilities using its Custom Search feature. The Custom Search option can be found in the main menu under Configuration > Custom:

 

Suppose we want to find any mention of Screaming Frog on our website, hallme.com. Selecting the Search option mentioned above will bring us to an empty Custom Search window:

Empty Custom Search window on the Screaming Frog application.

 

Click Add to create the first Filter, exemplified below:

 

In the Enter Regular Expression field, we will enter our query string which is “Screaming Frog” and click OK.

Note: custom search isn’t case sensitive. Searching for “Screaming Frog,” “screaming frog,” or “Screaming frog” will return the same results.

Query string "Screaming frog" typed into the Enter Regular Expression field.

 

Then we will enter “hallme.com” in the Enter URL to spider field and start the crawl:

hallme.com typed into the Enter URL to spider field

 

Now, we need to switch to the Custom tab to see the results from running custom procedures. To find the tab, we will click a drop-down arrow in the tab row:

Drop down arrow for navigating in the Screaming Frog application.

 

Then, click on Custom Search in the drop-down list:

Custom Search in drop down list in the Screaming Frog application.

 

Once we have switched to the Custom Search tab, we need to change the Filter from All to Contains: Screaming Frog:

Filter drop down selecting Contains: Screaming Frog.

 

Our search for “Screaming Frog” on hallme.com returned 12 URLs. Columns contained in custom search are Address, Content, Status Code, Status, and Occurrences. URL #10 (https://www.hallme.com/blog/screaming-frog-simplified/) returned the largest number of occurrences, 32, because it is our previous blog post on Screaming Frog. Other pages with a much lower number of occurrences contain links to that blog post:

URL occurrences listed from the site crawl.

 

Let’s look at URL #11 (https://www.hallme.com/blog/page/5/) which has six occurrences. To find the occurrences, we will search through the content rendered on the front end of the page and then search through the page’s source code. Our search on the front end returned three occurrences. One occurrence is in the name, “Screaming Frog Simplified,” and two occurrences are in the abstract:

Occurrences of "screaming frog" on the front end of a URL from hallme.com

 

Searching through the source code, we find three additional occurrences. There are three anchor elements with title attributes that contain “Screaming Frog”:

Occurrences of "screaming frog" in the source code of a URL from hallme.com.

Searching for Tracking Code

A common use of the custom search feature is to find which pages contain Google Analytics, Google Tag Manager code, or any other tracking code. There are two options in the Filter drop-down: Contains and Does Not Contain. We will search for GTM, but you can enter a specific tracking number for your own search:

Tracking code filter searching for Google Tag Manager code.

 

This filter shows pages that contain the code:

 

Switch between filters to see pages that contain and do not contain tracking code.

 

This filter will show pages that do not contain the code:

Saving Custom Search Results

You can save your custom search results. You can create a CSV/Excel file or create a Screaming Frog file. To save results as CSV/Excel, use the Export button to the right from the Filter drop-down:

 

To create a Screaming Frog file, click File > Save:

It will create a Screaming Frog file with the extension “.seospider.” When clicked, the Screaming Frog app will start and you will see it populated with your custom search data.