Search engine optimization (SEO) is about improving visibility in organic search results. The goal of search engine optimization is therefore to be found more easily in the results of search engines such as Google, Bing, and Co. for relevant search terms and to be clicked on more often – apart from the paid places for search engine advertising.
The entire article focuses on the search engine Google. This is okay for two reasons:
1. Google has by far the largest market share in Germany (over 90 percent).
2. Other search engines basically work the same way. Exact details about the algorithms are known – neither for Google nor for comparable search engines.
Crawling
Crawlers or spiders are bots that follow hyperlinks on the World Wide Web and thus literally open up a network of connected online resources. This so-called crawling is also carried out by search engine bots such as Googlebot in order to find new or updated pages on the Internet. If necessary, these pages are then added to the search engine index (indexing, see below).
The crawling of the respective domain can be controlled by bots with the robots.txt file. This can be stored as a text file under the path /robots.txt (i.e.: https://www.example.com/robots.txt) for a subdomain and contains instructions in a single line.
These are structured as follows:
- User-Agent: This allows making instructions for certain user agents. E.g. “User-agent: Googlebot” for instructions that should only apply to Google’s crawler.
- Disallow: Denies crawler’s access to certain website areas or individual URLs.
- Allow: Allows crawlers to access specific website areas or individual URLs. If in doubt, overwrites a previous disallow statement.
- Sitemap: This shows the address of the sitemap(s) or the sitemap index file.
Why would you even want to exclude sections of a website from crawling?
In practice, many websites generate URLs that offer no real value or are simply not intended to be found in organic search results (such as landing pages for other online marketing channels).
For large websites, this can lead to the reach of a crawl budget limit – i.e. the maximum number of URLs that Googlebot can access in a certain period of time. To concentrate this capacity on the most important and to speed up the crawling and indexing of new pages, you should block irrelevant content from search engines.
Tooltip
The Google Search Console can track which URLs were crawled if Google can read and understand the robots.txt file and if Googlebot can access individual URLs for crawling. If you want to crawl a website yourself, you can use SEO spiders like Screaming Frog.
Indexing
Websites that Google has crawled and found relevant are included in the search engine index. The respective content is described in the index entry and the URL is given.
The quickest way to find out whether a particular URL is in the Google index is to do a Google search for the site: URL. If there is no search result here, the URL is not included in the search engine’s index.
A canonical tag pointing to a non-original URL also prevents indexing by search engines. In the case of different reachable URLs that have the same content, this tag is the better choice.
Why should you even exclude website areas from indexing?
In some cases, URLs must not be accessible via the results of search engines (e.g. for data protection reasons). Or there is no good URL criterion to exclude certain page types from crawling via the robots.txt file.
Yet, it is possible to automatically provide them with a RobotsMeta-Noindex-Tag as an alternative to excluding them from crawling. This would only be recommended if the crawl budget is not important (see above).
Tooltip
The Google Search Console also helps with checking and monitoring indexing. It also provides information about which URLs were crawled but not indexed – and for what reason.
Ranking
Content
The primary task of a search engine is to find content that matches the search query. For example, if the search query is “panda bear t-shirt,” users will be as dissatisfied with panda bedding as they will be with t-shirts printed with cats. As a website owner or SEO expert, you should signal the search engine as comprehensively as possible on what info your website can provide.
And if you want to rank at the top of Google for searches like “can I have a pet panda?” you need to offer the best content—nothing more, nothing less.
Backlinks
Another important ranking signal is links that point to a domain or its subpages from other domains. It’s not just about the sheer number, but above all about the quality and thematic relevance of the backlinks.
So the most valuable links are from trusted websites with authority on the topic (more on this under Offpage SEO, see below).
Page Experience
Various factors are important to ensure that the user experience is as pleasant as possible on all devices. The loading time (page speed), the suitability for mobile use, and the security of the website should be mentioned in particular.
On-page SEO
Keywords, Keywords, Keywords
Search engine users start their search by entering a search query. As the person responsible for SEO, you want to optimize the appropriate page on a domain for these search queries. For this, you use the keyword in headlines, in texts, in the page title, and in the meta description. Studies show that search queries often consist of three or more words.
Pages should therefore not only be optimized for one main keyword but cover a large number of keywords. In addition, alternative terms and questions can be processed in texts.
Keywords are searched for with different frequencies. Those that have a high search volume are also called head keywords. There is a lower search volume for very specific search terms and longer phrases. They are known as long-tail keywords. Long-tail keywords are particularly exciting because of the less competition and higher conversion due to the precise search query.
Keyword research is therefore not just about the search volume, i.e. the question of how often a certain term is entered in search engines. Search intent is crucial. Why did the user make the search query? What has to happen after clicking on a search result so that the user is satisfied?
Basically, keywords can be divided into three categories:
- Informational: The user wants to know something, answer a question, or look up a fact.
- Transactional: The user wants to do something, e.g. buy, request, book, etc.
- Navigational: The user searches for a specific website or address.
Keyword density is how often a keyword is mentioned in a text in relation to the length of the text. With a keyword density of 3 percent, the keyword would have to appear three times in 100 words.
The WDF*IDF formula indicates the ratio of keywords in a text or on a website compared to other documents, such as the top 10 search results. Based on this, WDF*IDF tools determine which keywords should also appear in the text.
The results of these tools are not perfect. Ideally, you should use common sense to treat them as suggestions and not as strict guidelines for SEO copy.
Tooltip
Most keyword research begins in the Google Ads Keyword Planner. Here you can see how often certain keywords are searched for. In addition, you can also get ideas for related keywords and phrases with the entered keyword.
SERP optimization
Now that you have achieved a ranking on the search engine results page (SERP), there is only one step left to attract visitors to the website: the click.
A number of measures can be taken to increase the click rate, i.e. the proportion of searchers who not only see the search result but also click on it. To do this, you first need to know how individual search results are structured.
The page title (HTML =<title>) is one of the most important ranking criteria and should therefore contain the most important keyword. It should be 50 to 60 characters long. It is shown as a blue heading.
A good meta description also catches the eye as a gray snippet of text underneath (HTML = <meta name=”description”>). Even if the keywords used here are not important for the ranking, they are displayed in bold and indicate to the user that the search result matches the query.
Arguments should also be included here as to why exactly this search result is the best. Special characters such as ticks, hearts, or emojis can increase visibility, but they should only be used in moderation. The meta description should be 130 to 150 characters long. Both page title and meta description should be unique, i.e. different for each URL.
In the meantime, Google has somewhat contained the problem of spam links through algorithm updates and human quality checkers. Anyone who violates Google’s rules of the game runs the risk of losing all visibility in the search results through a penalty.
So if you want to build links that are safe in the long run and help a website achieve good search engine rankings, you should first know Google’s guidelines.
The most important prohibitions on backlinks in it are as follows:
- Buying or selling links (including ads, banners, native advertising, sponsored posts, advertorials, etc.)
- Excessive link exchange
- Article marketing at scale (guest articles with keyword links)
- Automated programs and services that create links to a website
- Press releases with links that contain search terms in the link text
- Links from low-quality directories or bookmarking sites
- Links from widgets distributed on other websites
- Links from the footers of various websites
- Comments in forums or under blog posts with links in the post or signature
- Hidden Links
It is always explicitly about backlinks that pass on PageRank. Of course, Google does not ban all advertorials, banner ads, widgets, etc. that contain a link – however, these links must be marked with the no-follow attribute in order not to violate Google’s guidelines. This tag signals search engines that a specific link (or all outgoing links of a URL) should be ignored as a ranking signal.
Anchor texts
The anchor text (also link text) is the text that is displayed as a link for users to see. Anchor texts from incoming links are an important signal for the search engine algorithm as to what can be found on the respective website. However, nowadays you have to be careful that all of the backlinks (the so-called link profile) do not contain any patterns that indicate manipulation – such as a conspicuous number of links with keywords in the anchor text (so-called keyword links). Longer anchor texts made up of phrases that also contain the name of the website, for example, look more natural.
Internal linking
Internal linking means links from one URL to another URL on the same domain. These internal links can give search engines a strong signal as to which subpages of a website are important for which topics.
For example: if your website has a page with panda t-shirts, you show search engines the way to it with internal links. If those links also have a meaningful anchor text, so much the better. In the case of internal links, keyword links may be set without inhibitions.
Tooltip
Link databases like Majestic, Moz, Ahrefs, etc. provide insights into the backlinks of other websites. They build their own web indexes by mimicking the behavior of Googlebot and following any links they find on the web. As many links as Google don’t know of any of these tools, and they most likely rank those links differently than major search engines do, too.
Link databases can be used well, especially for analyzing the competition: who is already linking to your competitors? Can you learn anything from the link strategies of other companies? You can also test the quality of a linking website when building the link: how well is the website linked? Is it worth asking there for a link? How much would such a link be worth in case of doubt (although buying a link is of course forbidden in theory, see above)?
More SEO topics
In this article, you now have learned the most important basics of search engine optimization. Of course, the subject of SEO is much more complicated – there is a lack of literature about it. I’d recommend the following list:
• International SEO: for websites that need to be found in other countries and in other languages. Book: International SEO: The compact guide to hreflang, domain strategies, tools + more by Markus Hövener.
• Local SEO: for websites that are to be found locally, ie with physical addresses as points of sale or for services (branches, restaurants, doctors, gyms, etc.). Reading tip: Local SEO: Basics & Tips for Local Search.
• Image SEO: E-Book Image SEO 2018 by Martin Missfeldt.
Conclusion
The complex field of search engine optimization is evolving with the demands of users and search engines on modern websites. Investing in the long-term SEO strategy and website performance usually pays off – even if it requires expert knowledge and hard work.
Sources:
- Search Engine Market Share Germany Nov 2020 – Nov 2021
- Schmidt: Listing Google’s 200 Ranking Factors Would Reveal Business Secrets
- The State of Searcher Behavior Revealed Through 23 Remarkable Statistics
- Top Bing Searches (2021)
- Structured Data: Markup Help
- Webmaster Guidelines
- Limit outbound links for Google