Play Your Cards Right: Detecting Wildcard DNS Abuse

This post is also available in: 日本語 (Japanese)

Executive Summary

The domain name system (DNS) maps names to addresses so that computers can communicate. The directions within the DNS exist largely in records where a specific name (such as paloaltonetworks.com) is mapped to pieces of data, such as IP addresses (for example, 34.107.151[.]202). As the name suggests, wildcard DNS records are an exception to this pattern: They allow many domain names to be mapped to the same data.

Wildcard records facilitate DNS management in many constructive operations, for example, when a website owner is trying to direct users to an appropriate webpage if the users attempt to access a nonexistent subdomain. However, the flexibility of wildcard records also provides attackers with a variety of options for executing attacks with greater efficiency. Wildcard records allow attackers to easily direct users to malicious hosts via a nearly infinite number of domain names. This potential of wildcard DNS records has led attackers to deploy them for various purposes, including black hat search engine optimization (SEO), phishing campaigns and circumventing network protections. Distinguishing between domains using wildcard records for benign and malicious purposes poses a nontrivial challenge. Here, we describe some of the key characteristics of wildcard DNS abuse, and how recognizing them can help address this challenge.

Palo Alto Networks applies these principles in a wildcard DNS abuse detection system that efficiently flags domains that use wildcard DNS records for questionable or malicious activity. Our detections reveal multiple networks of domains involved in black hat SEO, the distribution of adult content or gambling services, and questionable video streaming.

The insights generated by this detector are available to Palo Alto Networks Next-Generation Firewall customers with security subscriptions, including DNS Security and Advanced URL Filtering.

Wildcard DNS Records

Before diving into the challenges of distinguishing between the good and bad of wildcard DNS usage, this section provides an overview of how wildcard DNS records work and of how they have been leveraged for both constructive and malicious purposes.

Wildcard DNS records allow authoritative DNS name servers to create responses to queries about domain names that do not technically exist within the DNS. As a somewhat simplified explanation: When domain registrants configure their authoritative DNS servers, the registrants give those servers a set of information about their domains. This information includes a list of hostnames and IP addresses where those hosts can be found. The list is called a zone file, and each hostname-address pair makes up a resource record. When an authoritative name server receives a query, the server will search through its zone file for records in which the hostname matches the name specified in the query and then send those records (there may be more than one) to the user. If no records match, the server will return a response indicating that the hostname does not exist. A wildcard record is the record that will provide matches for such nonexistent hostnames. As an example, imagine these are the records defined for example.com:

The *. at the beginning of the name in the last record in Figure 1 indicates this record is a wildcard. If a user sends a query for the IPv4 address of a subdomain of example[.]com other than www, the authoritative name server will use the wildcard record to generate a response telling the user that the IP address for that subdomain is 1.2.3[.]4. Figure 2 illustrates how this would work for the subdomain doesnotexist[.]example[.]com.

Wildcard records can simplify DNS administrators’ work by allowing them to specify entire groups of domain names that should all share the same resource, such as an IP address or mail server. Wildcard records also provide domain owners with an easy way to ensure that users will be directed to a helpful web page regardless of what subdomain is actually entered in the browser address bar. Some registrars highlight this capability to their customers. Others use wildcard DNS in conjunction with domain parking services to assess domain names’ values, or to advertise the availability of their domains (see Figure 3).

The features provided by wildcard DNS records make them an attractive option used by many popular domains. For example, 21 of the top 100 domains in the Tranco list of top sites use wildcard DNS records. Several of these domains are used by platforms that host user-generated content such as blogs or websites, and provide users with subdomains from which their content is served. Examples include GitHub pages and MyShopify. Some major search engines, such as Bing or Yandex, use wildcard DNS to redirect users either to the main search page or to an error page with additional links and suggestions. Even some top-level domains also use wildcard records.

The strengths of wildcard records also make them convenient tools for malicious parties. Researchers studying wildcard DNS record abuse have consistently found that a nontrivial percentage of domains using wildcard records were doing so to support activities such as blackhat SEO, or to evade attempts to block risky or questionable sites, such as those serving adult content or gambling sites. Others found that a substantial percentage of domains involved in phishing, spam or malware distribution used wildcard DNS records.

Detecting Wildcard DNS Misuse

Given that many services of all kinds use wildcard DNS records, the goal of detecting the abused wildcard records presents the challenge of finding their unique characteristics. This section describes observations from previous research into the characteristics of domains abusing wildcard DNS records and discusses our own approach and findings.

Characteristics of Domains Abusing Wildcard DNS Records

In the world of cybercrime, attackers often run large-scale campaigns or services relying on many domain names to direct users to malicious services or content. Attackers often register these domains in bulk, and these bulk registrations can be identified to provide hints that a domain is likely to be used for malicious purposes. Such hints may be particularly helpful for distinguishing between types of domains using wildcard DNS. In past studies, researchers noted that domains known to abuse wildcard DNS records were often registered in bulk, and a relatively high percentage used the same IP addresses or authoritative name servers. However, there are some challenges to using bulk registration as a key differentiator between benign and malicious use of wildcard DNS records. First, whois records, which provide registration information, often obscure the registrant for privacy reasons, making it difficult to identify bulk registrations. Second, high levels of concentration among domains using wildcard records do not always provide a strong indication of abuse. Some hosting or DNS management providers may provide wildcard records by default or encourage their users to configure their domains with wildcard records. The same providers may also provide infrastructure or authoritative DNS name servers to their clients. These scenarios could easily result in many benign domains with wildcards using the same name servers and IP addresses.

Domains used for malicious or suspicious activity are often only used for a short period of time. The longer the domain is part of an attacker’s operation, the greater the chance that it will be detected as malicious and blocked by security systems. Once the domain is flagged, attackers can no longer benefit from its use, and therefore must cycle it out for a newer domain. For wildcard domains, researchers noted that domains abusing wildcard records were generally considered “disposable,” and measured relatively short lifespans among domains used within the malicious campaigns they monitored. Thus, one differentiator between benign and malicious uses of wildcard DNS records may be the age of the domain.

Another key characteristic differentiating between domains using wildcard records for constructive purposes and those abusing the records is the rate at which the webpage content changes. This feature is particularly important for those using wildcards to support black hat SEO campaigns. In this scenario, attackers may deploy a strategy that involves serving dynamically generated content from large numbers of interlinked subdomains. The goal is to undermine web crawlers’ defenses against attacks attempting to keep them inside a single site. By trapping the crawler within the attacker’s websites for an extended period, the attacker can improve the rank of its domains. Wildcard records can support this strategy by allowing attackers to generate the subdomains in the links that connect pages without also needing to create corresponding DNS records for each subdomain.

Wildcard DNS Abuse Discovery

For our detection, we leverage a large passive DNS (pDNS) data set to effectively identify domains using wildcard DNS records and filter these domains based on key characteristics of the domains. Note from the example shown in Figure 2 that the response for doesnotexist[.]example[.]com generated from the wildcard does not show that the wildcard record exists. To figure this out, the user would have to ask the server directly for the IP address of *.example[.]com. Checking all domains for wildcard records is impractical, however. To efficiently search for malicious or suspicious domains, we use passively collected DNS data and hints from previously detected domains to regularly build lists of new domains to be checked.

Using information from whois records allows us to filter out many domains quickly. For the rest, we perform several checks, evaluating characteristics of these domains. The system builds its knowledge base as it runs, iteratively checking domains, and identifying related domains that also use wildcard records, thus allowing us to track entire campaigns using wildcard DNS records for less-than-honest purposes. In the weeks we have been running this detector, we have identified over 4,000 domains abusing wildcard DNS for questionable SEO campaigns, or to promote sites related to gambling, adult content or questionable video streaming sites. The next section explores a few of the cases we identified.

Real World Cases of Wildcard DNS Abuse

Case Study: Suspicious SEO

Website owners constantly vie for users’ attention. To get this attention, websites depend heavily on search engines, since users rely on these to find relevant content. To improve the chances that a search engine will return a particular website in the results for a given search, web designers can use SEO techniques to give search engine crawlers information about the content of a site and improve the site’s rank with the search engine. There are good and bad ways to do this. The bad ways aim to manipulate ranking without actually doing the hard work to provide meaningful content to users. These techniques include “keyword stuffing” (filling a page with words not necessarily relevant to the content of the page, but chosen to increase rankings), or building up networks of domains that link to each other solely for the purpose of increasing page rank, and automatically generating content for pages to hide the irrelevance or similarity of these pages from search engine algorithms designed to detect malicious SEO.

Several of the domains detected by our system show evidence of black hat SEO, including networks of domains evidently involved in the same campaign. These include multiple networks comprising dozens or thousands of domains with an identical layout serving a variety of articles with no coherent topic (see Figures 4a and 4b).

While these domains may not currently be used to actively deliver any malware, they also do not provide any meaningful content to users and are using tactics for promoting their sites that undermine effective search engine operations.

Case Study: Gambling Redirect

Several hundred of the domains identified by our wildcard abuse detector contain a script redirecting users to a site with content related to gambling. For these domains, wildcard DNS records can be a tool to help circumvent censorship. Domain operators can generate various subdomains in order to circumvent some approaches to blocklisting. Many of the gambling domains are presented in Chinese, suggesting their main target audience is inside China. As gambling is illegal in China, evading blocklists would be a priority for these operators. One cluster of domains featured a benign-looking landing page, with a popup offering a monetary reward for new customers (see Figure 5a). Following the link or trying to close the box leads to the gambling site (see Figure 5b).

Case Study: Suspicious Video Streaming

Another group of several dozen domains was used for some questionable sites providing video streaming services. Streaming or downloading licensed content is illegal in many contexts. Even apart from these issues, sites providing video streaming services also commonly provide viruses along with their services, making these sites questionable at best.

Conclusion

We highlighted the importance of investigating wildcard DNS usage and detecting the abuse of these records. Wildcard DNS records have legitimate use, but are also a valuable tool for miscreants executing a variety of serious attacks. If interpreted carefully, the appearance of wildcards in a domain’s DNS records provides a hint that the domain may be used for malicious purposes. Our detector has found thousands of such domains abusing wildcard DNS records.

Palo Alto Networks detects domains abusing wildcard DNS records and assigns them to the grayware category through our security subscriptions for Next-Generation Firewalls. These subscriptions include DNS Security and Advanced URL Filtering. Through this detector, we protect our customers from risks associated with the types of domains discussed above.

Additional Resources

• Dangling Domains: Security Threats, Detection and Prevalence
• DNS Rebinding Attack: How Malicious Websites Exploit Private Networks
• Detecting and Preventing Malicious Domains Proactively with DNS Security
• Fast Flux 101: How Cybercriminals Improve the Resilience of Their Infrastructure to Evade Detection and Law Enforcement Takedowns
• The History of DNS Vulnerabilities and the Cloud