WatchDog: Exposing a Cryptojacking Campaign That’s Operated for Two Years

This post is also available in: 日本語 (Japanese)

Executive Summary

Unit 42 researchers are exposing one of the largest and longest-lasting Monero cryptojacking operations known to exist. The operation is called WatchDog, taken from the name of a Linux daemon called watchdogd. The WatchDog mining operation has been running since Jan. 27, 2019, and has collected at least 209 Monero (XMR), valued to be around $32,056 USD. Researchers have determined that at least 476 compromised systems, composed primarily of Windows and NIX cloud instances, have been performing mining operations at any one time for over two years.

Cryptojacking is the process of performing cryptomining operations on systems which are not owned and maintained by the mining operators. Malicious cryptojacking operations are currently estimated to affect 23% of cloud environments, up from 8% in 2018. This increase is primarily caused by the meteoric rise in cryptocurrencies’ valuation. The global market for blockchain, the technology behind cryptocurrency, is anticipated to reach $60.7 billion by 2024, and criminal organizations and actor groups are trying to cash in on this.

Within this blog, Unit 42 researchers provide an overview of the WatchDog cryptojacking campaign. The WatchDog miner is composed of a three-part Go Language binary set and a bash or PowerShell script file. The binaries perform specific functionality, one of which emulates the Linux watchdogd daemon functionality by ensuring that the mining process does not hang, overload or terminate unexpectedly. The second Go binary downloads a configurable list of IP addresses net ranges before providing the functionality of targeted exploitation operations of identified NIX or Windows systems discovered during the scanning operation. Finally, the third Go binary script will initiate a mining operation on either Windows or NIX operating systems (OS) using custom configurations from the initiated bash or PowerShell script. WatchDog’s usage of Go binaries allows it to perform the stated operations across different operating systems using the same binaries, i.e. Windows and NIX, as long as the Go Language platform is installed on the target system.

Researchers have mapped out the infrastructure behind the mining operations. They have identified 18 root IP endpoints and seven malicious domains, which serve at least 125 malicious URL addresses used to download its toolset.

Unit 42 reported on Graboid, a wormable Monero mining operation on Docker Hub, in October 2019. Graboid was the largest known mining operation to date in terms of the total number of active systems. At the time of its operation, it consisted of at least 2,000 exposed and compromised Docker Daemon APIs systems. Each Graboid miner was operational 65% of the time, meaning around 1,300 compromised Docker containers were mining at any one time. Additionally, Graboid could have also achieved higher processing speeds due to the configuration script utilizing all available container central processing units (CPUs). However, Graboid was only known to operate for up to three months before its Docker Hub images were removed.

WatchDog, on the other hand, does not rely on a third-party site to host its malicious payload, allowing it to have remained active for more than two years at the time of this writing.

It is clear that the WatchDog operators are skilled coders and have enjoyed a relative lack of attention regarding their mining operations. While there is currently no indication of additional cloud compromising activity at present (i.e. the capturing of cloud platform identity and access management (IAM) credentials, access ID or keys), there could be potential for further cloud account compromise. It is highly likely these actors could find IAM-related information on the cloud systems they have already compromised, due to the root and administrative access acquired during the implantation of their cryptojacking software.

Palo Alto Networks Prisma Access is configured to detect each of WatchDog’s 18 IP addresses, seven domains and their associated URL addresses through PAN-OS. Prisma Cloud also detects the usage of malicious XMRig processes used by the WatchDog miner operating in cloud environments that have Prisma Cloud Compute Defender installed.

Public Mining Pools

Unit 42 researchers have identified three XMR wallet addresses within WatchDog configuration files. These configuration files are downloaded alongside the WatchDog mining binaries and contain the XMR wallet address and the mining pool(s) to be used during the mining operations. See Figure 1 for an example of the configuration file config.json.

Examining all known config.json files used by WatchDog, Unit 42 researchers have identified three XMR wallet addresses as:

43zqYTWj1JG1H1idZFQWwJZLTos3hbJ5iR3tJpEtwEi43UBbzPeaQxCRysdjYTtdc8aHao7csiWa5BTP9PfNYzyfSbbrwoR

82etS8QzVhqdiL6LMbb85BdEC3KgJeRGT3X1F3DQBnJa2tzgBJ54bn4aNDjuWDtpygBsRqcfGRK4gbbw3xUy3oJv7TwpUG4

87q6aU1M9xmQ5p3wh8Jzst5mcFfDzKEuuDjV6u7Q7UDnAXJR7FLeQH2UYFzhQatde2WHuZ9LbxRsf3PGA8gpnGXL3G7iWMv

These three XMR wallet addresses are used with at least three public mining pools and one private mining pool to process mining operations, performance, functionality and payments.

^{Table 1. Public and private mining pools used by the WatchDog miner.}

The following eight screenshots illustrate the findings gathered from the f2pool, nanopool and the GNTL public mining pools for each of the three XMR wallets identified.

f2pool mining pool

Figures 2 and 3 illustrate the XMR address beginning with “43zq” being heavily used within the f2pool public mining pool, pulling in roughly 200 Monero. Meanwhile, the XMR wallet address starting with “82et” operates at a much lower scale and has only pulled in 2.3 XMR (see Figures 4 and 5).

Nanopool mining pool

XMR address beginning with “82et” was less active within the f2pool public mining pool, but it is more involved within the nanopool public mining pool (see Figures 6 and 7) than the XMR wallet address beginning with “43zq” (see Figures 8 and 9). However, the nanopool mining operation only equates to a fraction of the total XMR mined by the WatchDog mining operation as a whole, with 6.8 XMR coins mined to date.

GNTL XMR mining pool

A single configuration file has been identified that links the potential of the wallet that begins with “87qa” to all three public mining pools listed here, but only GNTL displayed any mining operations related to the “87qa” XMR wallet (see Figure 10). However, this XMR wallet address does not seem to be greatly used within the WatchDog operations. As of this writing, only .59 XMR has been mined from GNTL using the “87qa” XMR address (see Figures 10 and 11).

Out of the data collected from all three of the public mining pools, Unit 42 researchers calculated an average of 1,037KH/s hash rate from the XMR wallets across the public mining pools. Researchers then developed an estimate of the current number of systems actively participating in the cryptomining operation. Researchers conservatively estimate that an average of 476 systems are actively involved within the WatchDog mining operation at any one time.

This estimation was calculated using the documentation on CPU architecture from several of the largest cloud providers. All cloud providers advertise the use of Intel Xeon E5 and AMD EPYC CPUs for a majority of their cloud VM instances.

We can use the popular XMR mining software XMRig’s benchmark hash calculator to calculate the hash rates for mid-range Intel Xeon E5 and AMD EPYC series 7 processors. A single thread on each processor can produce an estimated hash rate of 543 H/s (hashes per second) for the AMD EPYC series 7 and 544 H/s for the Intel Xeon E5. When taking into account the WatchDog miner configuration file, config.json, the miner will use at most four threads on the compromised system (see Figure 12).

This will result in the compromised system processing a total average of 2,172-2,176 H/s using at most four threads as per the configuration guide. With an average total of 1,037 KH/s (thousand hashes per second) of processing for the total WatchDog miner operation, this leaves a potential total of 476 systems participating in the mining operation at any one time.

The number of systems would depend upon the VM instance type that was compromised and used. It is important to note that not every compromised system would be able to process XMRig operations to the same scale. It is possible that double this estimated number, nearly 900 systems, could be operating at any one time. This size of a mining operation is achievable if smaller, less robust, cloud VM instances were compromised and used to process XMR hashes.

WatchDog Infrastructure

The WatchDog miner has been active since at least Jan. 27, 2019, as witnessed from the public mining pool data. Since that time, a number of malware samples have been identified that point to WatchDog infrastructure, specifically the initialization bash script that begins the system and mining configuration process for newly compromised systems.

Through analysis of these initialization bash scripts, Unit 42 researchers were able to track how WatchDog actors set up mining operations on a compromised system. The authors of the script tipped their hand to show how they set up and configure their mining infrastructure. Within every known operation, the initialization bash script is downloaded onto the compromised system and performs a series of functions. Several of the functions are common to a majority of cryptojacking operations, namely the removal of cloud security tools, the removal of previously installed and known malicious cryptomining software, and then the downloading and setup of the customized malicious cryptomining software. However, the WatchDog bash script miner also hardcodes a primary and secondary URL address that are used to download the WatchDog mining toolkit (see Figure 13).

Using these primary and secondary URL addresses, Unit 42 researchers were able to map a rough estimation of the network infrastructure used by the WatchDog miner operators.

The following Maltego chart illustrates the overall size of the known operation infrastructure used by WatchDog (see Figure 14).

To date, there are currently 18 known IP addresses and seven known domains hosting at least 125 URLs that have served or continue to serve the WatchDog miner malware and configuration files. While the majority of the malware appears to be focused on *NIX OS systems, there are several Windows OS binaries that are also hosted on several of the known host systems.

^{Table 2. The 18 known IP addresses associated with the WatchDog miner.}

^{Table 3. The seven known domains associated with the WatchDog miner.}

For a full list of the known URL Addresses associated with the WatchDog mining operation, see the Indicators of Compromise (IOC) section of this blog.

Researchers found that several of these host systems were still operational at the time the research for this blog was being conducted. Due to the live status, researchers were able to pull down several of the malicious files for further analysis. A full IOC breakdown of the files and their SHA-256 hash is listed below within the IOC section.

WatchDog Malware Breakdown

Unit 42 researchers selected five interrelated malware samples to explain their functionality. The cryptojacking operation appears to begin with a bash script, newdat.sh, which defines the downloadable content for three separate Go binary files and one JSON configuration file config.json. The Go binaries detailed within this blog are a network scanner and exploitation binary called networkmanager, a process monitoring binary called phpguard, and a version of the malicious XMRig cryptomining software called phpupdate.

newdat.sh

Unit 42 researchers have identified four different filenames for bash scripts that perform the same infrastructure, network scanning and system configuration operations. These file names are init.sh, newinit.sh, newdat.sh and update.sh.

There are eight unique operations within the initialization script:

• Environmental setup
  • Configure file and directory read/write permissions and save downloaded files to preconfigured locations.
• Uninstallation of cloud security tools
  • Namely Alibaba Cloud Security Center and Tencent Cloud Security Operations Center.
  • This is a common operation used by several cryptojacking operations including groups such as Rocke and TeamTnT.
• Download toolkit
  • Download three Go Binaries and a configuration file.
• kill_miner_proc
  • Killing known mining processes.
• kill_sus_proc
  • Killing previously installed WatchDog mining processes.
• downloads
  • Downloads IP address ranges to be used for scanning.
• unlock_cron
  • Unlocks the /etc/crontab file.
• lock_cron
  • Locks the /etc/crontab file.

Perhaps one of the most useful script operations identified by Unit 42 researchers is the section pertaining to the download location of the WatchDog toolkit. As illustrated within the previous infrastructure section, the scripts detailed which endpoints are currently hosting the malicious cryptojacking files.

As can be seen in Figure 15, there are hardcoded links within the newdat.sh script that point to URL addresses and identify the miner binary, the configuration files, the scanning binary and the WatchDog process – and even another version of the initial script itself. This could allow the actors to update the active miners in near realtime.

Each of these binaries will be investigated in the following sections. First up is the Go language scanning binary, networkmanager.

networkmanager

The networkmanager binary is a UPX-compressed Go language binary designed to scan networks and, when a vulnerable target is identified, attempt to compromise that identified system using a robust set of built-in application exploits. Researchers have identified two different file names used by the actors to name their binaries that perform the same scanning and exploitation function. Those names are networkmanager and networkservice.

While scanning operations are initiated via the newdat.sh bash script detailed above, the scanner binary will perform the actual scanning and exploitation operations. The WatchDog scanning binary uses a file composed of 60,634 individual Chinese IP net ranges, which is downloaded during the system detection phase of the networkmanager binary. Within the Go binary’s main initialization function, sym.go.main.ipc.download_ipdb, the networkmanager binary requests and then downloads one of two possible IP address netrange files:

http://83.97.20[.]90/cccf67356/ip_cn.txt

http://83.97.20[.]90/cccf67356/ips_cn.txt

The IP address net ranges were stored in binary format and upon conversion to ASCII revealed the targeted IP address net ranges (see Figure 16).

Unit 42 researchers downloaded these files, and at the time of their download, both of the files appear to contain the same content, as they both have the same SHA-256 hash, ad3efb9bfd49c379a002532f43cc4867a4f0b1cd52b6f438bb7a8feb8833b8f8. These two identical files will be used by the pnscan or masscan processes to scan the network ranges for potential victims.

At the time of their download for this blog, these two files only appear to contain Chinese-related IP addresses. It is likely the actors behind WatchDog are able to update the binaries to include any number of IP address network ranges they wish to target. This is likely the case as Unit 42 researchers have identified victims of the WatchDog miner operating outside of the China IP address space, specifically within the United States and Europe.

Continuing on, loaded within the networkmanager Go binary are 33 individual exploits functions, 32 individual remote code execution (RCE) functions and several shell grab functions (see Figure 17).

The following applications are specifically targeted within the scanning binary:

• CCTV exploit
  • It is currently unknown if the target is a CCTV appliance or if there is another moniker “cctv” could stand for.
• Drupal
  • Versions 7 and 8.
• Elasticsearch
  • CVE-2015-1427 (Elasticsearch sandbox evasion – version before 1.3.8 and 1.4.x before 1.4.3)
  • CVE-2014-3120 (Elasticsearch before 1.2)
• Apache Hadoop
• PowerShell
  • Encoded command-line operations.
• Redis
• Spring Data Commons
  • CVE-2018-1273, versions prior to 1.13-1.13.10, 2.0-2.0.5
• SQL Server
• ThinkPHP
  • Versions 5.x, 5.10, 5.0.23
• Oracle WebLogic Server
  • CVE-2017-10271 – versions 10.3.6.0.0, 12.1.3.0.0, 12.2.1.1.0 and 12.2.1.2.0)

The reference to “tmp_0324_scan” has been witnessed before, within a May 19, 2019, blog post from the forum.90sec.com, a Chinese-language Information Security group. The 90sec blog highlights a deep dive of a cryptojacking exploitation event targeting Apache Hadoop, Redis and ThinkPHP applications.

Of note, the bash script highlighted within the blog follows the same formatting as the newinit.sh shell script used by the WatchDog miner (see Figure 18). Aside from the different filenames and IP addresses, the two formats are practically identical.

Additionally, the references to “tmp/0324/scan” within the 90sec post are listed within the same format as witnessed within the networkmanager binary functions (see Figures 17 and 19).

It is clear that the activity being monitored by 90sec on May 19, 2019, is the same cryptojacking malware family researchers are seeing today in the form of the WatchDog miner. Several similarities can be observed between the past and present forms of the malware, such as that the same exploits appear to be used. However, newer techniques have been developed and implemented within the more current version of WatchDog. Specifically, we see this in relation to the phpguard binary.

Also of note, the denisenkom/go-mssqldb library is added to Go binary which allows for SQL DB functions to be accessible through the Go Language, including remote connections, error handling, bulk operations, logging and data manipulation (see Figure 20).

The Go binary is also loaded with the Google Cloud library Go Civil to allow for the usage of a Gregorian calendar with exactly 24-hour days, 60-minute hours, and 60-second minutes, as well as the Github Redis Go Library, allowing for Redis service control by the binary.

phpguard

Phpguard is a UPX-compressed Go language binary designed to protect the mining software during operation. It performs the functions of monitoring system processes and scheduled tasks or CronJobs to ensure the mining software is running. Unit 42 researchers have identified two different filenames for binaries that perform the same protective function, phpguard and sysguard.

Through the use of the custom Go library “tmp_0324_dog_platform” (see Figure 21), the Go binary is able to control the XMRig mining software in either Windows or NIX systems.

Additionally, the binary embeds the mining software within the relevant OS through scheduled tasks, as is the case in Windows systems (see Figure 22). This can also happen via CronJobs, as is the case with NIX systems (see Figure 23).

The binary will also continually crawl through each of the OS running processes to ensure that the mining process is running (see Figure 24).

The binary is designed to ensure the mining process is protected. If this is a first run for the binary, it will set the new process for protection. If the mining software has not been started, it will start the mining software. And if the software has yet to be downloaded, the binary will begin the download process (see Figure 25).

phpupdate

The phpupdate process is the XMRig mining software used by the WatchDog miner. Unit 42 researchers have identified three different filenames for binaries that perform the same mining operations, phpupdate, zzh and trace.

There is little to disclose about the WatchDog miner’s version of XMRig or its mining operations that is outside of known industry mining software operations. It offers a fully configurable operational menu, allowing the user to specify the following mining attributes (see Figure 26):

• URL of the mining pool.
• Mining algorithm (or the desired coin).
• Username.
• Password.
• Proxy information.
• Sending of a keep alive packet
• Size of packet, and more.

While the miner can be controlled by the phpguard Go binary, as was described within the section just prior, the mining software can also be operated through direct user interaction.

Conclusion

The WatchDog mining operation has been in progress since at least Jan. 27, 2019, and has collected at least 209 Monero CryptoCoins (XMR), valued at least $32,056 USD. The WatchDog actors are using cloud-efficient cryptojacking malware, through the use of UPX-compressed Go language binaries, ensuring they are able to compromise both Windows and Linux operating systems – assuming those systems have the Go platform installed. At this time, the WatchDog mining infrastructure is known to include 18 IP addresses and seven domains. These malicious endpoints continue to host or have hosted at least 125 URL addresses used to download the WatchDog mining toolkit. Additionally, the scanning and exploitation binary, networkmanager, is loaded with 33 unique exploits, including 32 RCE functions. The WatchDog mining operation is quite large, as at least 476 compromised systems are estimated to be mining at any given time.

It is clear that the WatchDog operators are skilled coders and have enjoyed a relative lack of attention regarding their mining operations. While there is currently no indication of additional cloud compromising activity at present (i.e. the capturing of cloud platform IAM credentials, access ID, or keys), there could be potential for further cloud account compromise. It is highly likely these actors could find IAM-related information on the cloud systems they have already compromised, due to the root and administrative access acquired during the implantation of their cryptojacking software.

Palo Alto Networks Prisma Access is configured to detect each of WatchDog’s 18 IP addresses, seven domains and their associated URL addresses through PAN-OS. Prisma Cloud also detects the usage of malicious XMRig processes used by the WatchDog miner operating in cloud environments that have Prisma Cloud Compute Defender installed.

Indicators of Compromise

IP Addresses

Domains

URL Addresses

Files