The Art of Software Security Assessment

blog December 19th, 2006

Overview

Our goal with the book was pretty clear—we wanted to write a book about how to find security vulnerabilities using static manual analysis (i.e. reading the code.) This is the technique we’ve been using with success in our day to day jobs for nearly a decade, yet there’s very little written about it out there. Most existing security literature that covers vulnerability discovery focuses on black-box testing, fuzz testing, or automated bug detection with source code or binary analysis tools. The remaining majority of the deep-tech security resources tend to focus less on vulnerability discovery and instead concentrate more on techniques for the exploitation of known vulnerabilities. All of these topics are interesting, challenging, and certainly valuable and useful in the grand scheme of things, but we decided to focus on our particular specialty: static analysis.

So, we managed to write the entire book without actually talking much about exploitation techniques, but, near the end of the editing process, we realized that the book really needed some discussion of exploitation to tie everything together and give readers the proper context. As much as we dreaded writing yet another explanation of buffer overflows, we bit the bullet and put this chapter together. Our goal was basically to explain the whole memory corruption problem to a technical reader that might otherwise be unacquainted with modern attack techniques. We wanted to give readers the background necessary to appreciate the vulnerabilities in the rest of the book, but we didn’t want to spend time recreating all of the deep technical content that is covered comprehensively in other resources.

That said, we did spend some time giving a basic overview of some of the standard protection mechanisms present in many contemporary OSs, as they provide significant hurdles to successful (and reliable) exploitation of memory corruption vulnerabilities. We felt that failing to mention these technologies would give a false impression about the current state of software security, as they are utilized in the default configurations of several popular systems. Moving forward, the presence of these technical controls will increasingly become the rule rather than the exception.

External Resources

Koziol et al’s The Shellcoder’s Handbook
Hoglund and Mcgraw’s Exploiting Software
Phrack magazine
Uninformed magazine

External References

igorsk’s "Reversing Microsoft Visual C++ Part I: Exception Handling." (Page 179)
Matt Pietrek’s "A Crash Course on the Depths of Win32 Structured Exception Handling." (Page 180)
Solar Designer’s "JPEG COM Marker Processing Advisory." (Page 183)
Phrack 57-09 - "Once Upon a Free." (Page 184)
LSD’s Windows Shellcode. (Page 188)
Alexander Anisimov’s "Defeating Windows XP SP2 Heap Protection and DEP Bypass." (Page 192)
Nicolas Falliere’s "A New Way to Bypass Windows Heap Protections." (Page 192)
Oded Horovitz and Matt Connover’s "Windows Heap Exploitation." (Page 192)
Phantasmal Phantasmagoria’s "The Malloc Maleficarum." (Page 192)
The PaX project. (Page 193)
Litchfield’s article on bypassing SEH. (Page 195)

Further Reading and Discussion

There are so many excellent resources on exploit writing out there that it’s a bit overwhelming trying to provide references to them. Certainly, the Shellcoder’s Handbook, Phrack, and Uninformed are excellent resources, and they should teach you the ins and outs of exploit creation. We hear that there’s a second edition of the Shellcoder’s Handbook in the works, which we’re definitely looking forward to. Gera’s exploitation challenges are awesome, and there are probably lots of other good sites in the same vein. There are several other books on exploitation techniques that we’ve heard good things about from people trying to learn the craft. We’ll ask around and maybe put up some more links in a little bit.

Even though we didn’t talk about it too much in our book, we’ve written a lot of exploits over the years. Here are a few take-aways, based on our experience:

- Memory corruption affects a program in such a subtle way that it has to be taken seriously even if it seems innocuous or limited.

Exploit writing is the most fun when it’s hard. When there are several limitations placed on how you can adversely affect a program, it becomes a creative endeavor. Basically, you isolate each "bad" or "odd" thing you can do and think of it as an atomic action in what is essentially a new programming language that exists in the bloody remnants of the exploited process. You try to use all of these odd behaviors and tricks in concert to "solve" the system and win.

- The only way to demonstrate exploitability is to write an exploit.

Most of the time, when you’re doing an assessment, it makes no sense to try to write an exploit for every thing you find, barring special circumstances. Normally, the developer can just fix it and move on, and it doesn’t matter all that much if it was actually exploitable. It’s occasionally useful to be able to throw together an exploit in order to authenticate yourself to an alpha-male developer or cut through lame excuses, but it’s also incredibly time consuming.

However, if it actually matters in the real world if an issue is exploitable, then you have to write an exploit. There are so many little things that crop up when you try to actually exploit an issue; there’s inevitably that check you missed in some header file, or the unrelated piece of code three functions up that inadvertently stops your attack through sheer happenstance. Even if your analysis is spot-on, you still have to deal with the compiler’s will, not to mention any security mechanisms running on the target.

In short, if you’re doing proactive software assessment, writing an exploit is rarely justified (from a business perspective.) If you’re doing vulnerability research or "hostile" third party software assessment, writing an exploit is usually a necessity.

- The landscape is changing with all of the new security mechanisms.

Things are changing, and writing reliable exploits is getting much harder. We are certainly living in interesting times.

- There are other good reasons for writing exploits. (updated 1/1/07)

So, as Dave Aitel hints at in his comment, we probably overstate the case by saying this: In short, if you’re doing proactive software assessment, writing an exploit is rarely justified (from a business perspective.) Here are a couple reasons why writing exploits in an app-sec consulting environment might make sense:

1. If you work for a business that leverages exploits commercially, then developing and honing these skills obviously is a good idea. Consultancies like Immunity and CORE-SDI have products and services that monetize exploit code. (I’ve always wanted to use the word monetize.) Obviously you aren’t going to turn around and sell exploit code for a bug that you found on a client’s dollar in that client’s product, unless you had the world’s most insane NDA/MSA and/or the business ethics of Gol Pot (Pol Pot’s lesser-known evil twin brother.) That said, it probably does make for an excellent way to keep sharp by making you write exploits for real-world vulnerabilities. Also, those guys probably have some serious tool-chain and infrastructure that minimizes the time investment involved, and writing lots of exploits probably helps them flesh out their tool-chain.

2. Exploits can make your work concrete and real. As Dave said, it makes for a good presentation at the end of the gig.

3. Writing exploits can be interesting and fun. No matter how old and jaded you are, you still get a slight adrenaline bump when you see an exploit you wrote for a bug you found work for the first time. Dave makes the point that if you want to hire and keep top-tier talent, you have to make the work interesting.

• Discussion