This document discusses the importance of instrumentation for effective fuzzing. It notes that while fuzzing may seem simple, it actually requires significant effort, target code adaptation, and input corpus minimization. Instrumentation is key to determining code coverage, finding new paths, and prioritizing inputs that lead to crashes or new code coverage. The document provides examples of instrumentation techniques using binary rewriting and hardware features and discusses how to set up fuzzing when source code is available versus when it is not. It also outlines some current gaps in fuzzing techniques.
3. The myth
• Fuzzing is easy
• Fuzzing is simple
• Instrumentation is left as an exercise to the
reader
4. The truth
• Fuzzing requires effort
• Generally requires adapting the target code
• Most of the time requires to build a corpus of inputs
• Requires minimizing the corpus
• Requires instrumentation:
– Did my target crash?
– On what input?
– Are my new inputs useful?
5. The hurdles
• Tool selection
• Tool integration
• Reliability
• Scale
• A bug found prevents fuzzer from reaching
further areas of code
7. Before
• 2 approaches:
– Mutate data forever (randomly, byte flip, …)
– Model data, mutate fields separately (Spike,
Peach, Codenomicon, …)
• Run for some iterations or until all states are
modeled
• hope for the best
8. Today
• Genetic algorithms => retain only best inputs
for further mutation
1. Mutate best input
2. Send to target
3. Measure impact based on some metric
4. Discard or prioritize input, back to 1.
9. Code coverage
• Code coverage is the most used metric
• Tells you if an input has triggered new code paths
• All tools try to measure code coverage one way or another
• Can be achieved :
– binary instrumentation (PIN, DynamoRIO)
– static rewriting (Dyninst)
– kernel probing (perf)
– HW (intel BTS => branch trace store)
10. How does it work
• Model control flow using basic blocks
• Discard unconditional edges (JMPs)
• First approach, trace callgraph
• Hard to compare 2 callgraphs
• Best approach: retain edge count
• Provides an unordered code coverage heatmap
12. Compare code coverage maps?
• Gained edges - lost edges > 0?
• Simple, but will crush path divergence
• Solution, keep track of interesting diverging paths
• When no new edges, check edge hitcounts
• Higher hitcounts, mean you control a loop
boundary
14. Corpus minimization
• You have collected all xml documents or IM
packets from the internet
• What is the minimal set of inputs which
achieves maximal code coverage?
• Open all inputs and record code coverage
• Keep only valuable inputs
15. In practice
• No open source tools to achieve this
• Notable exception, with source on Nix for files
=> afl-cmin to the rescue
• Otherwise, a good base is runtracer, drcov or
coco.cpp pintool
• Building the minset is up to you after that
18. A few obvious questions first
• Do you have source code?
• Where does it take input from?
– Network
– File
– …
• Do you already have valid inputs?
– Packets
– Pdf
– …
19. First of all
• Turn on coredumps
• Throw whatever you have at the binary
• dd if=/dev/urandom bs=1024 count=1 | nc
localhost 1234
• Or mutate some corpus inputs with radamsa
• Keep CPU busy whilst you figure out a plan
• Now think
20. You have source code
• Find a way to get it to work with American
Fuzzy Lop
• AFL “batteries included”
• AFL works great:
– File input
– Amazing performance/reliability (forkserver)
– Instrumentation/stats built in (ASM instrumentation)
– Scaling (distributed fuzzing)
• Limitations:
– Network fuzzing
– Any form of daemon
21. Wrapping for AFL
• Target can read from stdin or argv, your good
• Otherwise, write a wrapper around your target
functions
• Read_from_stdin(char *buf) { target_func(buf);
exit() }
• Problem: complex when functions are tightly
coupled (globals, complex structs, …)
22. No source?
• Things start to get messy
• Options:
– Afl-qemu
– Afl-pin
– Afl-dyninst
– Honggfuzz (Linux or requires HW support)
– …
23. Mo problem
• Idea is always the same
• Through instrumentation, get code coverage info
• Bind it someway to AFL:
– AFL-qemu => Use Qemu userland to hook BBLs
– AFL-PIN => Use PIN to hook BBLs, no forkserver
support
– AFL-Dyninst => static rewrite to hook BBLs
25. Gaps
• Smart fuzzing network daemons
• Corpus minimization
• Windows support
• Triaging (exploitable doesn’t work on cores)
• We need to build bricks, not solutions
What tool does the job (Peach, AFL, libFuzzer, …) ?
How do I collect crashes, triage them, instrument?
My fuzzer crashed after a 2 weeks run
How do I synchronize my fuzzers/targets?
Stagefright talk at Defcon. Audience hadn’t heard about AFL. Presenter had to fix a ton of bugs for the fuzzer to make progress.
Count the edges, and variation in the edges
Example tcp+3397 => tcp+3411: Will count as 3 edges in code coverage map
AFL is an impressive tool. Check out how it works for
Building a product for fuzzing is very similar to building a product for Q&A. You need to make loosely coupled modules which can be unit tested.
This makes fuzzing jobs way easier
Having a ball of code of 5M LOC with a trail of dependencies makes it very hard to fuzz.