Capture the Flag (CTF) is a popular computer security exercise in which teams competitively attack and/or defend programs in real time. CTFs are currently expensive to build and run; each is a bespoke affair, with challenges and vulnerabilities crafted by experts. This not only limits the educational value for players but also restricts what researchers can learn about human activities during the competition. In this work, we take steps towards making CTFs cheap and reusable by extending our LAVA bug injection system to add exploitable vulnerabilities, enabling rapid generation of new CTF challenges. New LAVA bug types, including memory corruption and address disclosure, form a sufficient set of primitives for program exploitation.
We used these techniques to create AutoCTF, a weeklong event involving teams from four universities. In order to assess how AutoCTF differed from a handmade CTF we conducted surveys and semi-structured interviews after the event. We evaluated both challenge realism and relative effort expended on bug finding and exploit development. Our preliminary results indicate that AutoCTF can form the basis for cost-effective and reusable CTFs, allowing them to be run often and easily. These CTFs can be used to train new generations of security researchers and provide empirical data on human vulnerability discovery and exploit development.