Reverse Engineering on Windows: A Focus on Malware

Posted on 2020-02-25 by Josiah Smith

Our CTO, Pedram Amini, and colleague Ero Carrera have open-sourced all the materials from a two-day reverse engineering class they taught over the years at BlackHat, the last instance being at Blackhat 2009 Federal. Written in LaTeX + Beamer, the course materials can be rendered in both slideshow (PDF) and article (PDF) modes. Additionally, the courseware includes malware samples and all requisite references, scripts, tools, exercises, and solutions. The course overview and description that follows is preserved from a decade ago. As part of our mission, we look to give back to the community by way of open-source software, free (as in beer) access to data, and sometimes silly homages. We hope these materials provide value for practitioners looking to get into the field of reverse engineering.


Reverse engineering has evolved from a "dark art" traditionally restricted to the elite few, to a learnable methodology using public and commercial tools. Vulnerability researchers utilize the art to go beyond the reachable depth of traditional fuzzer technology and locate the more obscure finds. Because of advancements in today's malicious code, analysts can no longer rely solely on live-analysis techniques for mapping the internal workings of malware. In general, more and more researchers are finding the need to peek "under the hood". This class is meant to impart a cutting-edge understanding of malicious code analysis upon attendees, ultimately taking them to an advanced level of reverse engineering skills applicable to other security domains.

What You Will Learn

This course was designed for students who have an introductory/basic understanding of x86 assembly and reverse engineering as well as more advanced students wishing to refresh their skills and learn new approaches to familiar problems. The course will cover the basics of x86 assembly and pattern recognition, Windows process memory layout, tools of the trade (such as IDA Pro and OllyDbg), the PE file format and basic exploitation methodologies abused by worms to penetrate a target system (stack/heap overflows). As this course is focused on malicious code analysis, students will be given real-world virus samples to reverse engineer. The details of executable packing, obfuscation methods, anti-debugging, and anti-disassembling will be revealed and re-enforced with hands-on exercises.

Toward the end of the course more advanced reverse engineering techniques with applications to malicious code analysis will be taught—including:

  • Various approaches to automation
  • Malware classification
  • Applications of binary matching/diffing

Course Structure

This is a two-day course where the notion of "rapid response" is taken into consideration with each aspect, focusing on techniques and methodologies that can be applied in a timely and effective manner. We will force you to learn shortcuts and put your mouse to rest. At the completion of this course, students will walk away with applicable real-world knowledge that can be directly applied to various reverse engineering-related tasks, especially with regards to malicious code analysis.

This course is by no means a two-day lecture. Instead, you will be engaged in a number of individual and group hands-on exercises to reinforce and solidify everything that is taught in the class. Some of the exercises are held in a competitive nature, followed by a class discussion to pinpoint elegant approaches and solutions that various individuals or groups may have used. Despite the fact that the course is held in Vegas, take-home exercises will be available for the type-A personalities attending the course.

malware-analysis reverse-engineering

Get The InQuest Insider

Find us on Twitter for frequent updates, follow our Blog for bi-weekly technical write-ups, or subscribe here to receive our monthly newsletter, The InQuest Insider. We curate and provide you with the latest news stories, field notes about innovative malware, novel research / analysis / threat hunting tools, security tips and more.