wrongbaud
Details
Updates / Information
For regular updates regarding class materials, follow wrongbaud and voidstarsec on twitter.
Course Overview
This is a four session course that covers the basics of reverse engineering software with Ghidra. For each session there are exercises to be completed that can be found on the project github page.
Exercises and materials can be found here.
Hardware Requirements
- 8GB RAM
Software Requirements
- Docker (or an Ubuntu 18.04 VM)
- The Ghidra SRE Tool
- Requires JDK Version 11.0.2 - Windows, Linux, OSX
Getting Started
- Download Ghidra from here
- Download the exercises / Docker container from here
git clone https://github.com/wrongbaud/hackaday-u
- Build the docker container (Note: You can also use an Ubuntu 18.04 VM if you're doing this, skip to step 5)
cd hackaday-u/docker docker build . -t hackaday
- Test the Docker container (If using Ubuntu 18.04, skip to step 5!)
docker run --rm -it hackaday /bin/bash
- Run a challenge binary as a test!
root@522471199b16:/home/hackaday# ./hackaday-u/session-one/exercises/c1 Please supply the password! root@522471199b16:/home/hackaday# ./hackaday-u/session-one/exercises/c1 test Wrong answer, we'd never use test as the password!
The goal of these challenges is to bypass or provide a proper password. Over the course of the sessions the amount of information that you have to provide will change and the complexity of the passwords will increase.
Course Goals
- Familiarize students with the basic concepts behind software reverse engineering
- x86_64 Architecture Review
- Identifying C constructs in assembly code
- Disassembly vs Decompilation
- Teach students how to use the Ghidra SRE tool to reverse engineer Linux based binaries
- Basic navigation and usage
- How to identify and reconstruct structures, local variables and other program components
- Demonstrate and explain the methodologies used when approaching an unknown program with Ghidra
- Where to start when looking at an unknown binary
- How to quickly gain an understanding of an unknown program
- Provide challenges and "crackme" exercises so that students gain hands on experience with Ghidra
Prerequisites / Resources
- C Programming Language Review and Tutorials
- Introduction to 64 Bit Assembly
- NASM x86_64 Cheat Sheet
- Excellent x86 Review
- Intel Manual - Volume 1
- Intel Manual - Volume 2
- Online compiler with assembly output
Playlist for the Reverse Engineering with Ghidra series:
https://www.youtube.com/playlist?list=PL_tws4AXg7auglkFo6ZRoWGXnWL0FHAEi
Course Classes
Collapse
-
Class 1 video
Lutetium • 06/23/2020 at 18:13 • 0 commentsReverse Engineering with Ghidra Class 1
Class 1 outline
0:00 - Presentation Outline
2:50 - What is Software Reverse Engineering?
4:12 - Software Engineering Review
24:54 - x86_64 Architecture Review
45:10 - Ghidra Overview and Basic UsageQuestions can be sent to superconference@hackaday.io
-
Office Hour Questions 6/25/20
wrongbaud • 06/26/2020 at 11:45 • 0 commentsOffice Hour Notes from 6/25/20
Questions:
- When is code obfuscation executed?
- There are various levels of code obfuscation, sometimes the source code itself is obfuscated, and other times it’s applied to the machine code.
- What do you consider .NET stuff? Is it a binary file or something else?
- It is technically a binary - but when reversing things build on the .NET framework one would want to start with the dotpeek tool: http://www.jetbrains.com/decompiler/
- Do people obfuscate by handwriting assembly or are there obfuscating compilers?
- Obfuscation can be performed in a number of ways. For example, there are obfuscating assemblers, and various compiler tricks that can be done to aid in obfuscation.
- How often is obfuscation in play?
- It depends on your target, you’ll find it often in games and other things that require some sort of DRM, but it’s less common when looking at embedded firmware images for example.
- It would be great if you can give some pointer on how to identify packed or encrypted code using Ghidra
- There is a plugin here: https://github.com/d3v1l401/FindCrypt-Ghidra that can be used.
- Identifying packed or encrypted code can also be done by performing entropy analysis on the code segment. Binwalk can do this with the -E option.
- Any binary can be reversed?
- Yes, technically speaking anything that contains machine code that can eventually be run by the CPU can be reverse engineered.
- Do you have any resources for extracting binaries from a platform/uC?
- Yes! Below are some examples that I have done on my website: https://wrongbaud.github.io
- On embedded systems do you often see heap being used? Or is deterministic memory (stack) more common?
- This depends entirely on what the system is used for - if it’s running an RTOS or a Linux based OS, then you’re going to see heap usage. Smaller microcontrollers may not have space/resources to implement a memory allocator and will rely on statically sized buffers in SRAM.
- Is the memory for AH / AX / EAX / etc. shared? i.e. can you access 8 bits of AX by accessing AH?
- Yes, the various representations of these registers can be used to access those specific size ranges.
- Is there a universal reference to the instruction set for x86_64?
- Yes, the Intel instruction set architecture reference is linked on the course page.
- x86-64 has a flat 64bit memory model so RAM, as well as PCIe peripherals, can end up in memory space, correct?
- Technically this is correct, however there are memory protections in place to try to prevent these regions from being accessed. The operating system / mmu will protect these regions of memory, as well as the drivers utilizing them from being accessed.
- What are 'high level' differences between Ghidra and Ida Pro? [understand it may just be OpenSource vs not]
- There are many differences between the two, and we will go over these during the second class session!
- Will we be touching on what to do if Ghidra can’t find cross-references because the pointers are some +value off from the virtual addresses in this course? (trying to reverse some firmware blob)
- When looking at firmware blobs, properly creating a memory map is very important, and may be the reason why you’re having issues with XRefs.. This can be done from within ghidra by clicking on the memory viewer, or by writing a loader / script to perform this for you. It is important to also create relevant RAM regions when working with firmware images as these are often where the XRefs will be located.
- Thank you for the amazing tour of the tools, but what is the "goal" - what can we expect to do with all this? :)
- The goal of this course is to familiarize students with the concepts behind reverse engineering software, and provide a base understanding of how to use Ghidra to solve binary puzzles and challenges.
- By the end of this course, students will be comfortable loading x86_64 ELF files into Ghidra and be able to analyze them.
- I thought EABI was for embedded...
-
Class 2 Video
Lutetium • 06/30/2020 at 17:11 • 0 commentsReverse Engineering with Ghidra Class 2
Class 2 outline
Intro: 0:0
Assembly Language / Applying Function Signatures: 3:08
Imports and Exports: 8:49
Control Flow Statements in Assembly Language: 10:23
Switch Statements in Assembly Language: 18:10
Loops in Assembly Language: 24:34
Variables in Assembly Language: 32:42
Functions in Assembly Language: 39:46
Heap Memory: 48:08
Array Accesses in Assembly Language: 50:11
-
Class 2: Q&A
wrongbaud • 07/03/2020 at 15:18 • 0 comments- Where do we get the exercises? The only info I've gotten is on the Eventbrite.
- The exercises are stored on the github page, located here: https://github.com/wrongbaud/hackaday-u
- I tried Gidra on two firmware images I had: an esp32 and STM32 dev board. Since it's a binary blob, it did not provide the ELF info on architecture, so I had to fill it out by hand, and there are many ARM options. I chose ARM Cortex, but it didn't seem to work that well. How do you pick arch from the many ARM options? What would be the right one for this firmware?
- In order to determine the proper CPU architecture, you should start with any applicable datasheets. A lot of the MCUs in those series’ that were mentioned use Cortex cores, but analysis will fall short if you do not properly define the appropriate memory regions, which can be acquired from the relevant datasheet.
- Why arent functions like main() for c++ automatically set to the right parameters?
- The decompiler tries not to make too many assumptions for these function prototypes and uses the context that is provided by the instructions in use - this allows things to be more generic and causes fewer failures, but also means that users have to sometimes identify and add the appropriate types. In short - you don't want to assume too much such that it breaks other use cases.
- Can you take thins like [rbp+8] and give them symbolic names for local variables?
- Yes, if you right click the label that is being used, you can rename the variable to something different.
- Also for the exercises, should we use docker on windows or do WSL and git clone the repo?
- They have been tested within the docker container, and in an Ubuntu 18.04 VM, so I would recommend sticking with one of those two. If you have issues with docker, reach out and we’ll try to help you.
-
Class 3 Video
Lutetium • 07/07/2020 at 02:07 • 0 commentsReverse Engineering with Ghidra Class 3
Class 3 Outline
0:00 Intro
2:36 - SRE Tool Landscape
8:03 - Structs: ASM, Identificaion and Ghidra Analysis
20:19 - Pointers: ASM, Identificaion and Ghidra Analysis
35:30 - Enums: ASM, Identificaion and Ghidra Analysis
40:00 - x86_64 System Calls
45:40 - File Operations
51:02 - Ghidra Tips: Patching, Bookmarks, Searching, CommentsQuestions can be sent to superconference@hackaday.io
-
Class 3 Q&A
wrongbaud • 07/10/2020 at 10:24 • 0 commentsClass 3 Q&A
- How are we using docker for this class?
- Docker is used to run the exercises in, you can also use an Ubuntu 18.04 virtual machine if you prefer
- What is a (the most?) common example of being able to pull in a header file? Doesn't most of RE activities assume you don't have the source?
- If you are aware of an open source library that the program may be using, you can import header files from that. Or perhaps if you are reverse engineering a custom kernel module, a lot of the structs in use are likely from the mainline kernel
- Can you touch on using tools like Ghidra to remove calls to say a dongle attached to the system?
- This would be entirely system dependent and more context would be needed. Are you trying to functions? What is the end goal? etc
- How do you do expressions in Ghidra? e.g. the last exercise which did a complicated shift and arithmetic---I ended up using Octave to calculate it, but Ghidra must have it as well but I couldn't find it.
- Aside from PCODE emulation, I am not aware of a way to directly evaluate the resulting decompilation. This would require using an external emulator of some sorts.
- Could you give a couple of examples of what is IDA good at that nobody else has, and vice versa Ghidra and R2
- IDA: Good at c++ demangling, windows PDB parsing, strong decompiler
- R2: Extensible, can easily be expanded upon with plugins, community support, open source
- Ghidra: Decompiler support for every processor, open source, actively developed
- Is it possible to demo running Ghidra alongside a debugger? I know the ret-sync plugin exists but I've had trouble with it
- Right now the built in debugger in in alpha testing and hopefully they will be releasing it with the next official release
- Could you use Ghidra to reverse engineer itself?
- Ghidra is Java based, and open source so there would be little reason to reverse engineer it specifically when the source code is hosted up on github
- Should all struct members take up the same amount of space?
- No, it depends on multiple factors - the architecture of the target system, the compiler optimization settings and of course the members of the struct itself!
- Does running a syscall, by definition, execute instructions defined in the kernel? How does that carry over to an embedded context?
- The syscall instruction does a number of things, but most importantly it puts the value of IA32_LSTAR MSR into PC, this will jump to valid code that will handle the syscall properly, think of it as similar to an interrupt vector table on an embedded processor.
- I was surprised there wasn’t any Ghidra feature for syscall analysis.
- As of right now I am not aware of any plugins that do this, but it would make for a great side project!
- A question I've been having is why I often see extra (typically repeated) arguments in the decompiled output. For example, in many of the examples the functions that add two parameters are known to only take two parameters, but Ghidra shows them as having 3 or 4 being provided. What does it mean/why does it happen? How do I fix it in Ghidra?
- This happens because the decompiler makes a lot of assumptions and often times these assumptions are simply incorrect. You can fix this up by changing the types of variables on the stack such that they are the proper size.
- are there any special considerations needed to reverse a proprietary kernel module?
- Nope! They are just ELF files, this is a good example of when one might want to import header files from the kernel source depending on the driver and what other subsystems it uses, for example if your driver uses USB URB objects / structs you could import those in order to make it simpler.
-
Class 4 Video
Lutetium • 07/14/2020 at 21:54 • 0 commentsReverse Engineering with Ghidra Class 4
Class 4 Outline
0:00 - Intro
3:14 - Ghidra: Loading External Libraries
10:31 - Ghidra: Patch Diffing and Analysis
19:30 - Ghidra: Checksum Tool
21:38 - Ghidra: Memory Manager
25:39 - Ghidra Internals: PCODE and SLEIGH
39:00 - Ghidra Extensions
45:00 - Ghidra Scripting Overview and ExamplesQuestions can be sent to superconference@hackaday.io
Enjoy this course?
ShareDiscussions
Seems like the 5. Run a challenge binary as a test! instructions above are wrong:
./hackaday-u/session-one/challenges/c1
should be: ./hackaday-u/session-one/exercises/c1
Also, II had to use "docker build . -t hackaday --no-cache" because the apt-get update didn't work properly. I tried combining the command with the install using an &&, but then the DEBIAN_FRONTEND="noninteractive" didn't work and the build hung with tzset. Maybe 'export DEBIAN_FRONTEND="noninteractive"' would have worked, but I did not try it.
Good catch, I'll update the description! We changed the layout of the github repo after these instructions were written. I'm not sure why the container build gave you issues however, it may have been something specific to your machine
If you'd be interested in running a team collaboration focused Class over at https://www.ghidra-server.org/ we'd be happy to host a class playground and make some arrangement for swift accounts.
Thanks! We'll let you know when we kick off the next iteration of this course!
Hi! where are you guys having discussions? Is there a Discord server or something like that? Are office hours streamed and if so where can I get a link to the class? Thanks!
Ops, nevermind :) New to the platform... just found the chat feature.
When you post solutions for these assignments, please post the source that produced these binaries. Looking at the struct example I'm completely confused how any reasonable code could produce the assembly I'm seeing.
When the class is over, we will provide solutions to those that signed up and can also include the source code for the exercises. One thing to keep in mind is that the exercises are written with the goal of testing your knowledge of the underlying concepts first, and sanity/cleanliness second! - they are meant to be challenging and somewhat bespoke!
Having issues fininding the struct in the exercises, in the struct file I see `RAX=>local_38,[RBP + -0x30]` however the slides it says to look for `rbp-0x10` but not seeing any minus offsets on the address other than than `RAX=>local_38,[RBP + -0x30]`. Am i just being dense?
If you look at the notation, you'll notice that it's adding a negative value, so it's still a subtraction operation. The assembly listings in the slides were generated by Objdump and not Ghidra!
Quick question would it be possible to put a note on how to find entry points on microprocessor firmware?
That will be dependent on your micro controller - if you have a datasheet, the information will be available there, otherwise you can use information inferred by the various startup instructions / interrupt vectors. Perhaps we can do a firmware RE class in the future for those interested.
Hi, could someone please help me to get the new exercises into the docker container? I'm unfamiliar with docker and with git. Here is what I did:
1. From within the hackaday-u directory, I ran "git pull"
- I can see that the session-two folder was added, and some new exercises under session-one/exercises/.
2. From within hackaday-u/docker, I ran "docker run --rm -it hackaday /bin/bash" but in the shell the session-two folder and the updates to session-one/exercises weren't there
3. Exited the docker shell, used the docker dashboard to make sure no container was running
4. From within hackaday-u/docker, I ran "docker build . -t hackaday"
5. From within hackaday-u/docker, I ran "docker run --rm -it hackaday /bin/bash" again; still no luck.
Thanks
You will likely have to remove the old container/image and rebuild, or from the respository within the running container you can run git pull - then you can commit those changes back into the container.
Any chance you can provide high level "next steps" for someone who is primarily interested in games, particularly MMOs and emulation in C++? There's so much to learn it can be challenging figuring out where to focus.
The core reversing skills are still going to be relevant for any game you're reversing. So starting there and working up towards more complex targets wouldn't hurt. Reversing C++ can be challenging but there are some good resources out there
If there's a preferred order for working on the exercises, it would be helpful to encode that in the filenames. Session one did for the first four, but session two didn't.
The exercises are to be done in the order that they are presented in the slides which are also in the repository.
Off-topic, but: could I have installed Ghidra and the JRE in the Docker container, and would there be any advantage to doing that?
You could have built Ghidra in the docker container, but as far as I understand, there not a way to run a GUI app such as Ghidra from a Docker container.
There is a zoom link to the office hours in the public chat here: https://hackaday.io/messages/room/288398
Hi, I got a segfault when trying to run the binaries from the repo; I'm running Ubuntu 20.04 from WSL. Are there additional dependencies I should be aware of?
They've been run and tested on Ubuntu 18.04 - which challenges are giving you a problem? The Docker container should run all of the exercises properly.
Had a segfault trying to run dobby out of elf-exercises. The binaries in the exercises folder are running fine. I'm using Ubuntu on Windows Subsystem for Linux, so it can't run Docker.
Hey, thanks for the first session. Quick question about slide 35 (asm example): when we jump to `_greater`, why does `RAX` change from `0x3FFF` to `0x2FFF`?
Great catch - it definitely should not do that - that was a PPT mistake, I'll edit them and re-upload later today
finished all the challenge cracks! Super fun. Thanks for showing that renaming argc and argv trick. It makes the decompiled C so much more readable.
Got challenge three and four cracked! Super fun. Thanks for putting these together. Literally shouted out loud when I got them haha.
Great work! We'll be putting out a few more later on today
Will the videos be posted for later study?
I will be working during these hours...
Yes they will be - the page will be updated with the video links when they are ready
For anyone confused about the adding a byte happening as I was to get the start of the string in the challenges, argv in C is an array of char pointers. The first pointer, 0 indexed, will always point to the name of the program itself and the second is the first command line argument passed. Char pointers are a byte long so we're accessing the user provided argument at address argv + 8.
Been a bit since I messed around with C so this is possibly only a revelation to me haha.
Great point! We're going to talk about this in more detail next week, nice work!
joined the waitlist for session one and I'm in on session two. Is session one actually sold out or is this an eventbrite issue?
I believe it is actually sold out, however we will be releasing the videos of the classes as well so you will still be able to access the material even if you're not present for the actual class!
I'm in the same boat - Is it possible for you to release the videos or course outline from session 1 before session 2 starts, so we aren't too far behind in the following sessions?
Thanks!
Is there any availability left for this class? I learned about it from the hackaday YouTube channel, and have been trying to figure out how to sign up. Thanks.
The eventbrite page is here: https://www.eventbrite.com/e/hackaday-u-reverse-engineering-with-ghidra-tickets-109681391996 - we will be releasing the videos of the classes as well so you will still be able to access the material even if you're not present for the actual class.
Will the video's be released before the next class? Just wanting to know if it's worth signing up for the others if the first class is full -- you know, so you're caught up. Thanks again.
bump ;P anyone still breathing in here? Question: course shows 7 classes but Im only seeing 4. where/if access the remaining 3?