We’ve created a small guide to get you started with CTF and more or less infosec in general. There are a few selected resources for each of the major CTF disciplines that should help you get up to speed in those.
Basic Linux/Unix skills and some knowledge of programming should suffice to begin with.
CTF in general
Other than that you’ll usually learn a lot from writeups for CTF challenges (especially for ones you tried but couldn’t solve). A writeup is just a guide walking you through the solution for a challenge. There is a CTF writeup repository on github that contains a lot of them. You’ll also find writeups for the corresponding CTFs on ctftime.org.
You should also make yourself familiar with ctftime.org. The site keeps a global team rating for all CTFs and upcoming events will be announced there. You’ll also find some other useful things around the site.
How to get started in…
For a nice overview on memory corruption bugs and exploitation see this white paper History of Memory Corruption Attacks (pdf) or Project Zero’s blogpost series on memory corruption vulnerabilities. That should get you up to speed on memory corruption bugs so you can recognize these vulns in the challenges and then do more research on them if needed.
To develop some more practical skills you can take a look at the microcorruption game. It’s a browser based exploitation course guiding you through memory corruption exploitation on an embedded platform. It’s designed for beginners but you will need to read up on the topics on your own.
Also, you should give the overthewire wargames mentioned above a try.
For a decent introduction to Return Oriented Programming (ROP) see this article.
The cryptopals challenges are a nice set of exercises that guide you through modern cryptography and how to break it. You’ll also be doing quite a lot of programming throughout the course.
The Web Application Hackers Handbook is basically the bible of web security. It will take some time reading through the book but it’s definitely worth it.
You can also take a look at the following online challenges/courses for learning some web security:
Moreover there is the Open Web Application Security Project which also has some nice resources on web security.
To get started with reversing you’ll probably want make yourself familiar with some assembly first. There are a lot of guides on that topic out there, just pick one and write a HelloWorld Linux program in assembly.
After that a good idea is to grab a disassembler, compile some small C code snippets and load the resulting binary into your disassembler of choice to see what the compiler generated. This will give you a good feeling for assembly and help you recognize code patterns and functions in disassembled binaries. You can also take a look at some crackmes (for example from here) to get some hands-on experience.
While not really a core CTF discipline some good coding skills will definitely proof useful (not only for CTF obviously). There are some nice sites to practice on out there, for example talentbuddy or hackerrank.
Also, there are some websites organizing regular coding contests like the Google Code Jam, topcoder or codeforces.