Metasploit Shellcode Hiccup in Exploitation (English)

Bui Quang Minh

With software vulnerability, most demo exploits make use of Shellcode taken from metasploit.com of HD Moore. This is really a success of HD Moore.

However, I found a small problem in the Shellcode of HD Moore. If a person does not have a thorough grasp of Shellcode, he or she might encounter the circumstances where Shellcodes are not executed even though he or she has already been able to redirect control of the program.

More specifically, at the beginning of Shellcode, HD Moore often uses his own decoder (though there are several other methods). Let look more closely at the fourth instruction: fnstenv [esp-0Ch]. This instruction saves FPU environment into the memory location pointed by esp-0Ch.

Basically, this instruction works just fine. But, let's imagine, if ESP is pointing right at the start of the shellcode, and the control redirection is achieved via "jmp ESP". As a result, this instruction unintentionally overwrites some part of the shellcode or even lies onto the currently executed instruction "fnsenv". The consequence is that shellcode cannot run successfully.

This is really an issue because exploitation making use of "jmp esp" is very popular or even the basic matter in software vulnerability exploitation. Of course, I believe that many people have encountered this problem and haven't been able to resolve it, or have overcome it but do not totally understand the reason why. And this is the main idea of this short paper.

OK, we have finished with the theory. And here come the solutions, two solutions actually:

- Create a new shellcode which does not use ESP.

- Add a lot of NOP instruction (0x90) at the beginning of shellcode.