Using NGINX as UDP load balancer at high rate

Discussion in 'Nginx' started by A_H80, Sep 30, 2023.

  1. #1
    Hi, I’m running nginx on Kubernetes and I want to use it as a UDP load balancer at high rate.

    nginx.conf:

    
    user  root;
    
    worker_processes  auto;
    
    
    
    events {
    
      worker_connections  1024;
    
    }
    
    
    stream {
    
      upstream app_node {
    
      server <receiver_ip>:<receiver_port> fail_timeout=60s weight=1;
    
      }
    
    
      server {
    
      listen <nginx_port> udp;
    
      proxy_pass app_node;
    
      proxy_protocol on;
    
      set_real_ip_from 0.0.0.0/0;
    
      proxy_bind $server_addr:$remote_port;
    
      }
    
    }
    Code (markup):

    I try to test it by generating high traffic, such as 30,000 packets per second on average for 5 minutes, and each packet size is 1 KB.

    I use tcpdump for monitoring packets that nginx receives and sends.

    Tcpdump command for receiving packets:

    
    tcpdump -i <receiving_inteface> dst <nginx_pod_ip> and port <nginx_port> -w receceived_30000.pcap
    Code (markup):
    Tcpdump command for sending packets:

    
    tcpdump -i <sending_inteface> src <nginx_pod_ip> and dst <receiver_ip> and port <receiver_port> -w sent_30000.pcap
    Code (markup):
    But the count of packets captured by the two commands is not the same. It seems that nginx drops packets:

    Result of first tcpdump command:

    
    9000000 packets captured
    
    9000000 packets received by filter
    
    0 packets dropped by kernel
    Code (markup):

    Result of second tcpdump command:

    
    8997898 packets captured
    
    8997898 packets received by filter
    
    0 packets dropped by kernel
    Code (markup):
    And this difference will increase if we increase the rate of sending packets.

    So what is the problem here? Is our configuration wrong, or has something else happened?
     
    A_H80, Sep 30, 2023 IP