ich habe einen Server mit C++ und der BOOST ASIO Bibliothek geschrieben.
Der Server soll möglichst viele Verbindungen annehmen (läuft auf einer Debian Maschine) und als Gateway zwischen Client und Server fungieren, sprich pro verbundenem Client zwei offene Sockets. Das System hat nur ein Ehternet Interface.
Nun ist es so dass ab einer bestimmten Anzahl von offenen Verbindungen das ganze System sehr zäh wird, CPU Auslastung aber trotzdem nie mehr wie wie 10% beträgt.
Meistens dümpelt die Auslastung zwischen 0% und 5%. Bei ca. 800 - 1000 verbundenen Clients (netstat -apn | grep 443.*ESTABLISHED | wc -l) nimmt der Server nur nach mehreren Sekunden neue Verbindungen an.
Auch der locale Verbindungsaufbau mittels openssl s_client -connect localhost:443 dauert über 10 Sekunden.
Laut nload habe ich einen Durschnittlichen Traffic von 500 Kilobyte/s rein und raus.
Ich habe schon sämtliche Werte im System hochgeschraubt, allerdings bin ich mir nicht sicher wie ich das Bottleneck sicher bestimmen kann.
Wenn ich den SSL Server neustarte, klappt die Verbindung in einem Sekundenbruchteil. Unter /var/log/ konnte ich den Logfiles keine Fehler erkennen.
Hat jemand eine Idee ?
Folgende Änderungen habe ich an der /etc/sysctl.conf angehängt, welche schon eine Besserung gebracht haben.
Code: Alles auswählen
# Use the full range of ports.
net.ipv4.ip_local_port_range = 1024 65535
# Maximum number of remembered connection requests, which are still did not receive an acknowledgment from connecting client. The default value is 1024 for systems with more than 128Mb of memory
net.ipv4.tcp_max_syn_backlog = 8192
# Limit of socket listen() backlog, known in userspace as SOMAXCONN, default is 128
net.core.somaxconn=8192
# max receive buffer for all type of connection
net.core.rmem_max=16777216
# max send buffer for all type of connection
net.core.wmem_max=16777216
# default receive buffer size for all type of connection
net.core.rmem_default=65536
# default send buffer size for all type of connection
net.core.wmem_default=65536
#The tcp_mem variable defines how the TCP stack should behave when it comes to memory usage. ...
#The first value specified in the tcp_mem variable tells the kernel the low threshold. Below this point, the TCP stack do not bother at all about putting any pressure on the memory usage by different TCP sockets. ...
#The second value tells the kernel at which point to start pressuring memory usage down. ...
#The final value tells the kernel how many memory pages it may use maximally. If this value is reached, TCP streams and packets start getting dropped until we reach a lower memory usage again. This value includes all TCP sockets currently in use."
net.ipv4.tcp_mem='16777216 16777216 16777216'
#The first value tells the kernel the minimum receive buffer for each TCP connection, and this buffer is always allocated to a TCP socket, even under high pressure on the system. ...
#The second value specified tells the kernel the default receive buffer allocated for each TCP socket. This value overrides the /proc/sys/net/core/rmem_default value used by other protocols. ...
#The third and last value specified in this variable specifies the maximum receive buffer that can be allocated for a TCP socket."
net.ipv4.tcp_rmem='4096 65536 16777216'
#This variable takes 3 different values which holds information on how much TCP sendbuffer memory space each TCP socket has to use. Every TCP socket has this much buffer space to use before the buffer is filled up. Each of the three values are used under different conditions. ...
#The first value in this variable tells the minimum TCP send buffer space available for a single TCP socket. ...
#The second value in the variable tells us the default buffer space allowed for a single TCP socket to use. ...
#The third value tells the kernel the maximum TCP send buffer space."
net.ipv4.tcp_wmem='4096 65536 16777216'
Code: Alles auswählen
Limit Soft Limit Hard Limit Units
Max cpu time unlimited unlimited seconds
Max file size unlimited unlimited bytes
Max data size unlimited unlimited bytes
Max stack size 8388608 unlimited bytes
Max core file size 0 unlimited bytes
Max resident set unlimited unlimited bytes
Max processes 15794 15794 processes
Max open files 65536 65536 files
Max locked memory 65536 65536 bytes
Max address space unlimited unlimited bytes
Max file locks unlimited unlimited locks
Max pending signals 15794 15794 signals
Max msgqueue size 819200 819200 bytes
Max nice priority 0 0
Max realtime priority 0 0
Max realtime timeout unlimited unlimited us
