首页 > 其他 > 详细

线上问题: ifconfig 出现drop

时间:2021-01-29 23:42:58      阅读:63      评论:0      收藏:0      [点我收藏+]

技术分享图片

目前准备通过  

netstat -s    ethtool -S  cat /proc/net/dev   cat /proc/net/snmp 

cat  /sys/class/net/<NIC>/statistics/

查看drop 统计

同时通过sar -n DEV 1 5 查看流量

tcpdump 抓包分析 报文特征

已经交给前场提取数据

其实想用dropwatch分析但是 现场不支持

技术分享图片
1、首先内核必须大于等于2.6.302、编译内核时应该加上“NET_DROP_MONITOR=y”;
View Code

明天处理!!!!!

 

目前收集的信息中有比较重要有用的信息为:

     rx_no_buffer_count: 180972127
     rx_missed_errors: 127669376
技术分享图片
root@localhost / # cat /proc/net/snmp        
Ip: Forwarding DefaultTTL InReceives InHdrErrors InAddrErrors ForwDatagrams InUnknownProtos InDiscards InDelivers OutRequests OutDiscards OutNoRoutes ReasmTimeout ReasmReqds ReasmOKs ReasmFails FragOKs FragFails FragCreates
Ip: 2 64 43266916579 0 58 0 0 0 43266916521 39496151188 19 1096 0 0 0 0 687465 0 1577650
Icmp: InMsgs InErrors InDestUnreachs InTimeExcds InParmProbs InSrcQuenchs InRedirects InEchos InEchoReps InTimestamps InTimestampReps InAddrMasks InAddrMaskReps OutMsgs OutErrors OutDestUnreachs OutTimeExcds OutParmProbs OutSrcQuenchs OutRedirects OutEchos OutEchoReps OutTimestamps OutTimestampReps OutAddrMasks OutAddrMaskReps
Icmp: 10609 83 10226 217 0 0 0 151 15 0 0 0 0 43171445 0 43171273 0 0 0 0 26 146 0 0 0 0
IcmpMsg: InType0 InType3 InType8 InType11 OutType0 OutType3 OutType8
IcmpMsg: 15 10226 151 217 146 43171273 26
Tcp: RtoAlgorithm RtoMin RtoMax MaxConn ActiveOpens PassiveOpens AttemptFails EstabResets CurrEstab InSegs OutSegs RetransSegs InErrs OutRsts
Tcp: 1 200 120000 -1 638924225 792069897 51631303 76843627 105 43070509878 47284276280 4631429 0 60909693
Udp: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
Udp: 195179897 532 1998 218221001 0 0
UdpLite: InDatagrams NoPorts InErrors OutDatagrams RcvbufErrors SndbufErrors
UdpLite: 0 0 0 0 0 0
root@localhost / # netstat -s
Ip:
    317249965 total packets received
    58 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    317249907 incoming packets delivered
    841450781 requests sent out
    19 outgoing packets dropped
    1096 dropped because of missing route
    687465 fragments received ok
    1577650 fragments created
Icmp:
    10609 ICMP messages received
    83 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 10226
        timeout in transit: 217
        echo requests: 151
        echo replies: 15
    43171452 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 43171280
        echo request: 26
        echo replies: 146
IcmpMsg:
        InType0: 15
        InType3: 10226
        InType8: 151
        InType11: 217
        OutType0: 146
        OutType3: 43171280
        OutType8: 26
Tcp:
    638924267 active connections openings
    792070011 passive connection openings
    51631325 failed connection attempts
    76843631 connection resets received
    98 connections established
    120843208 segments received
    39642604 segments send out
    4631433 segments retransmited
    0 bad segments received.
    60909716 resets sent
Udp:
    195179954 packets received
    532 packets to unknown port received.
    1998 packet receive errors
    218221066 packets sent
    0 receive buffer errors
    0 send buffer errors
UdpLite:
TcpExt:
    947 resets received for embryonic SYN_RECV sockets
    148679 packets pruned from receive queue because of socket buffer overrun
    531225716 TCP sockets finished time wait in fast timer
    347738973 delayed acks sent
    2698538 delayed acks further delayed because of locked socket
    Quick ack mode was activated 2595825 times
    28481169 packets directly queued to recvmsg prequeue.
    6717820 bytes directly in process context from backlog
    1033332179 bytes directly received in process context from prequeue
    1010153350 packet headers predicted
    14758935 packets header predicted and directly queued to user
    3242078974 acknowledgments not containing data payload received
    3880126246 predicted acknowledgments
    1536 times recovered from packet loss due to fast retransmit
    13263 times recovered from packet loss by selective acknowledgements
    1733 congestion windows recovered without slow start by DSACK
    136942 congestion windows recovered without slow start after partial ack
    16400 TCP data loss events
    TCPLostRetransmit: 639
    1574 timeouts after reno fast retransmit
    14245 timeouts after SACK recovery
    78 timeouts in loss state
    50060 fast retransmits
    4432 forward retransmits
    16193 retransmits in slow start
    4159092 other TCP timeouts
    86 classic Reno fast retransmits failed
    552 SACK retransmits failed
    22768427 packets collapsed in receive queue due to low socket buffer
    3110543 DSACKs sent for old packets
    57636 DSACKs sent for out of order packets
    34337 DSACKs received
    9 DSACKs for out of order packets received
    4438261 connections reset due to unexpected data
    4890686 connections reset due to early user close
    21201 connections aborted due to timeout
    TCPSACKDiscard: 2
    TCPDSACKIgnoredOld: 22762
    TCPDSACKIgnoredNoUndo: 7791
    TCPSpuriousRTOs: 1396
    TCPSackShifted: 42109
    TCPSackMerged: 43323
    TCPSackShiftFallback: 63448
    TCPBacklogDrop: 1435
    TCPDeferAcceptDrop: 139642
    TCPTimeWaitOverflow: 171130
IpExt:
    InMcastPkts: 5
    InBcastPkts: 1213607
    InOctets: 1673780726
    OutOctets: -1198119745
    InMcastOctets: 140
    InBcastOctets: 104766229
root@localhost / # ethtool -S eth1
NIC statistics:
     rx_packets: 27862085998
     tx_packets: 1262711762
     rx_bytes: 2039732992330
     tx_bytes: 125303834249
     rx_broadcast: 35401078
     tx_broadcast: 26016705
     rx_multicast: 54576108
     tx_multicast: 808459
     rx_errors: 0
     tx_errors: 0
     tx_dropped: 0
     multicast: 54576108
     collisions: 0
     rx_length_errors: 0
     rx_over_errors: 0
     rx_crc_errors: 0
     rx_frame_errors: 0
     rx_no_buffer_count: 1571897
     rx_missed_errors: 1390
     tx_aborted_errors: 0
     tx_carrier_errors: 0
     tx_fifo_errors: 0
     tx_heartbeat_errors: 0
     tx_window_errors: 0
     tx_abort_late_coll: 0
     tx_deferred_ok: 0
     tx_single_coll_ok: 0
     tx_multi_coll_ok: 0
     tx_timeout_count: 0
     tx_restart_queue: 0
     rx_long_length_errors: 0
     rx_short_length_errors: 0
     rx_align_errors: 0
     tx_tcp_seg_good: 0
     tx_tcp_seg_failed: 0
     rx_flow_control_xon: 0
     rx_flow_control_xoff: 0
     tx_flow_control_xon: 220291
     tx_flow_control_xoff: 221680
     rx_long_byte_count: 2039732992330
     rx_csum_offload_good: 27739152135
     rx_csum_offload_errors: 0
     rx_header_split: 0
     alloc_rx_buff_failed: 0
     tx_smbus: 0
     rx_smbus: 0
     dropped_smbus: 0
     rx_dma_failed: 0
     tx_dma_failed: 0
root@localhost / # ethtool -S eth2
NIC statistics:
     rx_packets: 1261353932
     tx_packets: 27862955528
     rx_bytes: 125372925112
     tx_bytes: 2040275968171
     rx_broadcast: 25614225
     tx_broadcast: 35423396
     rx_multicast: 808438
     tx_multicast: 54576132
     rx_errors: 0
     tx_errors: 0
     tx_dropped: 0
     multicast: 808438
     collisions: 0
     rx_length_errors: 0
     rx_over_errors: 0
     rx_crc_errors: 0
     rx_frame_errors: 0
     rx_no_buffer_count: 180972127
     rx_missed_errors: 127669376
     tx_aborted_errors: 0
     tx_carrier_errors: 0
     tx_fifo_errors: 0
     tx_heartbeat_errors: 0
     tx_window_errors: 0
     tx_abort_late_coll: 0
     tx_deferred_ok: 0
     tx_single_coll_ok: 0
     tx_multi_coll_ok: 0
     tx_timeout_count: 0
     tx_restart_queue: 0
     rx_long_length_errors: 0
     rx_short_length_errors: 0
     rx_align_errors: 0
     tx_tcp_seg_good: 0
     tx_tcp_seg_failed: 0
     rx_flow_control_xon: 0
     rx_flow_control_xoff: 0
     tx_flow_control_xon: 0
     tx_flow_control_xoff: 0
     rx_long_byte_count: 125372925112
     rx_csum_offload_good: 1228845122
     rx_csum_offload_errors: 10
     rx_header_split: 0
     alloc_rx_buff_failed: 0
     tx_smbus: 0
     rx_smbus: 0
     dropped_smbus: 0
     rx_dma_failed: 0
     tx_dma_failed: 0
root@localhost / # cat /proc/net/dev
Inter-|   Receive                                                |  Transmit
 face |bytes    packets errs drop fifo frame compressed multicast|bytes    packets errs drop fifo colls carrier compressed
    lo: 384657885396 1478036246    0    0    0     0          0         0 384657885396 1478036246    0    0    0     0       0          0
 bond0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  eth0: 1820898757 15863793    0    0    0     0          0         0 26863219776 242854331    0    0    0     0       0          0
 teql0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  sit0:       0       0    0    0    0     0          0         0        0       0    0    0    0     0       0          0
  eth1: 2039734404414 27862095641    0 1390    0     0          0  54576108 125317855145 1262724651    0    0    0     0       0          0
  eth2: 125385953071 1261365428    0 127669376    0     0          0    808463 2040277248858 27862964224    0    0    0     0       0          0
  eth3: 27566182128841 397592489315    0    0    0     0          0    769327 70776340796697 437380843982    0    0    0     0       0          0
  eth4: 71536212472219 436926157198    0 14415212551    0     0          0  54574179 27741862259957 401183191271    0    0    0     0       0          0
  eth5: 39944246780346 43763462640    0 2456    0     0          0       277 35696516892117 36611633993    0    0    0     0       0          0
  eth6: 35697179600486 36611633978    0    0    0     0          0  76995504 39944845468824 43763451889    0    0    0     0       0          0
eth1.1:       0       0    0    0    0     0          0         0     1026      13    0    0    0     0       0          0
eth2.1:       0       0    0    0    0     0          0         0      936      12    0    0    0     0       0          0
eth3.1:       0       0    0    0    0     0          0         0      936      12    0    0    0     0       0          0
eth4.1:       0       0    0    0    0     0          0         0      558       7    0    0    0     0       0          0
eth5.1:       0       0    0    0    0     0          0         0     1494      19    0    0    0     0       0          0
eth6.1:       0       0    0    0    0     0          0         0      468       6    0    0    0     0       0          0
View Code

查看驱动代码看下 rx_no_buffer_count 以及rx_missed_errors 是怎么来的吧

mpc = E1000_READ_REG(hw, E1000_MPC);

adapter->stats.mpc += mpc;

IGB_STAT("rx_missed_errors", stats.mpc),
IGB_STAT("rx_no_buffer_count", stats.rnbc),
 adapter->stats.rnbc += E1000_READ_REG(hw, E1000_RNBC);

这几个参数都是从 芯片寄存器读取出来的:

大概意思是;

rx_no_buffer_count = E1000_RNBC
rx_missed_error = E1000_MPC

在网上收了一下 RNBC 以及MPC 得到了如下一段话:

 Missed Packets Count – MPC 

  Counts the number of missed packets. Packets are missed when the receive FIFO has insufficient space to store the incoming packet. This can be caused because of too few buffers allocated, or because there
is insufficient bandwidth on the PCI bus. Events setting this counter causes ICR.Rx Miss, the Receiver Overrun Interrupt, to be set. This register does not increment if receives are not enabled.
These packets are also counted in the Total Packets Received register as well as in Total Octets Received.

rx_missed_errors与硬中断有关。也就是在DMA传送完,发送硬中断之前,网卡的FIFO缓冲已经满了,导致接收的数据要立即丢掉;

Receive No Buffers Count – RNBC
  This register counts the number of times that frames were received when there were no available buffers in host memory to store those frames (receive descriptor head and tail pointers were equal).
The packet is still received if there is space in the FIFO. This register only increments if receives are enabled (RCTL.RXEN is set). This register does not increment when flow control packets are received.

rx_no_buffer_count 应该是指在网卡通过DMA将设备FIFO中的skb->data传送到rx_buffer_info时,发现对应的rx_buffer_info还没有unmap,也就无法送到内核memory主存。也就是 这其实和软中断处理的速度有关

 

技术分享图片

 

 参考:https://lp007819.wordpress.com/2013/05/

 

线上问题: ifconfig 出现drop

原文:https://www.cnblogs.com/codestack/p/14341363.html

(0)
(0)
   
举报
评论 一句话评论(0
关于我们 - 联系我们 - 留言反馈 - 联系我们:wmxa8@hotmail.com
© 2014 bubuko.com 版权所有
打开技术之扣,分享程序人生!