What my application does is read data from kafka and hit another service via HTTP. I was seeing outgoing traffic slower in one box compare to others. I analysed tcpdump to that outgoing IP, logs from this box:
09:24:20.625288 IP (tos 0x0, ttl 64, id 16107, offset 0, flags [DF], proto TCP (6), length 7292)
localIP.57854 > externalIp.http: Flags [.], cksum 0x03fb (incorrect -> 0x614e), seq 52963:60203, ack 464, win 2518, options [nop,nop,TS val 205440553 ecr 262205407], length 7240: HTTP
09:24:20.640851 IP (tos 0x0, ttl 64, id 16112, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0xb2a7), seq 60203:63099, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.640897 IP (tos 0x0, ttl 64, id 16114, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0x46c8), seq 63099:65995, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.640930 IP (tos 0x0, ttl 64, id 16116, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0xedd7), seq 65995:68891, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.640940 IP (tos 0x0, ttl 64, id 16118, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0x22df), seq 68891:71787, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.640973 IP (tos 0x0, ttl 64, id 16120, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0x7fad), seq 71787:74683, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.641016 IP (tos 0x0, ttl 64, id 16122, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0x19e9), seq 74683:77579, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.641027 IP (tos 0x0, ttl 64, id 16124, offset 0, flags [DF], proto TCP (6), length 2948)
localIP.57854 > externalIp.http: Flags [.], cksum 0xf302 (incorrect -> 0xc26d), seq 77579:80475, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205422], length 2896: HTTP
09:24:20.644138 IP (tos 0x0, ttl 64, id 16132, offset 0, flags [DF], proto TCP (6), length 2223)
localIP.57854 > externalIp.http: Flags [P.], cksum 0xf02d (incorrect -> 0x6078), seq 89163:91334, ack 464, win 2518, options [nop,nop,TS val 205440557 ecr 262205425], length 2171: HTTP
09:24:20.660631 IP (tos 0x0, ttl 64, id 16134, offset 0, flags [DF], proto TCP (6), length 775)
localIP.57854 > externalIp.http: Flags [P.], cksum 0xea85 (incorrect -> 0x14c9), seq 90611:91334, ack 464, win 2518, options [nop,nop,TS val 205440562 ecr 262205426], length 723: HTTP
While at the same time in other box I see following:
09:26:53.610483 IP (tos 0x0, ttl 64, id 27441, offset 0, flags [DF], proto TCP (6), length 14532)
localIP.50978 > externalIp.http: Flags [.], cksum 0xcb4f (incorrect -> 0x3b5c), seq 151537:166017, ack 1390, win 1444, options [nop,nop,TS val 1613152807 ecr 262243666], length 14480: HTTP
09:26:53.610609 IP (tos 0x0, ttl 64, id 27451, offset 0, flags [DF], proto TCP (6), length 16713)
localIP.50978 > externalIp.http: Flags [P.], cksum 0xd3d4 (incorrect -> 0xed92), seq 166017:182678, ack 1390, win 1444, options [nop,nop,TS val 1613152807 ecr 262243668], length 16661: HTTP
09:26:53.632437 IP (tos 0x0, ttl 64, id 53481, offset 0, flags [DF], proto TCP (6), length 52)
localIP.51054 > externalIp.http: Flags [.], cksum 0x92bf (incorrect -> 0x5bcc), ack 464, win 1444, options [nop,nop,TS val 1613152812 ecr 262243674], length 0
09:26:53.638408 IP (tos 0x0, ttl 64, id 2460, offset 0, flags [DF], proto TCP (6), length 11636)
localIP.50892 > externalIp.http: Flags [.], cksum 0xbfff (incorrect -> 0x9468), seq 91408:102992, ack 927, win 1444, options [nop,nop,TS val 1613152814 ecr 262243675], length 11584: HTTP, length: 11584
I see big difference in field: length, while in first case it is quite small, while in later case it is large and whole data is being transferred very quickly. How is this length field determined, what factor impacts this?
TCP offload engine
) and operating systems you're using? – mrc02_kr Oct 11 '18 at 07:21