I have a zte router with Linux kernel 4.1.25 which I have installed an snmp agent (mini-snmpd) on it to control traffic.
As I was getting inaccurate data I investigated a bit and detected that ifconfig and /proc/net/dev both are giving wrong data, while zte programs (cspd/httpd) are providing correct values.
ifconfig returns this info for eth4 which is the interface Im plugged in:
RX bytes: 177365531 (161.1 MiB) TX bytes: 12507777123 (11.6Gib)
router webpage is showing a much larger and, I believe, correct values for this interface:
Bytes Received/Sent 100655800471/286284079917
If I do a speedtest-cli from my computer I can see the web counters increase according to the transmited data speedtest-cli reports that has been transferred (2/3GB in/out). However ifconfig barely moves, sometimes it increases +300MB, sometimes +100MB.
Speedtest by Ookla
Server: xxxxxx (id = 14979)
ISP: xxxxxx
Latency: 1.93 ms (0.12 ms jitter)
Download: 2339.50 Mbps (data used: 2.9 GB )
Upload: 2339.21 Mbps (data used: 2.1 GB )
Packet Loss: 0.0%
Could be there a reason of why kernel is not getting accurated data and however the program from zte does ?
Edit:
Well I believe I've found the cause, not a solution, if there is any.
There is some tunables in /proc/zte/sys/ffe
If I do: echo npu 0 > /proc/zte/sys/ffe/cmd
I start seeing good network graphs, but incoming traffic seems to be capped at about 1,7Gbps (I was obtaining ~2,35Gbps before)
If I do: echo ffe 0 > /proc/zte/sys/ffe/cmd
Download and upload speed dramatically fall to about 300mbps each.
I've been investigating about what npu and ffe stands for, and they appear to be some hardware network accelerators (Network Processing Unit and Fast Forward Engine).
Still I dont know why /proc/net/dev is not getting noticed of the total traffic passing through the npu and however a program in userspace is, and if this can be fixed somehow. There are some no_delete,timer,flowctrl options which im gonna play a bit with them to see if it is able to update net/dev file.
Edit 2:
Well, after investigating a bit more I've used strace on cspd process and by requesting eth stats on the router web I can see a write() of what looks like a call on cspd process function CmEthGetPortBasicStats(). Looking into cspd binary with ghidra I see that there is a fopen() of the file /dev/switch_dev, followed by an ioctl with value 0x38 (56) when it ask for basic stats, and other values for different things like setting speed,duplex, interface on/off... I've tried to create an ioctl program to retrieve the data from switch_dev. No sucess yet, only receiving a few bytes.
Edit 3:
Ok, so cspd binary has two functions:
https://i.stack.imgur.com/4OltN.png
This one is the function that does exactly what I wanted, it receives stats of all interfaces at once. This function is called when I check eth stats from the router webpage.
https://i.stack.imgur.com/gfl1x.png
And this one does the same but it does per interface and I've noticed that it is called when I run this command in console: sendcmd 1 switch_mgr getPortStats [iface number]
So despite not being able yet to directly interact with /dev/switch_mgr I've patched the mini-snmpd source code to generate a fake /proc/net/dev with added data from the sendcmd command, which mini-snmpd parses instead of the original to finally achieve what I was looking for:
I know the solution its not perfect, as Im changing an ioctl() call for 5 popen("sendcmd 1 switch_mgr...") but I dont really see any noticeable overhead due to that, and it provides correct numbers to mini-snmpd. Obviously this doesnt fix ifconfig as im creating a fake /proc/net/dev on /var/tmp/dev-fake, which ifconfig does not feed from that. Maybe I will compile a new static ifconfig later, but not a priority for me at this point.
For me this is enough, but I still would like some help if someone wants to help me to directly interact with /dev/switch_dev
Ethernet Port LAN5 Status Up/2500Mbps/Full Duplex
– Alex Aug 12 '22 at 17:37