[one-liner]: Monitoring Network Traffic with Netstat on Linux

Background

This is a quick guide to some not so obvious command-line options available in the Linux netstat(8) command. I hope you enjoy it.

Solution

1. Plain old netstat

Without any command-line arguments, “netstat” shows a list of network connections in your system, including TCP, UDP and UNIX socket connections. If you want to speed up things a bit, use “-n” (numeric) to prevent network lookups and display IP and port numbers instead of names.

1
2
3
4
5
6
7
8
9
10
11
12
% netstat -n
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 192.168.1.20:41157          173.194.73.104:443          TIME_WAIT   
tcp        0      0 192.168.1.20:57224          67.253.170.83:5222          ESTABLISHED 
...
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ]         DGRAM                    7590   @/org/kernel/udev/udevd
unix  2      [ ]         DGRAM                    11188  @/org/freedesktop/hal/udev_event
unix  22     [ ]         DGRAM                    10175  /dev/log
...

2. Seeing all connections with “netstat -a”

Just like the above, but shows all connections, including those in the LISTEN state. Good when you want to see all connections in one shot.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
% netstat -an
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:2143              0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:1986              0.0.0.0:*                   LISTEN      
tcp        0      0 127.0.0.1:2025              0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:59081               0.0.0.0:*                   LISTEN      
tcp        0      0 192.168.1.20:57224          67.253.170.83:5222          ESTABLISHED 
...
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  /var/run/cups/cups.sock
...

3. What I am listening to? “netstat -l”

Many people type “netstat -a | grep -i LISTEN“, but “netstat -l” will do the same: filter the output to show sockets in the LISTEN state only. Very useful to quickly see what is being “served” in your box. You can combine this with “-u” to only show UDP connections or “-t” to restrict the output to TCP connections only.

1
2
3
4
5
6
7
8
9
10
11
12
13
% netstat -ln
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
...

4. Who’s using that socket? “netstat -p”

With “-p”, netstat shows what program/pid is using a given socket. Very handy to find out who’s listening on a port or holding a connection open. A personal favorite of mine is “netstat -lput”, which displays all TCP and UDP sockets in the LISTEN state, plus the name and pid of the program listening on that socket.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
% netstat -lnp
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:111                 0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:22                  0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:631                 0.0.0.0:*                   LISTEN      -                   
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      4185/dropbox        
tcp        0      0 127.0.0.1:2143              0.0.0.0:*                   LISTEN      31544/ssh           
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node PID/Program name    Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 4185/dropbox        /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 4185/dropbox        /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  -                   /var/run/cups/cups.sock
...
1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
% netstat -lput
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 *:sunrpc                    *:*                         LISTEN      1403/rpcbind        
tcp        0      0 *:ssh                       *:*                         LISTEN      1753/sshd           
tcp        0      0 *:ipp                       *:*                         LISTEN      1578/cupsd          
tcp        0      0 *:db-lsp                    *:*                         LISTEN      4185/dropbox        
tcp        0      0 localhost.localdomain:lv-jc *:*                         LISTEN      31544/ssh           
tcp        0      0 localhost.loc:licensedaemon *:*                         LISTEN      2764/mono           
tcp        0      0 localhost.localdoma:ellpack *:*                         LISTEN      31544/ssh           
tcp        0      0 *:59081                     *:*                         LISTEN      1508/rpc.statd      
tcp        0      0 *:sunrpc                    *:*                         LISTEN      1403/rpcbind        
tcp        0      0 *:ssh                       *:*                         LISTEN      1753/sshd           
tcp        0      0 *:ipp                       *:*                         LISTEN      1578/cupsd          
tcp        0      0 *:46136                     *:*                         LISTEN      1508/rpc.statd      
tcp        0      0 localhost.localdomain:lv-jc *:*                         LISTEN      31544/ssh           
tcp        0      0 localhost.localdoma:ellpack *:*                         LISTEN      31544/ssh           
udp        0      0 *:bootpc                    *:*                                     30569/dhclient      
udp        0      0 *:sunrpc                    *:*                                     1403/rpcbind        
udp        0      0 greeneggs.bubba.net:ntp     *:*                                     30725/ntpd          
udp        0      0 localhost.localdomain:ntp   *:*                                     30725/ntpd          
udp        0      0 *:ntp                       *:*                                     30725/ntpd          
udp        0      0 *:32944                     *:*                                     1480/avahi-daemon:  
udp        0      0 *:ipp                       *:*                                     1578/cupsd          
udp        0      0 *:727                       *:*                                     1403/rpcbind        
udp        0      0 *:836                       *:*                                     1508/rpc.statd      
udp        0      0 *:db-lsp-disc               *:*                                     4185/dropbox        
udp        0      0 *:mdns                      *:*                                     1480/avahi-daemon:  
udp        0      0 *:30320                     *:*                                     30569/dhclient      
udp        0      0 *:60010                     *:*                                     1508/rpc.statd      
udp        0      0 *:sunrpc                    *:*                                     1403/rpcbind        
udp        0      0 localhost.localdomain:ntp   *:*                                     30725/ntpd          
udp        0      0 fe80::226:c7ff:fe85:ntp     *:*                                     30725/ntpd          
udp        0      0 *:ntp                       *:*                                     30725/ntpd          
udp        0      0 *:49602                     *:*                                     1508/rpc.statd      
udp        0      0 *:727                       *:*                                     1403/rpcbind        
udp        0      0 *:6095                      *:*                                     30569/dhclient

5. Details, lots of details: “netstat -e”

If you really want to see what’s going on, add the “-e” command-line switch to your netstat command. It will cause “extra” information to be printed, such as the username, process name and pid, and the inode number of each of your sockets.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
% netstat -le
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       User       Inode     
tcp        0      0 *:sunrpc                    *:*                         LISTEN      root       10342      
tcp        0      0 *:ssh                       *:*                         LISTEN      root       11714      
tcp        0      0 *:ipp                       *:*                         LISTEN      root       11049      
tcp        0      0 *:db-lsp                    *:*                         LISTEN      saml       101643     
tcp        0      0 localhost.localdomain:lv-jc *:*                         LISTEN      saml       1068913    
...
Active UNIX domain sockets (only servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  /var/run/cups/cups.sock
unix  2      [ ACC ]     STREAM     LISTENING     14932  /tmp/keyring-7S8f2X/control
unix  2      [ ACC ]     STREAM     LISTENING     15153  @/tmp/dbus-HqIwhBFTxL
...

6. Dial “o” for obscure: “netstat -o”

The “-o” command-line option causes the TCP timers to be displayed next to the connection. If you’re not into TCP/IP, the extra information will not make sense. For those who are, you’ll be able to see your TCP timers in real-time, and follow the progress of things like the KeepAlive timer, for instance. For a real treat, use “watch netstat -to”, sit down and watch the blinkenlights.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
% netstat -oa
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       Timer
tcp        0      0 *:ssh                       *:*                         LISTEN      off (0.00/0/0)
tcp        0      0 *:ipp                       *:*                         LISTEN      off (0.00/0/0)
tcp        0      0 *:59081                     *:*                         LISTEN      off (0.00/0/0)
tcp        0      0 greeneggs.bubba.net:59261   iad23s07-in-f7.1e100.n:http ESTABLISHED keepalive (11.92/0/0)
tcp        0      0 greeneggs.bubba.net:57224   cpe-67-253-170-:xmpp-client ESTABLISHED off (0.00/0/0)
tcp        0      0 greeneggs.bubba.net:57226   cpe-67-253-170-:xmpp-client ESTABLISHED off (0.00/0/0)
...
Active UNIX domain sockets (servers and established)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ACC ]     STREAM     LISTENING     101544 /home/saml/.dropbox/command_socket
unix  2      [ ACC ]     STREAM     LISTENING     101549 /home/saml/.dropbox/iface_socket
unix  2      [ ACC ]     STREAM     LISTENING     11051  /var/run/cups/cups.sock
unix  2      [ ACC ]     STREAM     LISTENING     14932  /tmp/keyring-7S8f2X/control
...

7. Interface status with “netstat -i”

Shows the status of your interfaces. Very handy to make sure everything is going smooth on the hardware side of things. Keep an eye on the ERR, DRP and OVR counters, as they can indicate trouble (10/100 mismatch anyone?).

1
2
3
4
5
6
% netstat -i
Kernel Interface table
Iface       MTU Met    RX-OK RX-ERR RX-DRP RX-OVR    TX-OK TX-ERR TX-DRP TX-OVR Flg
eth0       1500   0        0      0      0      0        0      0      0      0 BMU
lo        16436   0    33951      0      0      0    33951      0      0      0 LRU
wlan0      1500   0   317576      0      0      0   269995      0      0      0 BMRU

8. Continuous display: “netstat -c”

Makes netstat display output continuously. This command-line option can be used with any other form of netstat. I’d suggest using the “watch” command instead.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
% netstat -c
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State      
tcp        0      0 greeneggs.bubba.net:59261   iad23s07-in-f7.1e100.n:http ESTABLISHED 
tcp        0      0 greeneggs.bubba.net:57224   cpe-67-253-170-:xmpp-client ESTABLISHED 
tcp        0      0 greeneggs.bubba.net:57226   cpe-67-253-170-:xmpp-client ESTABLISHED 
tcp        0      0 greeneggs.bubba.net:43021   qc-in-f125.1e10:xmpp-client ESTABLISHED 
...
Active UNIX domain sockets (w/o servers)
Proto RefCnt Flags       Type       State         I-Node Path
unix  2      [ ]         DGRAM                    7590   @/org/kernel/udev/udevd
unix  2      [ ]         DGRAM                    11188  @/org/freedesktop/hal/udev_event
unix  22     [ ]         DGRAM                    10175  /dev/log
unix  3      [ ]         STREAM     CONNECTED     1324835 
...

9. TCP/IP statistics: “netstat -s”

Shows statistics about your TCP/IP stack. Requires deeper knowledge of the protocols to make some sense of it, but can be used by anyone to impress (scare?) friends, relatives and prospective girlfriends.

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
19
20
21
22
23
24
25
26
27
28
29
30
31
32
33
34
35
36
37
38
39
40
41
42
43
% netstat -s
Ip:
    303121 total packets received
    6 with invalid addresses
    0 forwarded
    0 incoming packets discarded
    281703 incoming packets delivered
    268905 requests sent out
    32 outgoing packets dropped
    640 dropped because of missing route
Icmp:
    2775 ICMP messages received
    1698 input ICMP message failed.
    ICMP input histogram:
        destination unreachable: 2774
        echo requests: 1
    2809 ICMP messages sent
    0 ICMP messages failed
    ICMP output histogram:
        destination unreachable: 2808
        echo replies: 1
IcmpMsg:
        InType3: 2774
        InType8: 1
        OutType0: 1
        OutType3: 2808
Tcp:
    9122 active connections openings
    17 passive connection openings
    2084 failed connection attempts
    916 connection resets received
    50 connections established
    289775 segments received
    274830 segments send out
    3049 segments retransmited
    0 bad segments received.
    2092 resets sent
Udp:
    18981 packets received
    233 packets to unknown port received.
    0 packet receive errors
    18608 packets sent
...

10. Boooooring: “netstat -r”

Use “ip route” instead of “netstat -r” to look cool and hip, and say “netstat is sooo 1975″ when somebody asks why you’re doing that.

1
2
3
4
5
% netstat -r
Kernel IP routing table
Destination     Gateway         Genmask         Flags   MSS Window  irtt Iface
192.168.1.0     *               255.255.255.0   U         0 0          0 wlan0
default         router1.bubba.n 0.0.0.0         UG        0 0          0 wlan0

11. Watching TCP connections

1
2
3
4
5
6
7
8
9
10
11
12
13
14
% watch "netstat -ton"
Every 2.0s: netstat -ton                                                                                                                                                          Fri Oct  5 23:58:58 2012
 
Active Internet connections (w/o servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State	Timer
tcp        0	  0 192.168.1.20:57224          67.253.170.83:5222          ESTABLISHED off (0.00/0/0)
tcp        0	  0 192.168.1.20:57226          67.253.170.83:5222          ESTABLISHED off (0.00/0/0)
tcp        0	  0 192.168.1.20:43021          173.194.76.125:5222         ESTABLISHED off (0.00/0/0)
tcp        0	  0 192.168.1.20:57275          67.228.181.220:80           TIME_WAIT   timewait (29.92/0/0)
tcp        0	  0 192.168.1.20:43024          173.194.76.125:5222         ESTABLISHED keepalive (31.78/0/0)
tcp        0	  0 192.168.1.20:47987          64.34.119.101:80            ESTABLISHED keepalive (1.31/0/0)
tcp        0	  0 192.168.1.20:48377          217.168.150.39:6667         ESTABLISHED off (0.00/0/0)
tcp        0	  0 192.168.1.20:42857          72.247.188.20:80            TIME_WAIT   timewait (49.02/0/0)
...

12. Combining switches (All TCP & UDP traffic, numerically, listening, with process ids)

1
2
3
4
5
6
7
8
9
10
11
12
13
14
15
16
17
18
% netstat -tulnp
(Not all processes could be identified, non-owned process info
 will not be shown, you would have to be root to see it all.)
Active Internet connections (only servers)
Proto Recv-Q Send-Q Local Address               Foreign Address             State       PID/Program name   
tcp        0      0 0.0.0.0:17500               0.0.0.0:*                   LISTEN      4185/dropbox        
tcp        0      0 127.0.0.1:2143              0.0.0.0:*                   LISTEN      31734/ssh           
tcp        0      0 127.0.0.1:1986              0.0.0.0:*                   LISTEN      2764/mono           
tcp        0      0 127.0.0.1:2025              0.0.0.0:*                   LISTEN      31734/ssh           
tcp        0      0 ::1:2143                    :::*                        LISTEN      31734/ssh           
tcp        0      0 ::1:2025                    :::*                        LISTEN      31734/ssh           
...
udp        0      0 0.0.0.0:111                 0.0.0.0:*                               -                   
udp        0      0 0.0.0.0:631                 0.0.0.0:*                               -                   
udp        0      0 0.0.0.0:727                 0.0.0.0:*                               -                   
udp        0      0 0.0.0.0:836                 0.0.0.0:*                               -                   
udp        0      0 0.0.0.0:17500               0.0.0.0:*                               4185/dropbox        
...

References

links

NOTE: For further details regarding my one-liner blog posts, check out my one-liner style guide primer.

This entry was posted in linux, monitoring, one-liner, redhat, rhel, Syndicated, sysadmin, tips & tricks. Bookmark the permalink.

Comments are closed.