| NFDUMP(1) | General Commands Manual | NFDUMP(1) |
nfdump — flow
display and analysis program
nfdump |
-r flowpath
[-w outfile]
[-f filterfile]
[-C config]
[-R filelist]
[-M dirlist]
[-O order]
[-t timewin]
[-c num]
[-a] [-A
aggregation] [-b]
[-B] [-I]
[-D nameserver]
[-G geoDB]
[-s statistic]
[-n num]
[-o format]
[-6] [-q]
[-N] [-i
ident] [-v
flowfile] [-E
flowfile] [-x
flowfile]
[-z=<compress>] [-J
compress] [-X]
[-Z] [-T]
[-V] [-h] [filter] |
nfdump reads the flow data from one or
more binary files, created by any nfdump collector nfcapd,
nfpcapd and sfcapd. It processes and lists the
flows in many different output formats and can create a wide range of
statistics.
nfdump has a very powerful flow filter to
process flows. The filter syntax is very similar to tcpdump, but adapted and
extended for flow filtering. A flow filter may also contain arrays of many
thousand IP addresses etc. to search for specific records.
nfdump can aggreagte flows according to a
user defined number of elements. This masks certain elements and allows to
sum up flow records matching the same values.
The combination of flow filtering and aggregation as input for any
flow statistics allows complex flow processing. Pre-filtered and aggregated
flow data may also be written back into a binary flow file, which again may
be processed with nfdump
nfdump can enrich the listing of flows
with geo location information and AS information, unless AS information is
already available in the flow records. IP addresses can be tagged with a two
letter country code, or with a longer location label containing the
geographic region, country and city. The geo location and AS information is
retrieved from the optional geoDB database, created by
the geolookup program from the nfdump tools.
geolookup uses the Maxmind
database GeoDB or GeoLite2 to
create a binary lookup database for nfdump Please
check the geolooup(1) man page for more details.
The options are as follows:
-r
flowpath-w
outfilenfdump This can be useful
to limit flows according to a flow filter and/or specific flow
aggregation.-f
filterfile-C
confignfdump tries to read by default
%prefix/etc/nfdump.config. This may be overwritten
by the environment valiable NFCONF which again may
be overwritten by this option -C. In order to
prevent reading any config file, even if it would exist set
-C none. A config file is not
required, but may be handy for often used output formats etc.-O
orderflowspacketsipkgopkgbytesibyteobyteppsippsoppsbpsibpsobpsbppibppobpptstarttendduration-t
timewin-c
num-a-A-A
aggregationprotosrcipdstipsrcip4/netsrcip6/netdstip4/netdstip6/netsrcnetdstnetsrcportdstportsrcmaskdstmasksrcvlandstvlansrcasdstasnextasprevasinifoutifnextbgpnextinsrcmacoutdstmacindstmacoutsrcmactossrctosdsttosmpls1mpls2mpls3mpls4mpls5mpls6mpls7mpls8mpls9mpls10routerodidopidxsrcipxdstipxsrcportxdstportnfdump automatically compiles the
appropriate output format for the selected aggregation elements unless
an explicit output format -o is given. The
automatic output format is identical to
-o 'fmt:%ts %td <fields>
%pkt %byt %bps %bpp %fl'where <fields> represents the selected aggregation tags.
-b-o format option.-B-b but tries to guess the
correct client to server direction. Automagically swaps flows if src port
is < dst port for TCP and UDP flows and src port < 1024 and dst port
> 1024. Some exporters do not really care sending the flows in proper
order. It's considered to be a conveniency option.-I-r
flowpath.-g-r
flowpath a one line summary, which can be easily
used by gnu plot.-D
nameserver-G
geoDBnfdump tries to read the environment
variable NFGEODB for the path of
geoDB. The option -G
overwrites NFGEODB. In order to prevent reading any
geoDB file, even if it would exist set
-G none.-s
statistic [:p
[/orderby]]statistic can be:
recordsrcipdstipipnhipnhbiproutersrcportdstportporttossrctosdsttosdirsrcasdstassrcgeodstgeoasinifoutififinamonamsrcmaskdstmasksrcvlandstvlanvlaninsrcmacoutdstmacindstmacoutsrcmacsrcmacdstmacinmacoutmacmaskprotompls1mpls2mpls3mpls4mpls5mpls6mpls7mpls8mpls9mpls10sysidnbarja3odidopidvrf/ivrfevrfivrfnamevrfnamNSEL/ASA statistics
eventxeventxsrcipxsrcportxdstipxdstportiacliaceixaceeacleaceexaceNAT statistics
neventnsrcipnsrcportndstipndstport% nfdump -s srcip -s
ip/flows/bytes -s record/bytes-n
num-s statistics as well as to
ordered output -O -or
-aggregated -records
-a The default is set to 10 for statistics and
unlimited for the other use cases. To disable the limit, set
num to 0.-o
formathas many
different output formats already predefined. format
may be one of the options below:
rawfmt:
userjsoncsvpipeAlready predefined fmt formats:
linelongbilinebilongglineglongextendednselnelThe nfdump config file may contain
additional formats. If you want to add new formats or change existing
ones, check the config file.
IPv6 addresses are printed condensed in any
fmt defined
format to prevent cluttering the output with large blank blocks. A
condensed IPV6 uses max 16 characters. If it is longer, then the middle
part of the IP is cut out and replaced be "..". For previewing
an output, this fits most needs. For a listing with the full IPV6
addresses add option -6.
-6-q-N-i
ident-r to ident-v
flowfile-E
flowfile-x
flowfile-z=lzo-z=lz4-z=bz2-J
compress-r flowpath Set
compress to 0 for no compression or to any of: 1 or
LZO, 2 or BZ2, 3 or LZ4. This option may be used for archiving flow files
and changing the compression to use less disk space.-X-Z-R
filelist-M
dirlist-T-Vnfdump version and exit.-hfilter selects, which records will be further processed. If no filter is given, all records will be processed. Otherwise, only those flows matching the filter will be processed. Any IP address in a filter may be specified as IPv4 or IPv6.
The filter syntax is similar to tcpdump but adapted and extended for flow records. The filter can be either specified on the command line after all options or in a separate file. It can span several lines. Anything after a '#' is treated as a comment and ignored to the end of the line. There is virtually no limit in the length of the filter expression. All keywords are case insensitive.
A single filter primitive filters a single element of a flow record. A filter consists of one or more primitives, which are linked together:
expr
and exprexpr
or
exprnot
expr and
(expr)Possible filter primitives:
@include
filecount
comp numberident
stringinetipv4inet6ipv6proto
protocoltcp,
udp, icmp,
ah, esp,
ipip, and many more or a protocol number, such as
6, 17 for protocol
tcp and
udp.
tun
proto protocolip
ipaddrsrc
ip ipaddrdst
ip ipaddror chain. (IP or IP
or IP). If ip is not specified with
src or dst the source or
destination IP may match.host
ipaddrhost
is just a synonym for ip (See above)
ip
in [iplist]src in
ip [iplist]dst
ip [iplist]or
chain, otherwise use an iplist If
ip is not specified with
src or dst the source or
destination IP may match.
net
network netmasksrc
net network netmaskdst
net network netmasknet
network/netbitssrc
net network/netbitsdst
net network/netbitsnet is not specified with
src or dst the source or
destination IP may match.
geo
geolocsrc
geo geocodedst
geo geocodegeo is not specified with
src or dst the source or
destination geo location code may match.
tunip
ipaddrsrc
tunip ipaddrdst
tunip ipaddrtunip is not
specified with src or dst
the source or destination tunnel IP may match.
port
comp numsrc
port comp numdst
port comp numport is not specified with
src or dst the source or
destination port may match.
port
in [portlist]src port
in [portlist]dst port
in [portlist]port is not specified with
src or dst the source or
destination port may match.
icmp-type
numicmp-code
numproto icmp.
engine-type
numengine-id
numsysid
numnfdump collector
internal assigned number. See also option -E
above.
if
numin if
numout
if numif is
not specified with in or
out the input or output interface may match.
as
comp numsrc
as comp numdst
as comp numprev
as comp numnext
as comp numnfdump supports 32-bit AS numbers every where.
Without or the source or destination AS may match. See
comp for the comparator details.
as
in [aslist]src as
in [aslist]dst as
in [aslist]prev as
in [aslist]next as
in [aslist]as is not specified with
src, dst, prev or next the
source or destination AS may match.
mask
bitssrc
mask bitsdst
mask bitsmask is not
specified with src or dst
the source or destination mask bits may match.
vlan
numsrc
vlan numdst
vlan numvlan is not specified
with src or dst the source
or destination vlan may match.
flags
tcpflagsflags S and not flags
AFRPUrouter
ip ipaddrnext
ip ipaddrbgpnext
ip ipaddrmac
macaddrin
mac macaddrin src
mac macaddrin dst
mac macaddrout
mac macaddrout src
mac macaddrout dst
mac macaddrmac
with any combination of a direction specifier as defined by CISCO v9 the
test is limited to those mac addresses only.
Otherwise multiple matches are possible. Without any specifiers any
mac address is tested against
macaddr
mpls
labelN comp numberN with
N as mpls label number 1..10 matches
number Filters according a specific number in the
mpls label stack.
mpls
eos comp numbermpls
expN comp numberN with N as mpls label
number 1..10 matches number
packets
comp numin
packets comp numout
packets comp numpackets and in
packets is equivalent and describes the number of packets from
source to destination. In case of a bi-directional flow (sent by an
exporter or combined by option --B ) the packet
counter for the reverse flow can be tested with out
packet
bytes
comp numin
bytes comp numout
bytes comp numbytes and in bytes is
equivalent and describes the number of bytes from source to destination.
In case of a bi-directional flow (sent by an exporter or combined by
option --B ) the byte counter for the reverse flow
can be tested with out bytes
flows
comp numtos
numflowdir
directionduration
comp timepps
comp numbps
comp numbpp
comp numobservation
domain id comp numberobservation
point id comp numberyaf or the nfdump
collector nfpcap can send payload data along the netflow
information. If such payloads are sent it can be filtered according
the filter primitives below:
payload
content 'string'payload
regex 'regex'payload
regex 'regex' flagsregex searches over the full payload length. A
' ' byte does not stop the match process.
regex must be quoted with single or double quotes:
'regex' or “regex” The regex engine understands the
following reduced syntax:
flags are optional can be:
payload
ja3 md5stringpayload
ja3 defined-s
ja3 statistic.
pf
action actionpf
reason reasonpf
rule ruleNrpf
dir in|outpf
interface interfaceNameclient
latency comp timeserver
latency comp timeasa
event eventasa
event comp numberasa event
denied reasonasa
xevent comp numxip
ipaddrsrc
xip ipaddrdst
xip ipaddrxip is
specified without src or
dst both IP addresses may match.
xport
ipaddrsrc
xport ipaddrdst
xport ipaddrxport is
specified without src or
dst both ports may match.
xnet
network/masksrc
xnet network/maskdst
xnet network/maskxnet is specified without
src or dst both IP
addresses may match.
ingress
ACL comp numberingress
ACE comp numberingress
XACE comp numberegress
ACL comp numbernat
event eventnat
event comp numbernip
ipaddrsrc
nip ipaddrdst
nip ipaddrnip is specified
without src or dst both IP
addresses may match.
It Cm nport Ar number
src
nport numberdst
nport numbernip is specified
without src or dst both
ports may match.
ingress
vrf numberpblock
start comp numberpblock
step comp numberpblock
end comp numberport in
pblocksrc port
in pblockdst port
in pblock=,
==, >, <, >=, <= To prevent collisions with bash
interpretation, alternative comparators are available:
EQ, LT, GT, LE, GE If comp is omitted, '==' is
assumed.This section describes how output formats are compiled.
nfdump has a lot of already pre-defined output
formats such as raw, json, csv etc. One line formats
as described for option -o can be compiled from
various elements of a flow record. As a flow record can contains man
different elements it is often useful to compile an output format for
specific needs.
The output format is specified by -o
“fmt: string”
string contains the field tags
to be printed as well as other characters if needed. A
tag starts with a % sign
followed by the field name. tags are separated by
spaces from other tags. Characters or other strings, not starting with a
% sign are copied literally to the output.
Example:
-o
“fmt:%ts %td %pr %sap -> %dap %pkt %byt %fl”This is the definition of the predined format
line. It adds the elements tstart
duration protocol source IP address/port followed by the literal
characters -> and destination ip address/port packets,
bytes, flows counter. Depending on the task, different output formats
are required to see the required fields of a flow record. You can either
extend a predefined format or specify a new one at the command line.
Example: Extend the predefined format long
with the the IP address of the sending router
-o
“fmt:%long %ra”Predefined formats can be extended by simply add their name with a
% sign somewhere in the format string. As described
under the output option -o
nfdump has already many formats
predefined. Most of the time, these format are good enough. Sometimes you
may need different formats, which can be compiled as described above. In
order to prevent adding the same often used output format each time you run
nfdump a new output format may be define in the
config file nfdump.conf The file nfdump.conf.dist
contains the definition of the already hard coded formats. These may be
uncommented and changed according to the specific needs. New formats may be
added using the following syntax:
fmt.newname = “fmt:%ts %td %pr
%sap -> %dap %pkt %byt %fl”with newname any new or existing definition of output formats. Existing formats are overwritten with the new definition.
The following list contains all tags, which are available to compile the output format:
%<format>%cnt%nfv%ts%tfs%tsr%te%ter%tr%trr%td%pr%exp%eng%lbl%sa%da%sap%dap%gsap%gdap%sp%dp%it%ic%sn%dn%nh%nhb%ra%sas%das%nas%pas%in%out%pkt%ipkt%opkt%byt%ibyt%obyt%fl%flg%tos%stos%dtos%dir%smk%dmk%fwd%svln%dvln%ismc%odmc%idmc%osmc%mpls1%mpls2%mpls3%mpls4%mpls5%mpls6%mpls7%mpls8%mpls9%mpls10%mpls%bps%pps%bpp%sc%dc%sloc%dloc%sasn%dasn%n%ipl%opl%nbid%ja3%sni%nbnam%odid%opid%pfifn%pfact%pfrea%pfdir%pfrule%nfc%evt%xevt%sgt%msec%iacl%eacl%xsa%xda%xsp%xdp%xsap%xdap%uname%nevt%ivrf%evrf%nsa%nda%nsp%ndp%pbstart%pbend%pbstep%pbsize%cl%sl%alnfdump processes files created by any
previous version of nfdump 1.6.x with some limitations for versions <
1.6.17. In order to convert flow files to the new 1.7.x binary format use
the following command to read//write files:
% nfdump -r oldfile -w
newfilePrint a statistic about the top 20 IP addresses, once sorted by flows and once by bytes
% nfdump -r flowfile -s
ip/flows/bytes -n 20Print two statistics, one about the source IP and one about the destination IP address limited to flow with either source or destination port 443
% nfdump -r flowfile -s srcip/bytes
-s dstip/bytes -n 20 'port 443'Print a statistic about the IP pairs, which exchanged most traffic.
% nfdump -r flowfile -s record/bytes
-A srcip,dstipPrint all flows in raw format with a HTTP header in the payload even if flow is not on port 80.
% nfdump -r flowfile -o raw
“payload regex 'GET|POST'”Print a statistic about all ja3 md5 sums for those flows, which a valid ja3 can be calculated
% nfdump -r flowfile -s ja5 -n 0
'payload ja3 defined'Aggregate all flows and write the result back to a binary file, sorted by the start time
% nfdump -r flowfile -a -Otstart -w
newfilenfdump returns 0 on success and 255 if
processing failed.
https://www.iana.org/assignments/ipfix/ipfix.xhtml
https://www.cisco.com/en/US/technologies/tk648/tk362/technologies_white_paper09186a00800a3db9.html
No software without bugs! Please report any bugs back to me.
| January 31, 2025 | Debian |