daily pastebin goal

tcpflow v1.2.6

a guest May 5th, 2012 54 Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
  1. tcpflow(1)                       tcpflow 1.2.6                      tcpflow(1)
  5. NAME
  6.        tcpflow - TCP flow recorder
  9.        tcpflow [-BcCehPpsv] [-b max_bytes] [-d debug_level] [-f max_fds]
  10.        [-i iface] [-r file] [-m minskip] [-X file.xml] [-F[outputformat]]
  11.        [-T[filenameetemplate]] [expression]
  14.        tcpflow is a program that captures data transmitted as part of TCP con‐
  15.        nections (flows), and stores the data in a way that is convenient for
  16.        protocol analysis or debugging.  A program like tcpdump(4) shows a sum‐
  17.        mary of packets seen on the wire, but usually doesn't store the data
  18.        that's actually being transmitted.  In contrast, tcpflow reconstructs
  19.        the actual data streams and stores each flow in a separate file for
  20.        later analysis.  tcpflow understands TCP sequence numbers and will cor‐
  21.        rectly reconstruct data streams regardless of retransmissions or out-
  22.        of-order delivery.
  24.        tcpflow stores all captured data in files that have names of the form
  26.        where the contents of the above file would be data transmitted from
  27.        host port 2345, to host port 45103.
  30.        -B     Force binary output even when printing to console with -C or -c.
  32.        -b     Max bytes per flow.  Capture no more than max_bytes bytes per
  33.               flow.  Any data captured for a flow beyond max_bytes from the
  34.               first byte captured will be discarded.  The default is to store
  35.               an unlimited number of bytes per flow.
  37.        -c     Console print.  Print the contents of packets to stdout as they
  38.               are received, without storing any captured data to files
  39.               (implies
  41.        -C     Console print without the packet source and destination details
  42.               being printed.  Print the contents of packets to stdout as they
  43.               are received, without storing any captured data to files
  44.               (implies -e When outputting to the console each flow will be
  45.               output in different colors (blue for client to server flows, red
  46.               for server to client flows, green for undecided flows).  -s ).
  48.        -d     Debug level.  Set the level of debugging messages printed to
  49.               stderr to debug_level.  Higher numbers produce more messages.
  50.               -d 0 causes completely silent operation.  -d 1 , the default,
  51.               produces minimal status messages.  -d 10 produces verbose output
  52.               equivalent to -v .  Numbers higher than 10 can produce a large
  53.               amount of debugging information useful only to developers.
  55.        -F[format]
  56.               Specifies format for output filenames. Format specifiers: t
  57.               prepends each filename with a Unix timestamp.  T prepends each
  58.               filename with an ISO-8601 timestamp.  c appends each filename
  59.               with a connection counter.
  61.        -FM    Include MD5 of each flow in the DFXML output.
  63.        -FX    Suppresses file output entirely (DFXML file is still produced).
  65.        -T[format]
  66.               Specifies an arbitrary template for filenames.  %A expands to
  67.               source IP address.  %a expands to source IP port.  %B expands to
  68.               destination IP address.  %a expands to destination IP port.  %T
  69.               expands to timestamp in ISO8601 format.  %t expands to timestamp
  70.               in Unix time_t format.  %V expands to "--" if a VLAN is present.
  71.               %v expands to the VLAN number if a VLAN is present.  %C expands
  72.               to "c" if the connection count>0.  %c expands to the connection
  73.               count if the connection count>0.  %# always expands to the con‐
  74.               nection count.  %% prints a "%".
  76.        -AH    Perform HTTP post-processing ("After" processing). If the output
  77.               file is
  78.          ,
  79.               Then the post-processing will create the files:
  82.               If the HTTPBODY was compressed with GZIP, you may get a third
  83.               file as well:
  85.               Additional information about these streams, such as their MD5
  86.               hash value, is also written to the DFXML file
  88.        -fmax_fds
  89.               Max file descriptors used.  Limit the number of file descriptors
  90.               used by tcpflow to max_fds.  Higher numbers use more system
  91.               resources, but usually perform better.  If the underlying oper‐
  92.               ating system supports the setrlimit() system call, the OS will
  93.               be asked to enforce the requested limit.  The default is for
  94.               tcpflow to use the maximum number of file descriptors allowed by
  95.               the OS.  The -v option will report how many file descriptors
  96.               tcpflow is using.
  98.        -h     Help.  Print usage information and exit.
  100.        -i     Interface name.  Capture packets from the network interface
  101.               named iface.  If no interface is specified with -i , a reason‐
  102.               able default will be used by libpcap automatically.
  104.        -m min_size
  105.               Forces a new connection output file when there is a skip in the
  106.               TCP session of min_size bytes or more.
  108.        -P     No purge. Normally tcpflow removes connections from the hash ta‐
  109.               ble after the connection is closed with a FIN. This conserves
  110.               memory but takes additional CPU time. Selecting this option
  111.               causes the std::tr1:unordered_map to grow without bounds, as
  112.               tcpflow did prior to version 1.1. That makes tcpflow run faster
  113.               if there are less than 10 million connections, but can lead to
  114.               out-of-memory errors.
  116.        -p     No promiscuous mode.  Normally, tcpflow attempts to put the net‐
  117.               work interface into promiscuous mode before capturing packets.
  118.               The -p option tells tcpflow not to put the interface into pro‐
  119.               miscuous mode.  Note that it might already be in promiscuous
  120.               mode for some other reason.
  122.        -r     Read from file.  Read packets from file, which was created using
  123.               the -w option of tcpdump(1).  Standard input is used if file is
  124.               ``-''.  Note that for this option to be useful, tcpdump's -s
  125.               option should be used to set the snaplen to the MTU of the
  126.               interface (e.g., 1500) while capturing packets.
  128.        -s     Strip non-printables.  Convert all non-printable characters to
  129.               the "." character before printing packets to the console or
  130.               storing them to a file.
  132.        -v     Verbose operation.  Verbosely describe tcpflow's operation.
  133.               Equivalent to -d 10 .
  136.        The expression specified on the command-line specifies which packets
  137.        should be captured.  Because tcpflow uses the the libpcap library,
  138.        tcpflow has the same powerful filtering language available as programs
  139.        such as tcpdump(1).
  141.        The following part of the man page is excerpted from the tcpdump man
  142.        page.
  144.        expression selects which packets will be dumped.  If no expression is
  145.        given, all packets on the net will be dumped.  Otherwise, only packets
  146.        for which expression is `true' will be dumped.
  148.        The expression consists of one or more primitives.  Primitives usually
  149.        consist of an id (name or number) preceded by one or more qualifiers.
  150.        There are three different kinds of qualifier:
  152.        type   qualifiers say what kind of thing the id name or number refers
  153.               to.  Possible types are host, net and port.  E.g., `host foo',
  154.               `net 128.3', `port 20'.  If there is no type qualifier, host is
  155.               assumed.
  157.        dir    qualifiers specify a particular transfer direction to and/or
  158.               from id.  Possible directions are src, dst, src or dst and src
  159.               and dst.  E.g., `src foo', `dst net 128.3', `src or dst port
  160.               ftp-data'.  If there is no dir qualifier, src or dst is assumed.
  161.               For `null' link layers (i.e. point to point protocols such as
  162.               slip) the inbound and outbound qualifiers can be used to specify
  163.               a desired direction.
  165.        proto  qualifiers restrict the match to a particular protocol.  Possi‐
  166.               ble protos are: ether, fddi, ip, arp, rarp, decnet, lat, sca,
  167.               moprc, mopdl, tcp and udp.  E.g., `ether src foo', `arp net
  168.               128.3', `tcp port 21'.  If there is no proto qualifier, all pro‐
  169.               tocols consistent with the type are assumed.  E.g., `src foo'
  170.               means `(ip or arp or rarp) src foo' (except the latter is not
  171.               legal syntax), `net bar' means `(ip or arp or rarp) net bar' and
  172.               `port 53' means `(tcp or udp) port 53'.
  174.        [`fddi' is actually an alias for `ether'; the parser treats them iden‐
  175.        tically as meaning ``the data link level used on the specified network
  176.        interface.''  FDDI headers contain Ethernet-like source and destination
  177.        addresses, and often contain Ethernet-like packet types, so you can
  178.        filter on these FDDI fields just as with the analogous Ethernet fields.
  179.        FDDI headers also contain other fields, but you cannot name them
  180.        explicitly in a filter expression.]
  182.        In addition to the above, there are some special `primitive' keywords
  183.        that don't follow the pattern: gateway, broadcast, less, greater and
  184.        arithmetic expressions.  All of these are described below.
  186.        More complex filter expressions are built up by using the words and, or
  187.        and not to combine primitives.  E.g., `host foo and not port ftp and
  188.        not port ftp-data'.  To save typing, identical qualifier lists can be
  189.        omitted.  E.g., `tcp dst port ftp or ftp-data or domain' is exactly the
  190.        same as `tcp dst port ftp or tcp dst port ftp-data or tcp dst port
  191.        domain'.
  193.        Allowable primitives are:
  195.        dst host host
  196.               True if the IP destination field of the packet is host, which
  197.               may be either an address or a name.
  199.        src host host
  200.               True if the IP source field of the packet is host.
  202.        host host
  203.               True if either the IP source or destination of the packet is
  204.               host.  Any of the above host expressions can be prepended with
  205.               the keywords, ip, arp, or rarp as in:
  206.                    ip host host
  207.               which is equivalent to:
  208.                    ether proto \ip and host host
  209.               If host is a name with multiple IP addresses, each address will
  210.               be checked for a match.
  212.        ether dst ehost
  213.               True if the ethernet destination address is ehost.  Ehost may be
  214.               either a name from /etc/ethers or a number (see ethers(3N) for
  215.               numeric format).
  217.        ether src ehost
  218.               True if the ethernet source address is ehost.
  220.        ether host ehost
  221.               True if either the ethernet source or destination address is
  222.               ehost.
  224.        gateway host
  225.               True if the packet used host as a gateway.  I.e., the ethernet
  226.               source or destination address was host but neither the IP source
  227.               nor the IP destination was host.  Host must be a name and must
  228.               be found in both /etc/hosts and /etc/ethers.  (An equivalent
  229.               expression is
  230.                    ether host ehost and not host host
  231.               which can be used with either names or numbers for host /
  232.               ehost.)
  234.        dst net net
  235.               True if the IP destination address of the packet has a network
  236.               number of net. Net may be either a name from /etc/networks or a
  237.               network number (see networks(5) for details).
  239.        src net net
  240.               True if the IP source address of the packet has a network number
  241.               of net.
  243.        net net
  244.               True if either the IP source or destination address of the
  245.               packet has a network number of net.
  247.        net net mask mask
  248.               True if the IP address matches net with the specific netmask.
  249.               May be qualified with src or dst.
  251.        net net/len
  252.               True if the IP address matches net a netmask len bits wide.  May
  253.               be qualified with src or dst.
  255.        dst port port
  256.               True if the packet is ip/tcp or ip/udp and has a destination
  257.               port value of port.  The port can be a number or a name used in
  258.               /etc/services (see tcp(4P) and udp(4P)).  If a name is used,
  259.               both the port number and protocol are checked.  If a number or
  260.               ambiguous name is used, only the port number is checked (e.g.,
  261.               dst port 513 will print both tcp/login traffic and udp/who traf‐
  262.               fic, and port domain will print both tcp/domain and udp/domain
  263.               traffic).
  265.        src port port
  266.               True if the packet has a source port value of port.
  268.        port port
  269.               True if either the source or destination port of the packet is
  270.               port.  Any of the above port expressions can be prepended with
  271.               the keywords, tcp or udp, as in:
  272.                    tcp src port port
  273.               which matches only tcp packets whose source port is port.
  275.        less length
  276.               True if the packet has a length less than or equal to length.
  277.               This is equivalent to:
  278.                    len <= length.
  280.        greater length
  281.               True if the packet has a length greater than or equal to length.
  282.               This is equivalent to:
  283.                    len >= length.
  285.        ip proto protocol
  286.               True if the packet is an ip packet (see ip(4P)) of protocol type
  287.               protocol.  Protocol can be a number or one of the names icmp,
  288.               igrp, udp, nd, or tcp.  Note that the identifiers tcp, udp, and
  289.               icmp are also keywords and must be escaped via backslash (\),
  290.               which is \\ in the C-shell.
  292.        ether broadcast
  293.               True if the packet is an ethernet broadcast packet.  The ether
  294.               keyword is optional.
  296.        ip broadcast
  297.               True if the packet is an IP broadcast packet.  It checks for
  298.               both the all-zeroes and all-ones broadcast conventions, and
  299.               looks up the local subnet mask.
  301.        ether multicast
  302.               True if the packet is an ethernet multicast packet.  The ether
  303.               keyword is optional.  This is shorthand for `ether[0] & 1 != 0'.
  305.        ip multicast
  306.               True if the packet is an IP multicast packet.
  308.        ether proto protocol
  309.               True if the packet is of ether type protocol.  Protocol can be a
  310.               number or a name like ip, arp, or rarp.  Note these identifiers
  311.               are also keywords and must be escaped via backslash (\).  [In
  312.               the case of FDDI (e.g., `fddi protocol arp'), the protocol iden‐
  313.               tification comes from the 802.2 Logical Link Control (LLC)
  314.               header, which is usually layered on top of the FDDI header.
  315.               Tcpdump assumes, when filtering on the protocol identifier, that
  316.               all FDDI packets include an LLC header, and that the LLC header
  317.               is in so-called SNAP format.]
  319.        decnet src host
  320.               True if the DECNET source address is host, which may be an
  321.               address of the form ``10.123'', or a DECNET host name.  [DECNET
  322.               host name support is only available on Ultrix systems that are
  323.               configured to run DECNET.]
  325.        decnet dst host
  326.               True if the DECNET destination address is host.
  328.        decnet host host
  329.               True if either the DECNET source or destination address is host.
  331.        ip, arp, rarp, decnet
  332.               Abbreviations for:
  333.                    ether proto p
  334.               where p is one of the above protocols.
  336.        lat, moprc, mopdl
  337.               Abbreviations for:
  338.                    ether proto p
  339.               where p is one of the above protocols.  Note that tcpdump does
  340.               not currently know how to parse these protocols.
  342.        tcp, udp, icmp
  343.               Abbreviations for:
  344.                    ip proto p
  345.               where p is one of the above protocols.
  347.        expr relop expr
  348.               True if the relation holds, where relop is one of >, <, >=, <=,
  349.               =, !=, and expr is an arithmetic expression composed of integer
  350.               constants (expressed in standard C syntax), the normal binary
  351.               operators [+, -, *, /, &, |], a length operator, and special
  352.               packet data accessors.  To access data inside the packet, use
  353.               the following syntax:
  354.                    proto [ expr : size ]
  355.               Proto is one of ether, fddi, ip, arp, rarp, tcp, udp, or icmp,
  356.               and indicates the protocol layer for the index operation.  The
  357.               byte offset, relative to the indicated protocol layer, is given
  358.               by expr.  Size is optional and indicates the number of bytes in
  359.               the field of interest; it can be either one, two, or four, and
  360.               defaults to one.  The length operator, indicated by the keyword
  361.               len, gives the length of the packet.
  363.               For example, `ether[0] & 1 != 0' catches all multicast traffic.
  364.               The expression `ip[0] & 0xf != 5' catches all IP packets with
  365.               options. The expression `ip[6:2] & 0x1fff = 0' catches only
  366.               unfragmented datagrams and frag zero of fragmented datagrams.
  367.               This check is implicitly applied to the tcp and udp index opera‐
  368.               tions.  For instance, tcp[0] always means the first byte of the
  369.               TCP header, and never means the first byte of an intervening
  370.               fragment.
  372.        Primitives may be combined using:
  374.               A parenthesized group of primitives and operators (parentheses
  375.               are special to the Shell and must be escaped).
  377.               Negation (`!' or `not').
  379.               Concatenation (`&&' or `and').
  381.               Alternation (`||' or `or').
  383.        Negation has highest precedence.  Alternation and concatenation have
  384.        equal precedence and associate left to right.  Note that explicit and
  385.        tokens, not juxtaposition, are now required for concatenation.
  387.        If an identifier is given without a keyword, the most recent keyword is
  388.        assumed.  For example,
  389.             not host vs and ace
  390.        is short for
  391.             not host vs and host ace
  392.        which should not be confused with
  393.             not ( host vs or ace )
  395.        Expression arguments can be passed to tcpdump as either a single argu‐
  396.        ment or as multiple arguments, whichever is more convenient.  Gener‐
  397.        ally, if the expression contains Shell metacharacters, it is easier to
  398.        pass it as a single, quoted argument.  Multiple arguments are concate‐
  399.        nated with spaces before being parsed.
  402.        The following part of the man page is excerpted from the tcpdump man
  403.        page.
  405.        To record all packets arriving at or departing from sundown:
  406.               tcpflow host sundown
  408.        To record traffic between helios and either hot or ace:
  409.               tcpflow host helios and \( hot or ace \)
  411.        To record traffic between ace and any host except helios:
  412.               tcpflow host ace and not helios
  414.        To record all traffic between local hosts and hosts at Berkeley:
  415.               tcpflow net ucb-ether
  417.        To record all ftp traffic through internet gateway snup: (note that the
  418.        expression is quoted to prevent the shell from (mis-)interpreting the
  419.        parentheses):
  420.               tcpflow 'gateway snup and (port ftp or ftp-data)'
  422. BUGS
  423.        Please send bug reports to simsong@acm.org.
  425.        tcpflow currently does not understand IP fragments.  Flows containing
  426.        IP fragments will not be recorded correctly.
  428.        tcpflow never frees state associated with flows that it records, so
  429.        will grow large if used to capture a very large number of flows (e.g.,
  430.        on the order of 100,000 flows or more).
  432.        There appears to be a bug in the way that Linux delivers packets to
  433.        libpcap when using the loopback interface ("localhost").  When listen‐
  434.        ing to the Linux loopback interface, selective packet filtering is not
  435.        possible; all TCP flows on the localhost interface will be recorded.
  437. AUTHORS
  438.        Originally by Jeremy Elson <jelson@circlemud.org>.  Substantially modi‐
  439.        fied and maintained by Simson L. Garfinkel <simsong@acm.org>.
  441.        The current version of this software is available at
  442.               http://www.afflib.org/
  444.        An announcement mailing list for this program is at:
  445.               http://groups.google.com/group/tcpflow-users
  447. SEE ALSO
  448.        tcpdump(1), nit(4P), bpf(4), pcap(3)
  452. tcpflow 1.2.6                     15 Jan 2011                       tcpflow(1)
RAW Paste Data