NETFlow records headers of all packets with information about source/destination IP address, destination port, protocol type, time and length of communication, number of packets/bytes etc. This information is needed for law-enforcement authorities.

How it works in a nutshell - data collection is set in ISPadmin and terminal router is set to send packets headers to ISPadmin server. The terminal router must be placed before NAT. Otherwise NATed public addresses of clients would be recorded instead of private ones, which is not correct.

Thus the NETflow module creates all necessary records of client communication. It is designed for direct monitoring of the client access into public networks / the Internet. The contents of headers of all packets are recorded.

The process is regularly checked whether NETFlow is running and data is collected. You can display the current status of the process by clicking to NETflow (NETFlow is not active!!! / NetFlow is enabled). If NetFlow is not running you will see a warning upon login to the system.

The module performs the following:

• Recording of end-client traffic
• Evaluation of data needed for creating a record of communication necessary
• Export to CSV
• Detailed statistics such as type of protocol (HTTP, SMTP) for certain time periods

The following data is recorded:

• Connection type
• Identifier for a user account
• ID of device of Client's service
• Date and time of connection start
• Date and time of connection end
• Other identifiers of interest
• IP Address
• Port
• Event status (un-/successful connection)

NETFlow module is not a part of the basic installation of ISPadmin. Those interested in it need to purchase it separately. At first, you will need secure sufficient data storage, then you configure your terminal router correctly and our technical support will activate the module for you.

You can check your license in Clients Home - you should see ACTIVE for NETFlow module. If not, please contact sales department at This email address is being protected from spambots. You need JavaScript enabled to view it. and apply for a new licence file. You will receive it straight away as an attachment to an e-mail. All you need to do is copy the file to /data/support/ispadmin/config/.

As you´re about to see from given examples storage size needed doesn´t correspond with your average connectivity. The results may be individual, depending on the method of communication, e.g. network with VoIP services produces more data due to recording of bigger amount of headers of small UDP packets.

In general, independent disks of 320GB/500GB are sufficient. We recommend you to connect the disk to RAID1 (mirror) for data safety. You may also store NETFlow data on your system disk in a separate data partition. Such solution is cheaper with slower search speed; write speed remains almost the same.

You can check your data storage in Statistics Statistika serveru Graphs, or using the following commands from the command line:

Total size of NETFlow data
du -hl --max-depth=1 /data/support/flow/default/

Size of recorded data for a selected month
du -hl --max-depth=1 /data/support/flow/default/2009/*

Size of recorded data per day
du -hl --max-depth=1 /data/support/flow/default/2009/2009-10/*

We have done some real-operation measuring with the following results:

Average data flow of 40~60 Mbit/s

• Data flow between terminal router and ISPadmin server is approx. 150~300 kbit/s.
• 50~60 MB of data is stored daily, giving a total of 10~12GB in 6 months.
• The impact on terminal router (MikroTik or a common device with Pentium 4) is insignificant.
• Look-up in the recorded data is more demanding and depends on server performance (mainly disk operations). This may be solved by optimization of search and hierarchy of stored data.

Average data flow of 200 Mbit/s

• Approx. 1.6 GB of data is stored daily
• That is 230 GB in 6 months

Average data flow of 23 Mbit/s (600 active clients)

• Approx. 280 MB is stored daily, i.e. 5.5 GB per month
• That is 33 GB in 6 months

Average data flow of 78 Mbit/s (1800 active clients)

• Approx. 330 MB is stored daily, i.e. 13.5 GB per month
• That is 80 GB in 6 months

Prepare new disk

NETFlow data disk should be independent. Use the following instructions to prepare it.

First, create a primary partition for the entire disk:
fdisk /dev/sdc

Then format it to xfs file system:
mkfs.xfs /dev/sdc1

Then edit /etc/fstab file and mount the partition:
nano /etc/fstab
/dev/sdc1             /data/support/flow  xfs      defaults                    0     0

mount /dev/sdc1

If there is not enough free space in the system partition for backups, you can set backup storing to this new disk:

mv /data/backup  /data/support/flow
ln -s  /data/support/flow/backup /data/backup

Install flow-tools

The following package is essential for NETFlow module:

apt-get install flow-tools

Then modify configuration file. Comment all the free lines and write the new configuration with >30100 port to the last line:

nano /etc/flow-tools/flow-capture.conf
-w /data/support/flow/default -n 100 -V 5 -N 3 0/0/30123

Then create a folder which will store data from flow-tools:

mkdir -p /data/support/flow/default

Finally, activate flow-capture service and check if it is running:

/etc/init.d/flow-capture restart
ps ax |grep flow

And now set your terminal router to send packet headers to your ISPadmin server.

Set up terminal router

How it works in a nutshell - data collection is set in ISPadmin and terminal router is set to send packets headers to ISPadmin server. The terminal router must be placed before NAT otherwise NATed public addresses of clients would be recorded instead of private ones, which is not correct. If your network is connected via numerous terminal routers configure them all to send the recorded data to ISPadmin.

Router functionalities are not limited to a significant extent.

Mikrotik

Display IP / Traffic flow menu in Winbox and add a new item with the following data:

• IP address of ISPadmin server
• Port 30123   (port number must correspond with port number in /etc/flow-tools/flow-capture.conf)
• Version 5

Interface must be set to ALL in Trafic Flow Settings, otherwise it won´t work in MikroTik. It is important to have a correctly synchronized time on this router, otherwise you will not be able to find any information properly. Set the synchronization via System / NTP Client and use e.g. ntp.nasa.gov as synchronization server.

Alternatively, use the following command:

/ip traffic-flow target add address=server_IP:Port version=5

Determine whether to display NetFlow menu or not in Settings System Settings General, ID netflow_button.

In NetFlow Select, there are all recorded data available:

You can export currently selected data in NetFlow Export, and provide it to law-enforcement authorities.