Understanding Disk Space Utilization


Like any other system resource, disk space is limited. Andrisoft Wanguard can store very large quantities of data, depending on its configuration. If you think it's using up too much disk space, read through this article to understand what might be the cause and how it can be prevented.


What is using up your precious disk space ?

You can control the amount of disk space that can be used for storing the database in Configuration » General Settings » Data Retention. Unless you have altered the default path for storing flows, graphs and dump files, you can use the ncdu utility or the following command to see what is using up your disk space, apart from the database (this may take up to a few minutes, depending on how many files you have):
du -ch -d 1 /opt/andrisoft/

Packet Traces (dumps)

If the /opt/andrisoft/dumps directory occupies too much disk space, that means you have either done large packet traces manually, or you have configured the software to capture large packet traces in Configuration » Network & Policy » Responses.

Solutions
  • Configure the software to delete old dump files automatically in Configuration » General Settings » Data Retention » Packet Traces.
  • Recheck the configuration of the Traffic Sample Capturing Action, going side by side with the User Manual, which explains each field in detail. Configuring a Max. Packets number is the best way to avoid large dump files.
  • If you need to clear up some disk space immediately, you can manually delete any files/directories from within the /opt/andrisoft/dumps directory but not the dumps directory itself. If you decide to delete directories or files manually, please allow the software to create them back automatically and do not attempt to recreate them by yourself.

Flows

If the /opt/andrisoft/flows directory occupies too much disk space, that means the Flow Sensor is receiving a very large amount of flows, and you have chosen to save them by activating the Flow Collector option.

Solutions
  • As a first attempt to lower the disk space consumption of flows, select the "Saved compressed flows" option within the Flow Sensor Configuration window. More information can be found here.
  • Configure the software to delete old flows automatically in Configuration » General Settings » Data Retention » Flow Collectors.
  • If you need to clear up some disk space immediately, you can delete any files/directories manually from within the /opt/andrisoft/flows directory but not the flows directory itself. If you decide to manually delete directories or files, please allow the software to create them back automatically and do not attempt to recreate them by yourself.

IP Graphs

If the /opt/andrisoft/graphs directory occupies too much disk space, that means you have either configured the software to generate graph data files for a very large number of IPs, or you have configured graph data files to contain an unnecessarily large amount of information.

Solutions
  • To find out how much disk space each IP graph file requires, go to Configuration » General Settings » Storage & Graphs. Tweaking the Accuracy of Round Robin Archive, Decoders, Stored Units or the Consolidation Functions modifies the IP graph file size, which is dynamically displayed in the bottom section of the Configuration window. It is possible that a lot of unnecessary information is stored for each IP and therefore it is best to reconsider the necessity of each option while consulting the User Manual here. The configuration is applied to all IP graph files, and it is currently not possible to choose different configuration options for different IP prefixes.
  • We strongly suggest disabling IP graphing for very large IPv4 prefixes or IPv6 prefixes from the IP Zone Configuration window (Configuration » Network & Policy » IP Zone). Depending on what you have configured in the Storage & Graphs window, the software calculates how much storage is required in case you choose to turn on IP graphing for any IPv4 prefix. This is displayed in the "Storage Requirements" column of the IP Settings section, in the IP Zone Configuration window. You can immediately see that turning IP graphing on for large IP prefixes is not a good idea.
  • *IMPORTANT* It is important to know that the software generates IP Graph files at their maximum size and updates them accordingly without modifying the size afterward. Therefore, you cannot rely on the fact that an IP has little traffic and won't occupy much space. An IP graph file is generated at the first packet that has that IP as a source or a destination address. A simple network scan on a /8 prefix for which IP graphing is enabled can flood your hard disk with full sized IP graphing files.
  • You can aggregate IP graphing information collected by multiple Packet Sensors or Flow Sensor interfaces by using the Sensor Cluster component.
  • If you need to clear up some disk space immediately, you can manually delete any files/directories from within the /opt/andrisoft/graphs directory but not the graphs directory itself. If you decide to delete directories or files manually, please allow the software to create them back automatically and do not attempt to recreate them by yourself. To delete the IP graph files which were not updated in the last 90 days, execute:
    find /opt/andrisoft/graphs/ -mtime +90 -type f –delete
    

If you require further assistance, please contact support andrisoft.com.



Author
Andrisoft Team
Date Created
2015-06-19 11:27:37
Date Updated
2018-09-07 23:23:12
Views
2967