27. General Settings » Data Retention

In Configuration » General Settings » Data Retention you can see how much disk space is used by the software, and you can set for how long the data should be kept.

✔ If the data is stored on an SSD disk, it’s essential to keep the disk usage below 80-90% to avoid performance losses due to Over Provisioning
✔ If the number of rows used by the database tables is in the millions, expect sudden performance losses caused by MySQL/MariaDB indexes
✔ As a rule of thumb, it’s best to only keep the data you actually need, for as long as you need it

DATA_RETENTION

Data Retention Settings fieldsets:

● Database – There are several classes of tables:
• Anomalies – These tables hold attack-related information. Keep this data for as long as possible
• Commands – These tables are useful for troubleshooting. They don’t grow up fast in normal conditions
• IP Accounting – This feature can create hundreds/thousands of tables because each table holds accounting information for only one day and one Sensor interface. Keep the number of retention days for these tables to a minimum
• Responses – These tables don’t grow very fast, and you should keep the data for as long as possible
• System Events – These events are stored into a single table called events. Keep the number of rows in this table under a few million to avoid Console performance issues
• Top Data – These tables usually grow the fastest because they store binary data for Sensor Tops, so it’s best to set a low retention period when the disk space is limited
● Flows and Packets – The filesystem paths for flows and packet dumps can be changed in General Settings » Graphs & Storage
● InfluxDB – If you use InfluxDB as a Graph Storage Engine in General Settings » Graphs & Storage, you can set its data retention period here
● RRD files – If you use RRDTool as a Graph Storage Engine in General Settings » Graphs & Storage, expand this fieldset to see the disk space consumed by it

The database is purged from old data once every hour, so the changes in Data Retention might not be immediately visible. The command for purging old data is:

[root@console ~]# sudo -u andrisoft /opt/andrisoft/bin/WANmaintenance

On some systems, MySQL/MariaDB will not free up the deleted data and instead it will reserve the resulted disk space for future use. To reclaim the newly freed up disk space, execute the following command on the Console server (it might take a very long time to execute):

[root@console ~]# /opt/andrisoft/bin/WANmaintenance optimize_db

Note

You can backup the whole database by executing on the Console server:

[root@console ~]# /opt/andrisoft/bin/WANmaintenance backup_db

You can backup the software configuration, by executing on the Console server:

[root@console ~]# /opt/andrisoft/bin/WANmaintenance backup_config