May 12, 2023

Data deduplication

It helps to save enormous amount of data storage. Sycope as first on the market used deduplication mechanism for NetFlow.

Data deduplication

Deduplication is a technology allowing elimination of the duplicate copies of repeating data. It is space-saving approach, which reduce amount of the disk space we need to save certain quantity of the data. Deduplication works by identifying multiple instances of repeated information, then eliminating redundancies, replacing duplicate information with a reference to the same information located elsewhere.

Deduplication help us to achieve a data reduction ration of 3-to-1. That feels a lot like magic and rainbows when you are talking about many terabytes of data!


Graph 1. How simple process of deduplication looks like

How Sycope use deduplication mechanism for NetFlow?

If the flows are duplicated from multiple sources, Sycope deduplicates data in order to retain a unique information record only. Apart from its other benefits, the deduplication mechanism allows the following:

• Presentation of actual traffic volume values, regardless of the filters applied.

• Displaying the traffic path based on NetFlow fields received for the same transmission from multiple routers.

Get a monthly dose of blog posts, tips and tricks

Sign-up for the newsletter and be updated about Sycope.

Sign-up for the newsletter
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.