Detection of anomalies and Data Drift in a time-series dismissal prediction system

Abstract

The purpose of the study is to develop a system that automatically processes data based on existing
and newly entered data, especially with the aim of ensuring high data quality by detecting and eliminating
anomalies. The quantile filtering method, Chebyshev’s inequality, Kolmogorov-Smirnov two-sample test, and
others should be noted among the methods used. In the course of the research, the theoretical aspects of the
methods, various principles of detecting anomalies for different types of data were considered and analysed.
Different principles and approaches applied to anomaly detection in different contexts were explored. The results
of the analysis and the selection of optimal methods for detecting anomalies in various types of data are important
for the effective functioning of the automatic data processing system. This will make it possible to achieve
accuracy and reliability in the detection of anomalies and ensure high quality of data used in the machine learning
system.