Data Poisoning
Data poisoning is an attack that manipulates the data a machine learning model is trained on, so that the trained model makes the errors the attacker chose for it.
What is Data Poisoning?
Definition
Data Poisoning is an attack on a machine learning model carried out through its training data rather than its code: the attacker inserts or alters examples so the model learns the behaviour they want. The result may be broad degradation of accuracy, or a targeted backdoor that behaves normally except on inputs carrying a specific trigger. It is hard to detect after the fact, because the artefact that misbehaves — the model — contains nothing obviously malicious, which is why defences concentrate on the provenance and validation of training data rather than on inspecting the finished model.
Role in cybersecurity
Data Poisoning plays an important role in building organizational resilience against cyber threats. Implementing appropriate mechanisms in this area is required by regulations such as NIS2, DORA and ISO 27001.