Git-Mirrors/advmlthreatmatrix
1
0
Fork 0
mirror of https://github.com/mitre/advmlthreatmatrix.git synced 2025-03-13 10:56:28 -04:00
Code Issues Actions Packages Projects Releases Wiki Activity

Added Attack on MT case Study

Browse Source
This commit is contained in:
ramtherunner 2020-11-15 17:24:42 -08:00 committed by GitHub
parent ce1a6c0a8e
commit 95cab8e31d
No known key found for this signature in database
GPG Key ID: 4AEE18F83AFDEB23
1 changed files with 23 additions and 4 deletions
Show Stats Download Patch File Download Diff File Expand all files Collapse all files

19
pages/case-studies-page.md
View File

@ -1,5 +1,6 @@
## Case Studies Page
- [ClearviewAI Misconfiguration](/pages/case-studies-page.md#clearviewai-misconfiguration)
- [GPT-2 Model Replication](/pages/case-studies-page.md#gpt-2-model-replication)
- [ProofPoint Evasion](/pages/case-studies-page.md#proofpoint-evasion)
@ -16,6 +17,24 @@ Attacks on machine learning (ML) systems are being developed and released with i
3. Range of ML Paradigms: Attacks on MLaaS, ML models hosted on cloud, hosted on-premise, ML models on edge.
4. Range of Use case: Attacks on ML systems used in both "security-sensitive" applications like cybersecurity and non-security-sensitive applications like chatbots.
----
## Attack on Machine Translation Service - Google Translate, Bing Translator, Systran Translate
**Summary of Incident:** Machine translation services (such as Google Translate, Bing Translator, and Systran Translate) provide public-facing UIs and APIs. These public endpoints can be used to steal an "imitation model" with near-production, state-of-the-art translation quality. Beyond demonstrating that IP can be stolen from a black-box system, the imitation model was used successfully transfer adversarial examples to the real production services. These adversarial inputs successfully cause targeted word flips, vulgar outputs, and dropped sentences on Google Translate and Systran Translate websites.
**Mapping to Adversarial Threat Matrix:**
- Using published research papers, the researchers gathered similar datasets and model architectures that these translation services used
- They exploit a public facing application to query the model and produce machine translated sentence pairs as training data
- Using these translated sentence pairs, researchers trained a substitute model ("model stealing / model replication")
- The replicated models were used to construct offline adversarial examples that successfully transferred to an online evasion attack
**Reported by:**
- Work by Eric Wallace, Mitchell Stern, Dawn Song and reported by Kenny Song (@helloksong)
**Source:**
- https://arxiv.org/abs/2004.15015
- https://www.ericswallace.com/imitation
----
## ClearviewAI Misconfiguration