Generative ML and CSAM: Implications and Mitigations

The adversarial use of generative machine learning models has been recognized in the study of mis- and disinformation for quite some time—historically, Generative Adversarial Networks have been used to generate realistic-looking (although relatively easily detectable) avatars for fake accounts on social media services.

With the release of freely available conditional Diffusion Models, visual generative machine learning models became more flexible and user-friendly, generating elaborate scenes based on user-supplied textual prompts.