Friday, March 29, 2024
HomeSECURITYgenerative AI and its role in creating content that incites violence and...

generative AI and its role in creating content that incites violence and extremism

-


A dangerous trend: generative AI and its role in creating content that incites violence and extremism

How unscrupulous cybercriminals use neural networks for their nefarious purposes.

According to researchers, malicious actors are increasingly misusing generative AI to spread material about child sexual abuse, disinformation, fraud and extremism, which is a very serious problem that requires extremely careful and skeptical attitude to any information found on the Internet.

“The explosion of generative AI has far-reaching implications for every corner of the web,” said Noam Schwartz, CEO and Founder Active Fence.

“We have identified three key trends of concern. First, we see that cybercriminals are now able to accelerate and amplify their operations through generative AI, leading to unprecedented mass production of malicious content. Second, those same participants are exploring ways to use AI by manipulating these models and discovering their inherent vulnerabilities. Finally, these evolving threats are putting increasing pressure on digital platforms to improve the accuracy and efficiency of their data training protocols,” Schwartz continued.

ActiveFence Researchers singled out The three main ways in which generative AI is abused for cybercriminal purposes are:

  • creating material about child sexual abuse, ranging from visual imagery to erotic stories;
  • generation of fraudulent images generated by artificial intelligence that mislead millions;
  • creating fake audio files promoting extremism;

Below we consider each of the points in more detail.


Child sexual abuse materialCSAM)

The researchers tracked a 172% increase in CSAM generated by generative AI in the first quarter of this year. Also found was a survey conducted by the administrators of the private dark web forum “Child Predator”, which polled nearly 3,000 cybercriminals about their use of generative AI.

The survey showed that 78% of respondents have or plan to use generative AI for CSAM, while the remaining 22% said they plan to try the technology. This and similar forums use generative AI algorithms to create sexual images as well as text descriptions, stories, and narratives.

Of course, in modern AI there are protective mechanisms against the generation of such vile and vile requests, however, each neural network has its own security flaws and, by choosing the optimal request, any restrictions can be bypassed. The same applies to neural networks that generate images. Although popular solutions like Midjourney have not experienced such problems, this is far from the only neural network capable of generating images.


Misinformation and fraudulent content

Although fraud and disinformation are not new concepts, generative AI has enabled attackers to create believable fraudulent images faster, more accurately, and with greater reach.

One of the AI-generated images found on Telegram by ActiveFence was inappropriately depicting Russian leader Vladimir Putin asking Chinese President Xi Jinping for support in resolving the conflict in Eastern Europe.

It is clear that upon a detailed examination of the AI-generated photos, one can notice obvious signs of a fake, but every month to determine deepfake more difficult due to the constant development of technology.


Violent extremism

The researchers also found numerous instances where attackers have used generative AI to create hyper-realistic malicious content that incites violence and promotes extremist propaganda. Often, cybercriminals use generative AI to create racist, nationalist or extremist manifestos or speeches.

ActiveFence has discovered an AI-generated bogus audio file that is using the growing political and economic unrest in the UK to “rock the boat”. The fabricated audio imitated a famous British reporter, inciting an uprising against the British government. The misleading manifesto contained instructions for acquiring weapons from the underground market and called for an attack on Britain’s national infrastructure.


Summarizing

The rapid development of generative artificial intelligence has serious implications for the online environment. Attackers are increasingly misusing this technology to create and distribute malicious and inappropriate content that poses a serious threat to the public conscience.

In the heyday of this kind of technology, one should be extremely critical of any information found on the Internet, no matter how realistic and plausible it may look.

In order to keep the online environment safe and clean of inappropriate content, these types of threats must be actively combated, using the joint efforts of technology companies, law enforcement agencies and society at large.



Source link

www.securitylab.ru

LEAVE A REPLY

Please enter your comment!
Please enter your name here

Most Popular