When ChatGPT and other generative AI tools were introduced to the world months ago, I predicted that sooner, more than later, we would have more identity impersonation scams on the rise. True enough, we saw a trickling of a few generative AI impersonations but in the past week, the trickling has become an overflow of such as promoted on Facebook.
These ads consist of prominent political, media and business figures such as Ramon Ang, the Gokongwei family, the Tulfos, Cathy Yang and Joel Villanueva. These chat heads follow a very specific script — that this is some sort of golden age and they have found a perfect money making opportunity to uplift their own lives — and you can get in on it too when you sign up. Of course, you end up in a phishing site and soon enough, your money is gone. You have been scammed.
The script is the same all througout, with the chat heads and voices of these personalities obviously AI-generated for more hawkish Internet user. But the average Facebook user may be deceived. Voice cloning software is easy to find these days and all you need to clone a voice is to record 30 seconds of their original audio, of which are widely available online since these are public personalities. You can tell it’s AI generated because the voices sound TikTok-ish, with no natural pauses or intonations, sounding more like a robot that sounds like the original person. The movements of the people are also AI-generated, as it is also easy to generate AI avatars of people these days. All you need is their likeness.
There are more on the rise and as a responsible Facebook user, I have been reporting these pages but these tool are so commoditized, it’s so easy to start a new page and post content. What worries me more is how Facebook seemingly lets this content proliferate. Is it all about the money? I hope not. We’re also unsure of how policing policies work these days as the policy on paper seems to differ in practice. In the Philippines, media used to have dedicated account representatives and policy we could coordinate with for such thing, but these departments have been silent for years after a refocusing away from media partnerships. Their latest policy update has to do with spammy content on Creator pages, which has nothing to do with this.
It is common for technology to outpace policy, but this is precisely why the Philippines needs a sandbox-type of policy making group that can operate and act like a startup within the government. Dr. Cris Liban recently wrote a column about operating within a sandbox environment here.
What do we do with scams?
Facebook seems to hold all the cards here but on our end we can do our part by telling our less tech-inclined family members to ignore these financial scams. We have to explain how these new ads can be made using AI, something that seems too ridiculous for them, but they are indeed possible today with the advent of NPUs and AI agents.
Reporting and blocking these pages work in the short term but unless Facebook’s policy team can truly flag these types of content, we really have to come to the realization that the AI literacy battle has only begun.
