Twitter challenges millions of accounts every week to determine if they’re bots or not

Twitter Chief Technology Officer Parag Agrawal said Thursday that the company challenges millions of accounts every week to determine if they’re run by a human or completely automated. 

The use of fake accounts have been a big concern for tech companies especially after revelations surfaced in 2016 that Russian trolls used social networks to sow discord among Americans during the US presidential election. Agrawal, who was speaking at the Collison from Home tech conference, said there are various categories of fake accounts that can be used by bad actors. 

Some accounts are completely automated while other accounts are bots with “humans in the loop.” There are also accounts run by humans that attempt to manipulate conversations and accounts that get compromised and then get used by bad actors.

“Any attempt to manipulate the conversation actually applies some combination of all four of these potentially to do so,” he said. 

Outside of challenging accounts, Agrawal said that Twitter suspends millions of accounts every month before a user sees them on their timelines or in search results. 

The company uses technology, he said, to monitor the behaviors of a large amount of accounts and detect patterns. Once Twitter sees “anomalies” that can’t be explained by “normal usage,” the company knows whether or not they should investigate certain accounts.

Figuring out whether a Twitter account is fake, he said, isn’t always easy because there are accounts with no profile picture and a limited number of tweets that look like bots but they’re actually run by humans. Twitter also allows users to use pseudonyms.

Still, he said, it isn’t the accounts that look fake that are potentially the most problematic. 

“The most dangerous fake accounts actually don’t look fake on the surface,” he said.