Twitter’s open-source algorithms are more complex than Musk suggests

Of all the big ideas Elon Musk has for Twitter, the one he’s launched with the most fervor is to make the platform’s algorithms open source.

The Tycoon Tesla proposed the plan prior to the disclosure of its takeover offer, reiterated it the day his offer was revealed, and threw it one more time after confirmation of the agreement.

Musk presented his proposal during a TED talk on April 14:

It is simply very important that people have both the reality and the perception that they can express themselves freely within the limits of the law. So one of the things I think Twitter should do is open up the algorithm.

Musk argued that disclosing what amplifies or downgrades tweets would reduce the risk of “behind-the-scenes manipulation.”

The approach has won support from some transparency advocates and critics of Twitter’s content moderation. They argue that this decision will reveal how Twitter determines what you see on your timeline – and what you don’t.

“It has the potential to turn Twitter into a truly trustworthy platform, where users understand why certain tweets appear at the top of the list, and any concerns about secrecy or behind-the-scenes bias are removed,” said Marc Linster. , CTO of open source database company EDB.

“These concerns have been endemic with Google and Facebook. This move to open source could be a game-changer for social media as a whole.”

Skeptics, however, questioned the feasibility of the plan. They note that Twitter is made up of various feeds, from the Trending section to your home timeline, each controlled by a complex mix of recommendation systems and human decisions.

These processes produce results that even their developers don’t fully understand. Some of them would have mocked Musk by adding a (now deleted) public repository to the company’s GitHub platform – with zero code.

Another problem is that the algorithms alone offer limited information.

There are various other factors behind the ranking of a tweet. They include the content that enters the platform, each user’s profile, algorithm training data, moderation rules, and the code that trained the models.

These constitute a huge reservoir of data, difficult to browse and expensive to disseminate.

“You can’t just open an ML [machine learning] model as if it were Bubble sort implementationsaid Steve Teixeira, vice president of product at Twitter.

Other complexities arise from the mutability of the system.

“Typically, recommendation patterns get recycled quite often and will continue to change over time,” said Bindu Reddy, CEO and co-founder of Abacus.AIan artificial intelligence startup, told TNW.

“While it’s also possible to publish all continuously trained models, that won’t be very useful either unless you understand exactly what inputs and outputs go into the model for predictions.”

There are also the potential dangers of the open-source proposal.

The information could be copied by competitors, be a tempting target for cybercriminals, and violate user privacy. It could also hamper another of Musk’s ambitions: “defeat spambots.”

On the other hand, open source offers new opportunities to find vulnerabilities and loopholes.

Reddy is optimistic about the potential benefits. She argues that the open source ranking algorithm will be useful for researching and assessing any biases.

She also hopes to find more information about the infrastructure components that influence what is reported and filtered in the feeds.

“To open sourcing these algorithms – and more importantly, these models – will go a long way towards transparency,” she said.

Another prominent proponent of the approach is Twitter co-founder Jack Dorsey.

The company’s former CEO suggested letting users choose which algorithm they use, if any.

Dorsey plans to create an open market for algorithms.

Users would pick the one that best suits their desires, from prioritizing nuanced conversations to highlighting a steady stream of thirst traps.

That sounds potentially idyllic — especially if it can keep my feed from constantly showing hateful Elon Musk tweets.

Sharon D. Cole