Is Meta’s system map the next step towards safer algorithms?

Meta’s Facebook and Instagram have been regularly scrutinized by their algorithms. From a macro perspective, all major technologies have been criticized for opaque algorithms and machine learning applications, most of which stem from the black box nature of AI. Today, most computer systems are a “black box,” meaning that engineers can see the input and its resulting output, but not the computer processes that led to it. These are silent on the solutions, how they work, why they know your preferences, and how they provide the best results. Uncovering the black box problem has become a key topic for researchers and ethical AI enthusiasts. Also, with companies like Meta, there is no single algorithm; there are several algorithms powered by variations of AI and human collaboration, making it harder to create a standard approach to understanding them.

The AI ​​system map

The AI ​​team responsible for Meta got a head start and presented a prototype of their tool for providing insights into the underlying architecture of an AI system, called the AI ​​System Card tool. System maps help explain how the system works with different levels of the map. This first AI system card discusses models including an AI system to better understand how the system works based on a user’s history, preferences, settings, and more. The pilot system board explains Instagram feed ranking which takes unseen posts from accounts followed by a user and then ranks them based on the likelihood of the user engaging with them.

The team imagined that experts and non-experts would both be the audience for the system maps to provide their feedback for the human interface in a reproducible and scalable way for Meta. Additionally, given the technically precise nature of the framework, it can easily grasp the nuances of how the system works, even on a large scale. Thus, it is “easily digestible for ordinary people using technologies”.

Inside an ML model with system boards

Different ML models work in different ways, and they usually work in conjunction with other models to produce effective results. Given this, models may interact differently depending on macro-systems, preventing model maps, a standard for model documentation, from fully explaining the AI ​​system. Meta explained this with an example stating: “Our image classification models are all designed to predict what is in a given image, they can be used differently in an integrity system that flags harmful content by versus a recommender system used to show people posts they might be interested in.”

System maps were developed as a hybrid between common XAI approaches to model maps and datasheets that explain how models are built and the data used to train them. The system maps approach was chosen because it is a means of understanding how model outputs are used in a larger product, the policy actions resulting from this use, and their impact on users. These cards can also be applied to translate languages, detect fashion objects and recognize English speech.

Instagram Feed Ranking

The pilot ranks Instagram’s feed to demonstrate how the system works and how the AI ​​feed ranking system dynamically delivers personalized experiences. The program works in seven stages.

The first stage gathers the unpublished publications of the accounts followed by the user or liked by his friends. After filtering posts that meet Instagram’s guidelines, the system predicts the user’s likelihood of interacting with the post by collecting attributes and comparing them with historical data on how often the user has interacted with it. the particular author. For example, the high probability that the user saves, presses or watches a message indicates a higher probability of interaction with it. The third step is to combine the probabilities into a single numerical score for each message, say 87%, and do the same for everyone. The fourth step repeats the first three steps for all other post types, then normalizes the scores for all. Once all messages have been rated, the system downgrades messages that have been classified as carrying false information. The penultimate step ensures that the feed contains multiple posts and prevents similar posts or posts from the same account. Finally, the system combines the posts and displays them on the user’s feed.

Meta also described the various components of the feed ranking system. These are

  • Sourcing Candidates System which finds all invisible contents of connections
  • Filtering system that removes content that violates guidelines
  • Scoring model that assigns a numerical value to the likelihood that a user will perform an action
  • Merge scores that combine different score categories
  • Boosts library, a set of rules that gives preference to recent posts
  • Integrity Downgrades, a rule that downgrades posts that are inappropriate but do not blatantly violate community guidelines
  • Sort by score which sorts posts in ascending order of score
  • Diversity of freshness that rearranges posts to display the most recent
  • Diversity of media types that rearranges posts to show different types of posts
  • Author Diversity which rearranges articles to show different authors
  • Hashtag diversity that rearranges posts to show different hashtags

Meta has also created a system board tools site where users can try their hand at interactive exercises to understand how the different components of feed ranking can work on sample user profiles. This update comes after an ongoing review of the company.

Sharon D. Cole