[펌] Facebook의 superhider와 피드 개발팀 일하는 방식

slate.com에 페이스북의 뉴스 피드 알고리즘에 대한 일부 내용이 공유되었다.

(대충 정리한 발번역)

Not everyone uses Facebook the same way, however. Facebook’s data scientists were aware that a small proportion of users—5 percent—were doing 85 percent of the hiding. When Facebook dug deeper, it found that a small subset of those 5 percent were hiding almost every story they saw—even ones they had liked and commented on. For these “superhiders,” it turned out, hiding a story didn’t mean they disliked it; it was simply their way of marking the post “read,” like archiving a message in Gmail.

Yet their actions were biasing the data that Facebook relied on to rank stories. Intricate as it is, the news feed algorithm does not attempt to individually model each user’s behavior. It treats your likes as identical in value to mine, and the same is true of our hides. For the superhiders, however, the ranking team decided to make an exception. Tas was tasked with tweaking the code to identify this small group of people and to discount the negative value of their hides.

That might sound like a simple fix. But the algorithm is so precious to Facebook that every tweak to the code must be tested—first in an offline simulation, then among a tiny group of Facebook employees, then on a small fraction of all Facebook users—before it goes live. At each step, the company collects data on the change’s effect on metrics ranging from user engagement to time spent on the site to ad revenue to page-load time. Diagnostic tools are set up to detect an abnormally large change on any one of these crucial metrics in real time, setting off a sort of internal alarm that automatically notifies key members of the news feed team.

Once a change like Tas’ has been tested on each of these audiences, he’ll present the resulting data at one of the news feed team’s weekly “ranking meetings” and field a volley questions from Mosseri, Allison, Marra, and his other colleagues as to its effect on various metrics. If the team is satisfied that the change is a positive one, free of unintended consequences, the engineers in charge of the code on the iOS, Android, and Web teams will gradually roll it out to the public at large.

Even then, Facebook can’t be sure that the change won’t have some subtle, longer-term effect that it had failed to anticipate. To guard against this, it maintains a “holdout group”—a small proportion of users who don’t see the change for weeks or months after the rest of us.

To speak of Facebook’s news feed algorithm in the singular, then, can be misleading. It isn’t just that the algorithm is really a collection of hundreds of smaller algorithms solving the smaller problems that make up the larger problem of what stories to show people. It’s that, thanks to all the tests and holdout groups, there are more than a dozen different versions of that master algorithm running in the world at any given time. Tas’ “hide stories” tweak was announced July 31, and his post about it on Facebook’s “News Feed FYI” blog passed largely unnoticed by the public at large. Presumably, however, the superhiders of the world are now marginally more satisfied with their news feeds, and thus more likely to keep using Facebook, sharing stories with friends, and viewing the ads that keep the company in business.

Facebook의 데이터 사이언티스트는 Facebook의 5% 사용자가 85%에 대해 "이 소식 보지 않기(hide story)"를 눌렀다고 한다. 

좋아요(like)와 리플(comment)를를 달았던 사람들이 "이 소식 보지 않기(hide story)"를 누른다고 한다.

(원래 의도는 hide story 기능은 사용자가 특정 소식을 보고 싶지 않다라는 강력한 신호로 여기는 것이었다.)

이 5%의 "superhider"라는 사용자는 싫어서 피드를 안보이게 하는게 아니라. "Gmail"를 쓰는 것처럼 '읽었다'라는 것을 표시하기 위해 "이 소식 보지 않기(hide story)"를 누른다고 한다.

Facebook의 뉴스 피드 알고리즘은 개별 사용자의 동작 하나 하나에 영향을 받지 않지만, 뉴스 피드가 랭킹으로 피드를 정리하는데, 이 부분에 대해서 예외를 주었다고 한다. superhider의 그룹를 식별하기 위해 코드를 변경해서 "이 소식 보지 않기(hide)"가 랭킹 점수를 깍는게 아니도록(discount) 수정을 했다고 한다. 

코드가 바뀐 뉴스 피드 알고리즘은 먼저 오프라인으로 시뮬레이션을 진행한 후, 직원 일부에 관련 코드를 테스트하고, 전체 오픈하기 전에 전체 Facebook 사용자 중 일부에게 테스트를 진행했다. 각 단계마다 변화 데이터를 수집했다. 진단 툴을 이용하여 이상하게 크게 변경한 메트릭 정보를 뉴스 피드 팀이 받도록 했다.

뉴스 피드 팀은 주단위로 "랭킹 미팅"을 가지고 피드 관련 결과 메트릭 데이터를 공유한다. 변경이 긍정적으면 관련 코드를 점점 대중에게 확산시킨다. 하지만, 변화가 애매한 것들은 "홀드아웃 그룹"에 유지시킨다.

7월 31일에 Facebook은 블로그(http://newsroom.fb.com/news/2015/07/news-feed-fyi-a-better-understanding-of-hide/)에 관련 글을 게재했다.(아마도 전체 배포된듯.) (Gmail처럼 쓰는 사용자를 위해서 이 소식 보지 않기(hide story)를 부정적인 신호로 여기지 않는 알고리즘을 배포했다. Facebook은 superhider가 계속 늘어나고 있음을 인지하고 있다.)

출처 : 

http://www.slate.com/articles/technology/cover_story/2016/01/how_facebook_s_news_feed_algorithm_works.single.html

참고 :

http://newsroom.fb.com/news/2015/07/news-feed-fyi-a-better-understanding-of-hide/