[spark] 셔플 관련 자료

scala 2018. 12. 4. 14:39



spark에는 여러 셔플(shuffle) 함수의 성능적인 부분을 설명한 자료이다.





Everyday I'm Shuffling - Tips for Writing Better Spark Programs, Strata San Jose 2015 from Databricks




책으로 보려면, 아래 책을 참고하깅 바란다. 


Scala and Spark for Big Data Analytics: Explore the concepts of functional programming, data streaming, and machine learning 


https://www.amazon.com/Scala-Spark-Big-Data-Analytics/dp/1785280848


저작자표시 비영리 동일조건

'scala' 카테고리의 다른 글

[spark] Utils: uncaught error in thread spark-listener-group-streams, stopping SparkContext java.lang.InterruptedException at java.util.concurrent.locks.AbstractQueuedSynchronizer$ConditionObject.reportInterruptAfterWait(AbstractQueuedSynchronizer.ja..  (0) 2019.01.19
[spark] Task 0 in stage 2.0 failed 1 times, most recent failure: Lost task 0.0 in stage 2.0 (TID 22, localhost, executor driver):  (0) 2019.01.17
[spark] 선형 회귀 , SGD, 평균 제곱 오차  (0) 2018.12.01
[spark] ERROR 25505: A read-only user or a user in a read-only database is not permitted to disable read-only mode on a connection. 해결하기  (0) 2018.11.29
[spark] 백프레셔(Backpressure)  (0) 2018.11.20
Posted by '김용환'
,

티스토리툴바