[spark] No output operations registered, so nothing to execute 에러

spark streaming 코딩하다 다음과 같은 에러가 발생했다. 

java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
at scala.Predef$.require(Predef.scala:224)
Exception in thread "main" java.lang.IllegalArgumentException: requirement failed: No output operations registered, so nothing to execute
at org.apache.spark.streaming.DStreamGraph.validate(DStreamGraph.scala:168)

 

Dstream을 처리할 때는 spark의 모든 API가 전부 다 허용되지 않는다.

 

Output operations allow DStream’s data to be pushed out to external systems like a database or a file systems. Since the output operations actually allow the transformed data to be consumed by external systems, they trigger the actual execution of all the DStream transformations (similar to actions for RDDs). Currently, the following output operations are defined:

 

Dstream 처리에 허영되는 api는 다음과 같다.

print() foreachRDD() saveAsObjectFiles() saveAsTextFiles() saveAsHadoopFiles()

http://spark.apache.org/docs/latest/streaming-programming-guide.html#output-operations-on-dstreams