Squirrel SQL 클라이트 툴에 

https://acadgild.com/blog/squirrel-gui-phoenix


MACOS에서 Squirrel SQL을 설치했지만 실행이 되지 않는다. 아마도 path 이슈일 것 같다.





간단히 해결한 방법은 다음과 같다.


먼저 설치 jar로 설치하고 디폴트 설치 위치로 /Applications/SQuirreLSQL.app/에 두게 한다.




alias squirrel='/Applications/SQuirreLSQL.app/Contents/MacOS/squirrel-sql.sh'


mkdir -p /Applications/SQuirreLSQL.app/Contents/MacOS/lib/


cp /Applications/SQuirreLSQL.app/Contents/Resources/Java/lib/* /Applications/SQuirreLSQL.app/Contents/MacOS/lib/


cp /Applications/SQuirreLSQL.app/Contents/Resources/Java/squirrel-sql.jar  /Applications/SQuirreLSQL.app/Contents/MacOS/




squirrel을 실행하면 클라이언트 툴이 실행된다.



Posted by '김용환'
,


spark readStream()으로 읽은 DataSet을 카산드라에 저장하는 예시 코드이다. 






import com.datastax.driver.core.Session

import com.datastax.spark.connector.cql.CassandraConnector

import org.apache.spark.sql.ForeachWriter




val spark = ...


val ds = spark.readStream()

...




val connector = CassandraConnector.apply(spark.sparkContext.getConf)

val session = connector.openSession


def processRow(value: (String, String, String, String)) = {

  connector.withSessionDo { session =>

    session.execute(s"insert into test.log(ktag, ts, uuid, log) values('  ${value._1}', '${value._2}', '${value._3}', '${value._4}'   )")

  }

}


    

val writer = new ForeachWriter[(String, String, String, String)] {

  override def open(partitionId: Long, version: Long) = true


  override def process(value: (String, String, String, String)) = {

    processRow(value)

  }

  override def close(errorOrNull: Throwable) = {

    println(errorOrNull)

  }


}


val query = ds.writeStream.queryName("test").foreach(writer).start


query.awaitTermination()





build.sbt에는 spark-cassandra-connector를 추가한다.

libraryDependencies += "com.datastax.spark" %% "spark-cassandra-connector" % "2.0.2"



Posted by '김용환'
,