Advertisement
Not a member of Pastebin yet?
Sign Up,
it unlocks many cool features!
- def valueWhen[T](m: String, n: String, v: T): Option[T] = if(m == n) Option(v) else None
- def longValWhen(m: String) = udf[Option[Long], String, Long]((n: String, value: Long) => valueWhen[Long](m, n, value))
- .withColumn("signupUserId", longValWhen("sign_up")($"name", $"userId"))
- g.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 24.0 failed 4 times, most recent failure: Lost task 0.3 in stage 24.0 (TID 875, 10.188.246.93, executor 0):
- java.lang.RuntimeException: Null value appeared in non-nullable field:
- - field (class: "scala.Long", name: "signupUserId")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement