Advertisement
Guest User

Untitled

a guest
Mar 27th, 2017
54
0
Never
Not a member of Pastebin yet? Sign Up, it unlocks many cool features!
text 0.59 KB | None | 0 0
  1. def valueWhen[T](m: String, n: String, v: T): Option[T] = if(m == n) Option(v) else None
  2.  
  3. def longValWhen(m: String) = udf[Option[Long], String, Long]((n: String, value: Long) => valueWhen[Long](m, n, value))
  4.  
  5. .withColumn("signupUserId", longValWhen("sign_up")($"name", $"userId"))
  6.  
  7. g.apache.spark.SparkException: Job aborted due to stage failure: Task 0 in stage 24.0 failed 4 times, most recent failure: Lost task 0.3 in stage 24.0 (TID 875, 10.188.246.93, executor 0):
  8. java.lang.RuntimeException: Null value appeared in non-nullable field:
  9. - field (class: "scala.Long", name: "signupUserId")
Advertisement
Add Comment
Please, Sign In to add comment
Advertisement