How to handle NULL or empty values with dataframe using scala

Discussion in 'Big Data and Analytics' started by Kishore Rajkumar, Jul 2, 2017.

  1. Kishore Rajkumar

    Joined:
    Feb 25, 2016
    Messages:
    4
    Likes Received:
    0
    Hi,

    I am working on the Movie Review Analysis project with spark dataframe using scala. I was trying to sort the rating column to find out the maximum value but it is throwing "java.lang.NumberFormatException: empty String" exception.

    Upon going through the data file, I observed that some of the rows have empty rating and runtime values.
    Can anyone please let me know if the data file which is given is a proper file?
    If yes, then how to handle NULL or empty values while working with dataframe?
     
    #1
  2. Megha_42

    Megha_42 Well-Known Member
    Simplilearn Support

    Joined:
    Dec 15, 2016
    Messages:
    206
    Likes Received:
    9
    #2
    Kishore Rajkumar likes this.
  3. _6230

    _6230 Well-Known Member
    Alumni

    Joined:
    Apr 4, 2017
    Messages:
    185
    Likes Received:
    8
    solution looking good. Thanks
     
    #3
  4. Kishore Rajkumar

    Joined:
    Feb 25, 2016
    Messages:
    4
    Likes Received:
    0
    Thanks Megha!
     
    #4

Share This Page