Welcome to the Simplilearn Community

Want to join the rest of our members? Sign up right away!

Sign Up

WIPRO - BIGDATA ACADEMY

Srinivasulu Kuruva

Member
Customer
Hi Deshdeep/Megha,

In Simulation Test 4 -> 5th Question, getting below error. Log file is not available in the mentioned path.

org.apache.hadoop.mapred.InvalidInputException: Input path does not exist: hdfs://cloudlabns/user/Simplilearn/2013-09-15.log

Attached is the screenshot.
 

Attachments

  • LogFile.JPG
    LogFile.JPG
    46.3 KB · Views: 14

Srinivasulu Kuruva

Member
Customer
Hi Deshdeep/Megha,

For Project 2, Am getting the below error,

scala> val kPoints = points.takeSample(false, K, 34)
org.apache.hadoop.mapred.InvalidInputException: Input Pattern hdfs://cloudlabns/loudacre/devicestatus_etl/* matches 0 files


Attached is the screenshot.
 

Attachments

  • Loudacre.JPG
    Loudacre.JPG
    34.9 KB · Views: 15

Dhrub K Thakur

Member
Customer
Hi Kannan,

val test_names = sqlContext.read.format("com.databricks.spark.csv").option("header","true").option("inferSchema","true").load("/usr/local/clo/ven/test.csv")
17/02/14 05:52:04 WARN TaskSetManager: Lost task 0.0 in stage 1.0 (TID 1, jabdc10498.statestr.com, executor 1): java.io.FileNotFoundException: File file:/usr/local/clo/ven/test.csv does not exist
But file exist.
When i change the file location it's working fine

val test_names = sqlContext.read.format("com.databricks.spark.csv").option("header","true").option("inferSchema","true").load("/usr/local/clo/ven/spark-1.6.0-cdh5.9.0/test.csv")
test_names: org.apache.spark.sql.DataFrame = [year: int, Id: int, Name: string, Sex: string]

Is there any condition to put file inside spark home directory?
 

J V M Anand

New Member
Customer
Word Count Map Reducer program and Doctor Patient Program
 

Attachments

  • AnandWordCount.txt
    1.5 KB · Views: 5
  • AnandDoctorPatient.txt
    1.7 KB · Views: 4

Narendra_46

Member
Customer
Hi,

Had anyone received the experience certificate after completion of case studies and 4 simulation exams?

Thanks
Narendra Sadivi
 

Prashant_Nair

Well-Known Member
Simplilearn Support
Alumni
Trainer
PFA the notepad used in the classroom session!!!!
 

Attachments

  • WiproDay1TrainingHbase.txt
    7.1 KB · Views: 15

Megha_42

Well-Known Member
Simplilearn Support
Hi All,

Please find the documents that were discussed by trainer Kannan in class, for batches 3 and 4.

Feel free to use it as a reference for working with Hive, Kafka and Spark.
Also, the rest of the files which a a little large for the community, have been included in a .zip and been sent to you over email.


Regards
Megha
 

Attachments

  • Graph1.txt
    13 KB · Views: 12
  • hive.txt
    579 bytes · Views: 13
  • kafka.txt
    812 bytes · Views: 11
  • spark_mysql.txt
    770 bytes · Views: 12
  • spark_ppt.txt
    59.8 KB · Views: 16
  • Spark_streaming.txt
    623 bytes · Views: 9
  • dataframe.txt
    992 bytes · Views: 12

Neetu Gupta

Member
Alumni
Customer
Hi Shivang/Deshdeep ,
I am trying project 2 . I tried many times . One stage i am getting exception always . So unable to process further .
I am attaching screen images here . Kindly look in to it & provide me some solution of it .
 

Attachments

  • 1.jpg
    1.jpg
    101.7 KB · Views: 12
  • 2.jpg
    2.jpg
    107.7 KB · Views: 12
  • 3.jpg
    3.jpg
    97.1 KB · Views: 12
  • 4.jpg
    4.jpg
    82.4 KB · Views: 12

Ruchi Saklani

Member
Alumni
Hello, I am trying to do the Project1(Banking) in Spark but I stuck here---
I created the dataframe but when I am querying the data its showing error, even df.show() also showing error, screenshot is attached for you reference. please help
 

Attachments

  • bank1.jpg
    bank1.jpg
    227.3 KB · Views: 5

VijayKumarg

Member
Customer
Hi.. I am actually working on simulation tests, First question is having '
from pyspark.sql import HiveContext'. When type in spark, i am getting below error, I am not sure how to work with pySpark.
scala> from pyspark.sql import HiveContext<console>:1: error: ';' expected but '.' found. from pyspark.sql import HiveContext ^

Please let me know if anything needs to be import or anything else.
 

archana_49

Member
Trainer
Hi Learners,
please find the code and notes.
 

Attachments

  • SumReducer.txt
    1.5 KB · Views: 4
  • WordCount.txt
    3.2 KB · Views: 6
  • WordCount.txt
    3.2 KB · Views: 4
  • Sqoop.txt
    17 KB · Views: 3

Ayush Singh

Customer
Customer
I have set the $JAVA_home variable to the given jdk specifications but when I use the command
cd $JAVA_HOME it shows as directory not found. How can I fix this issue?
 

Ayush Singh

Customer
Customer
I have the default 1.7.0_67-cloudera version as shown in my webconsole. Is it necessary to set the JAVA_HOME path to jdk1.8.0? This version is present in jdk64 folder which is not showing up the usr folder in my system. Kindly help me out.
 

Ayush Singh

Customer
Customer
In project 1, question 5:
"Check if age matters in marketing subscription for deposit "
exactly what has to be analyzed in this question? My take is that to check after what age the no. of subcriptions increase to above 50% of total subs. Any other way of analyzing this data?
 

A V S Aditya

Customer
Customer
Hello,

I am trying to work on the project using the cloudlab.
What is the java path that I need to use for the hadoop projects?

Thanks and Regards
Aditya
 

A V S Aditya

Customer
Customer
I have the default 1.7.0_67-cloudera version as shown in my webconsole. Is it necessary to set the JAVA_HOME path to jdk1.8.0? This version is present in jdk64 folder which is not showing up the usr folder in my system. Kindly help me out.
Can you help me how to set the path for JAVA?
It is not set by default in cloudlab webconsole.

Thanks
Aditya
 
Top