Clarifications needed on Hadoop course

Discussion in 'Simplilearn Support' started by sharathk.neo, Jun 24, 2015.

  1. sharathk.neo

    sharathk.neo Sharath
    Alumni

    Joined:
    Jun 15, 2015
    Messages:
    1
    Likes Received:
    0
    Can someone give some clarification on the below queries.

    1. Do i need to install Ubuntu OS in my system as the demos included in the course as all been done using Ubuntu OS?

    2. Do i need to install Ubuntu desktop client on HadoopPseudoDistributedServer VM provided by simplilearn?

    3. I ran the start-all.sh command in VM but unable to find NameNode and JobTracker services. I even tried hadoop namenode -format command then ran start-all.sh but it was of no use.

    4. Could you please provide a detailed document on the steps need to be followed by online training candidates so that we wont face any problems during installation and practising exercises?

    5. How do i install eclipse on VM?

    6. I followed the steps provided in the document provided in the download center to install ubuntu desktop version but the VM did not prompt me to intsall Ubuntu OS

    Thanks in anticipation
     
    #1
  2. Shruti

    Shruti -

    Joined:
    Nov 10, 2014
    Messages:
    61
    Likes Received:
    48
    Hi Sharathk - We have forwarded your query to our respective team. We should have a solution by EOD.
     
    #2
  3. Mukund K. Mishra

    Mukund K. Mishra Well-Known Member
    Staff Member Trainer

    Joined:
    Feb 18, 2015
    Messages:
    88
    Likes Received:
    12
    Hi,
    Please find below the solution for each of the queries.

    1. Do i need to install Ubuntu OS in my system as the demos included in the course as all been done using Ubuntu OS?

    Ans. You can install Ubuntu OS along with your existing Windows but then in that case you will be restricted to only one OS . The best solution is download vmwareplayer or VmWare Workstation or Oracle VirtualBox . In case you are using the image of Simplilearn you can open it in VMware player.

    2. Do i need to install Ubuntu desktop client on HadoopPseudoDistributedServer VM provided by simplilearn?

    Ans. Simplilearn has provided you virtual image of Ubuntu that is not an installer but an image which can be opened on any of the above tool

    3. I ran the start-all.sh command in VM but unable to find NameNode and JobTracker services. I even tried hadoop namenode -format command then ran start-all.sh but it was of no use.

    Ans. If you miss any of the step you will not be able to do it properly

    4. Could you please provide a detailed document on the steps need to be followed by online training candidates so that we wont face any problems during installation and practising exercises?

    Ans. - Installation step of hadoop 1.x
    ===============================================================================
    Download
    http://www.traffictool.net/vmware/ubuntu1404t.html
    -------------------------
    Details
    -------
    VM image size 959 MB
    Disk 20+100 GB
    VM RAM 1024 MB
    VMware Tools yes
    User/password user/password
    Root password password
    -----------------------------
    Your home directory is
    /home/user

    Create the below Folder Structure
    /home/user/lab/software
    /home/user/lab/hdfs/namenodep
    /home/user/lab/hdfs/datan1
    /home/user/lab/hdfs/checkp
    /home/user/lab/mapred/system
    /home/user/lab/mapred/system/local1



    Setup openssh (http://askubuntu.com/questions/318012/openssh-server-package-not-available-on-12-04-2)
    -------------------------------
    - try to install openssh-server (sudo apt-get install openssh-server)


    sudo apt-get install openssh-client
    sudo apt-get install openssh-server
    -----------------

    Now check that you can ssh to the localhost without a passphrase:
    $ ssh localhost


    If you cannot ssh to localhost without a passphrase, execute the following commands:
    $ ssh-keygen -t dsa -P '' -f ~/.ssh/id_dsa
    $ cat ~/.ssh/id_dsa.pub >> ~/.ssh/authorized_keys


    //Java Installation
    Download Download Oracle Java

    - Linux x86 136.76 MB jdk-7u71-linux-i586.tar.gz

    Create the installation folder
    - sudo mkdir -p /usr/lib/jvm
    sudo mv jdk-7u71-linux-i586.tar.gz /usr/lib/jvm

    Go to
    /usr/lib/jvm
    untar
    sudo tar-xvf jdk-7u71-linux-i586.tar.gz


    - make the following entry in /etc/profile

    JAVA_HOME=/usr/lib/jvm/jdk1.7.0_71
    PATH=$PATH:$JAVA_HOME/bin
    export JAVA_HOME
    export PATH

    Go to home typing cd
    and run following

    . /etc/profile

    java –version

    --------------
    //MySQl Installation

    sudo apt-get update
    sudo apt-get install mysql-server-5.6

    //The system will prompt for password in between - Provide password

    root@ubuntu1404:~# mysql -uroot -p
    Enter password: password

    //Hadoop 1 Installation

    create folder lab and under that software in home

    change in .bashrc file

    export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_71
    export HADOOP_INSTALL=/home/user/lab/software/hadoop-1.0.3
    export HADOOP_HOME=/home/user/lab/software/hadoop-1.0.3
    export PATH=$PATH:$HADOOP_INSTALL/bin

    - Go to home by typing cd command
    - . .bashrc

    /////
    hadoop-env.sh

    export JAVA_HOME=/usr/lib/jvm/jdk1.7.0_71

    ///
    core-site.xml

    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>



    <configuration>
    <property>
    <name>fs.default.name</name>
    <value>hdfs://localhost:8020</value>
    <final>true</final>
    </property>
    </configuration>
    ---
    hdfs-site.xml

    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>dfs.name.dir</name>
    <value>/home/user/lab/hdfs/namenodep</value>
    <final>true</final>
    </property>
    <property>
    <name>dfs.replication</name>
    <value>1</value>
    </property>
    <property>
    <name>dfs.data.dir</name>
    <value>/home/user/lab/hdfs/datan1</value>
    <final>true</final>
    </property>
    <property>
    <name>dfs.checkpoint.dir</name>
    <value>/home/user/lab/hdfs/checkp</value>
    <final>true</final>
    </property>

    </configuration>

    //////

    mapred-site.xml
    ----------
    <?xml version="1.0"?>
    <?xml-stylesheet type="text/xsl" href="configuration.xsl"?>

    <!-- Put site-specific property overrides in this file. -->

    <configuration>
    <property>
    <name>mapred.job.tracker</name>
    <value>localhost:8021</value>
    <final>true</final>
    </property>
    <property>
    <name>mapred.local.dir</name>
    <value>/home/user/lab/mapred/system/local1</value>
    <final>true</final>
    </property>
    <property>
    <name>mapred.system.dir</name>
    <value>/home/user/lab/mapred/system</value>
    <final>true</final>
    </property>
    <property>
    <name>mapred.tasktracker.map.tasks.maximum</name>
    <value>3</value>
    <final>true</final>
    </property>
    <property>
    <name>mapred.tasktracker.reduce.tasks.maximum</name>
    <value>3</value>
    <final>true</final>
    </property>
    <property>
    <name>mapred.child.java.opts</name>
    <value>-Xmx400m</value>
    </property>

    </configuration>

    ///
    hadoop namenode-format

    when prompted press capital Y





    ================================================================================




    5. How do i install eclipse on VM?

    Ans . If you download the GUI (Graphical User Interface) image of Ubuntu as mentioned above you can install Eclipse there else create jar fie in windows and using winscp take the file to appropriate folder of Ubuntu

    6. I followed the steps provided in the document provided in the download center to install ubuntu desktop version but the VM did not prompt me to intsall Ubuntu OS
    Ans.Follow the step provided in the response of your Q.4

    Please let us know in case you face any issue but cautiously follow the above steps.
     
    #3
    Linus likes this.
  4. Hello Mukund - I have had the same query and I am trying to follow the steps you have mentioned. Will let you know once it is resolved. Thanks for the solution!!
     
    #4
  5. sharath

    sharath Guest

    Hello Mukund,

    Could you please arrange a demo on hadoop installation as I am finding really difficult and confusing to install hadoop.

    Thanks,
    Sharath
     
    #5
  6. Ragunathan Parasuraman

    Alumni

    Joined:
    Jul 31, 2015
    Messages:
    2
    Likes Received:
    0
    Did you had the demo ? I am also finding it very hard to understand what needs to be done once we have entered the hadoop server with simplilearn credentials...
     
    #6
  7. Mukund K. Mishra

    Mukund K. Mishra Well-Known Member
    Staff Member Trainer

    Joined:
    Feb 18, 2015
    Messages:
    88
    Likes Received:
    12
    Please get in touch with your course coordinator for demo
     
    #7
    Gerald Jaideep(1121) likes this.

Share This Page