DS - Python | Jun 9 - Jul 14 | Anand

Discussion in 'Big Data and Analytics' started by K Manoj, Jun 9, 2018.

  1. K Manoj

    K Manoj Moderator
    Staff Member Simplilearn Support

    Joined:
    Aug 4, 2017
    Messages:
    196
    Likes Received:
    18
    *thread locked for this batch learners.
    Please post your queries below..
     
    #1
  2. Purushothaman Manickam(4220)

    Alumni

    Joined:
    Jan 22, 2013
    Messages:
    6
    Likes Received:
    0
    Anand, Will client allow to download Python and use Python as a language for doing Data science?
     
    #2
  3. Purushothaman Manickam(4220)

    Alumni

    Joined:
    Jan 22, 2013
    Messages:
    6
    Likes Received:
    0
    Anand, in today's call you mentioned data scientist should do ML, But ML is a capabililty of an AI systems to understand the data and derive patterns from the data and predict outcomes. So would a data scientist be involved in designing the AI systems as well? Pls advice.
     
    #3
  4. Purushothaman Manickam(4220)

    Alumni

    Joined:
    Jan 22, 2013
    Messages:
    6
    Likes Received:
    0
    Anand, As Analytics maturity continuum, you mentioned descriptive, diagnostic, predictive and prescriptive. Is Cognitive Analytics next level into analytics maturity?
     
    #4
  5. _23750

    _23750 New Member

    Joined:
    Feb 20, 2018
    Messages:
    1
    Likes Received:
    0
    Hello all,

    Thanks for the thread
     
    #5
  6. Shyam Sunder_1

    Joined:
    Oct 10, 2017
    Messages:
    4
    Likes Received:
    0
    Manoj, This thread seems to be different from the rest in the sense where do we post our queries. I want help from Anand on the launch of Jupyter from the Anaconda prommpt He said some conda. which I couldnt catch. thx,
     
    #6
  7. Shyam Sunder_1

    Joined:
    Oct 10, 2017
    Messages:
    4
    Likes Received:
    0
    Hi Manoj, Pls ignore my previous post. I have been able to install Py on my system. Its easy if you follow the steps from google
     
    #7
  8. Girish Verma

    Girish Verma Member

    Joined:
    May 24, 2018
    Messages:
    5
    Likes Received:
    0
    Hi Anand, Can you please share some data and steps to do the Hypothesis testing? And how to build an effective model based on the samples collected from the data.
    If you can also share some case studies that may also help.
    Thank You
    Girish Verma
     
    #8
  9. Ekanth

    Ekanth Member

    Joined:
    Jan 31, 2018
    Messages:
    9
    Likes Received:
    0
    Hi manoj
    class 2 session has not been uploaded.Iam able to download only class 1 session.

    Regards

    EKANTH
     
    #9
  10. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Purushothaman, Cognitive Analytics as you may have known, tries to use the same behavior of human brain. Human brain functions based on nueral activity (neurons). The Neural Network aspect of data science builds algorithms based on functioning of nuerons (our brain cells).
    The ANN, CNN (Artificial Neural Network), Convolutional Neural Network techniques build algorithms that learns in the same way as that of our brain. so, Cognitive analytics has its roots on Neural Networks. Both Cognitive Analytics and Neural Networks help with deep learning.

    Deep learning techniques help in optimization.

    So under optimization - you can group, deep learning (neural network, cog computing). optimization can also be thought of as AI. the word optimization is generic and hence can be confusing.
     
    #10
  11. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Please check the google link which has instructions to set up anaconda. Do let me know if you have issues.

    https://drive.google.com/drive/u/0/folders/15lUEFDncOaFai5wAnJjKcmJXFUdYf7aq
     
    #11
  12. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    By "Client" i hope you are referring to your employer or customer.

    Python is an Open Source language that is now being adopted by many organizations. That said, there are some "enterprise software compliance teams", that monitor the installation of software . please check with them and they would definitely suggest you a way to install python at your work. Hope this response helps
     
    #12
  13. sree (3421)

    sree (3421) Member

    Joined:
    Mar 16, 2014
    Messages:
    2
    Likes Received:
    0
    Hi Manoj,

    I am not seeing the Class 2 downloadable link. Could you please upload it.

    Sridhar
     
    #13
  14. Shyam Sunder_1

    Joined:
    Oct 10, 2017
    Messages:
    4
    Likes Received:
    0
    #14
  15. Shashi Kiran C Vijayendra

    Joined:
    May 26, 2018
    Messages:
    3
    Likes Received:
    2
    Hello All,



    In the last class, I felt that we jumped into the tools like Anaconda and Jupyter without a complete introduction to package managers. I am not sure if its only me or if there are other like me who do not have any background in python. So, I did some basic study of these tools and I am summarizing it here.



    Anand (or anybody), please correct\amend this information if I have got something wrong.



    The way I understand it, there are 3 package\environment managers:



    1. PIP - This is Python's package manager developed by MIT which runs on Python environment. PIP is a recursive acronym that can stand for either "Pip Installs Packages" or "Pip Installs Python". PIP installs any Python package in any environment.
    PIP plagued by issues like

    1. Does not perform all the dependency checks. One must read the package instructions (requirements.txt file) to understand the dependencies and install the pre-requisites. Without this a developer would face runtime errors in the program.
    2. This is not an environment manager - This is most applicable to developers who could be maintaining different environments for data science, web development etc.
    3. It affects the system python installation - This is applicable to Linux which comes with python installed in the system core. Packages installed directly affects the system python and any version specific programs or packages will be affected.


    1. Conda- This was developed by Continum Analytics and is a cross platform package and environment manager. Conda installs any package within conda environment.
    The advantages of Conda over PIP:

    1. Takes care of and installs all the dependent packages, including non-python dependencies.
    2. Allows installation, switching and management of different versions of packages
    3. Anaconda Navigator (GUI tool) facilitates creating and managing different environments without having to worry with the nitty-gritties of package management.
    4. It supports packages written in python, R etc. This is a general package manager.
    5. Does not affect the system python
    6. Very effective for data science projects; it brings in all the packages needed for data science and machine learning.


    Anaconda - This a full distribution of the central software in the PyData ecosystem, and includes Python itself along with binaries for several hundred third-party open-source projects. Alternatively there is something called 'Miniconda' which contains the package manager conda only. Conda will subsequently need to be used to install other package from the scratch.



    1. VirtualEnv - This is an environment manager which utilizes pip to manage packages create virtual environments. Helps manage the different packages and versions across virtual environments.


    For hardcore developers, there is something called PyEnv, which encompasses both Anaconda and VirtualEnv allowing developers to manage their projects using both Ananconda and VirtualEnv. Additionally this ecosystem also allows developers to manage projects on different versions of python.



    Note:

    1. PIP can be used to install conda.
    2. PIP can be used to install Jupyter
    3. Conda is built upon PiP - it uses PIP under the hood.
    4. Anaconda Navigator uses virtualenv under the hood to manage the environment




    References:

    Pip vs Conda : Differences and Comparisons.

    Which Python Package Manager Should You Use?

    https://jakevdp.github.io/blog/2016/08/25/conda-myths-and-misconceptions/

    https://en.wikipedia.org/wiki/Anaconda_(Python_distribution)

    https://en.wikipedia.org/wiki/Pip_(package_manager)
     
    #15
    prasanna MVS likes this.
  16. Shashi Kiran C Vijayendra

    Joined:
    May 26, 2018
    Messages:
    3
    Likes Received:
    2
    Jupyter Notebook- is a web-based interactive computational environment that allows you to run live code, embed visualization, explanatory text and even videos in one place. The embedded visualization reflects the changes in the data in real time. This combined with the power of word processing makes it a good notebook that has all the textual information, your code and immediate output, all in one place. It supports 40 programming languages, integration with big data, it can be shared using email, dropbox etc,



    Reference:

    <https://en.wikipedia.org/wiki/IPython>

    What is Jupyter Notebook?
     
    #16
    prasanna MVS likes this.
  17. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Hi, please write to the support team. they will provide you the link. i will inform them as well
     
    #17
  18. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Thank you Sashi Kiran for taking time and sharing your notes and insights on package managers. My apologies that i did not cover these in detail in the class. will spend some time on this this week.

    The reason i did not cover them in detail was because
    1. jupyter installation via anaconda is very easy and a no brainer.
    2. for starters of python , its better to go with one IDE than exploring everything. Hence i narrowed down on Jupyter which is both an ide used for learning and being used by enterprises as well.
    3. Jupyter is a notebook that automatically helps you in learning best practices in python (indentation, comments, function doc strings etc)
     
    #18
  19. Shashi Kiran C Vijayendra

    Joined:
    May 26, 2018
    Messages:
    3
    Likes Received:
    2
    No problem, lets use this form as effectively as possible.. I understand that you have a plan..
     
    #19
  20. _27727

    _27727 Darshan
    Alumni

    Joined:
    Mar 29, 2018
    Messages:
    3
    Likes Received:
    0
    Hello Anand,
    In Jupyter while typing python code it doesnt show any suggestions
    eg: if we type "pr" it should show some suggestions starting from "pr" like print
    any insights on this
     
    #20
    Last edited: Jun 18, 2018 at 1:05 AM
  21. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Dear Manoj,
    Jupyter has extensive keyboard shortcuts that can be customized to help with "code completion."
    one of the "code completion" features which is automatic with jupyter is,
    1. when you press tab key after a command, keyword or method or function, jupyter will suggest you with options or complete the command. please find attached screen shots for below


    below is an example, when i type pr and hit tab key, its showing me all the commands, keywords that start with "pr"


    upload_2018-6-18_11-29-33.png upload_2018-6-18_11-29-33.png


    here is another example when i hit tab key after a "." , gives me all the methods that can be used
    upload_2018-6-18_11-34-48.png
     
    #21
  22. Girish Verma

    Girish Verma Member

    Joined:
    May 24, 2018
    Messages:
    5
    Likes Received:
    0
    Hi Anand, We have completed Numpy and Pandas is in progress. Can you please share some case studies how these tools are helping Business in analytics and in real time problems?

    Thank You
    With Regards
    Girish Verma
     
    #22
  23. Girish Verma

    Girish Verma Member

    Joined:
    May 24, 2018
    Messages:
    5
    Likes Received:
    0
    Hi Anand,
    I am getting the error when I am trying to run any command after importing numpy. Please let me know how to resolve this.

    Numpy_Error.PNG Numpy_Error1.PNG
     
    #23
  24. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Hi Girish,
    This error occurs when you have NOT run "import numpy as np" statement, but trying to run the statements following it.
    please run the import numpy as np statement and then rerun the line with arr=np.array(my_list)
     
    #24
  25. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0

    Girish,
    Numpy and Pandas are very useful in analysing data and there by business. In the life cycle of a data anlytics project numpy and pandas are quintessential for data acquisition, data wrangling and data exploration.

    The use cases are everywhere. any business with data, can use numpy and pandas for anlysing the data.

    one real time example that i worked on is i did a churn analysis for a leading chain of business in the beauty industry. They wanted me to
    look at their data and solve the following problems
    1. identify gaps in their business process and articulate the problem for them
    2. customer churn
    2.1 how
    2.2 when
    2.3 why
    3. provide recommendations to them on how to retain clients
    4. create a digital marketing strategy for them to do targeted marketing and increasing their revenue

    As you can clearly see,
    bullet 1 - is a descriptive analytics problem where the client did not clearly know what the problem was and asked me to come out with
    problem formulation

    bullets2 - is a typical problem that every client has and they threw that in to the foray.
    2.1 to answer how clients are leaving them, i needed to do RFM (Recency, Freq, Monetary) analysis and Customer life time value
    calculation. This helped me to come out with various segments of customers and their characteristics

    2.3 Why clients are leaving. this is diagnostic analytics problem. We explored the data using statistical tools and
    extracted insights to find out business insights like 20% of clients who leave , leave because of lack of connects from the company

    bullet .2.2 - Prediction of Churn - Predictive Analytics.
    When are clients leaving, was a difficult proposition for us, because in businesses like beauty industry, there is no track of
    customer churn. we used statistical techinques like survival analysis to approximate the churn rate.

    bullet3 & 4 - based off of 1 and 2, i had to predict when the clients are churning. this is a predictive analytics problem

    is a prescriptive analytics problem where in, i provided recommendations for 1. customer retention 2. data quality up keep 3.
    targeted
    marketing strategy and 4. optimizing current process efficiencies


    There are several other case studies in kaggle.com where real time business use cases are posted by companies and seek data scientists to solve their problems.
    One such is https://www.kaggle.com/kaggle/sf-salaries.

    Happy to Help
     
    #25
  26. Girish Verma

    Girish Verma Member

    Joined:
    May 24, 2018
    Messages:
    5
    Likes Received:
    0
    Thanks Anand.
    It will be great if we can also cover some solutions and examples for our further classes.
     
    #26
  27. Girish Verma

    Girish Verma Member

    Joined:
    May 24, 2018
    Messages:
    5
    Likes Received:
    0
    It works now. Thanks for the help.
     
    #27
  28. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Sure Girish. As part of this course, all of you are supposed to do a capstone project at the end. I ask of you to focus all your learning in such a way that your project gets done as per the design.

    Once you complete this project offered by simplilearn, you will have good confidence on how to approach, plan and execute a data science project.

    Then you can slowly hone your skills by taking up projects from kaggle.
     
    #28
  29. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
     
    #29
  30. Ekanth

    Ekanth Member

    Joined:
    Jan 31, 2018
    Messages:
    9
    Likes Received:
    0
    Hi anand

    Exception handling and python operator notebook is missing from the google drive link that you have shared with us.Can you upload these files.

    Regards

    EKANTH
     
    #30
  31. Ekanth

    Ekanth Member

    Joined:
    Jan 31, 2018
    Messages:
    9
    Likes Received:
    0
    Hi anand

    Also one more doubt.For logical operations in python can we use & | in place of and or.

    Ekanth
     
    #31
  32. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Hi Ekanth,
    Please check again. Its there under specific folder called Exception Handling.
     
    #32
  33. anand.s.subramaniam

    Alumni

    Joined:
    Mar 28, 2018
    Messages:
    12
    Likes Received:
    0
    Yes Ekanth. They can be used.
     
    #33
  34. _12026

    _12026 New Member

    Joined:
    Sep 8, 2017
    Messages:
    1
    Likes Received:
    0
    where to post the doubts? we can do that here only?
     
    #34
  35. sree (3421)

    sree (3421) Member

    Joined:
    Mar 16, 2014
    Messages:
    2
    Likes Received:
    0
    Hi Anand,

    What is the best way to read the large Excel / CSV files? If i am trying to read through jupyter notebook, but the browser is hanging.
     
    #35

Share This Page