DS WITH Python |July 01 - July 19| Samridhi

Discussion in 'Big Data and Analytics' started by Kunal Guwalani, Jun 30, 2019.

  1. Kunal Guwalani

    Kunal Guwalani Well-Known Member
    Simplilearn Support

    Joined:
    Jul 17, 2018
    Messages:
    168
    Likes Received:
    14
    Hi All,

    Use this thread to ask python queries.
     
    #1
  2. _40206

    _40206 Member

    Joined:
    Sep 13, 2018
    Messages:
    2
    Likes Received:
    0
    Hi Samridhi,

    Can you please provide your Google drive link that was shared in Class1
     
    #2
  3. Maheswari L Patil

    Joined:
    May 16, 2019
    Messages:
    2
    Likes Received:
    0
    Assignment: Difference between remove, pop, del

    remove() will delete the matching element

    · del and pop removes the element at a specific index.

    · Pop is returns the object.

    · Remove is the only one that searches object (not index).




    remove() removes the first matching value and does not do anything with the indexing

    list = [1, 2, 3, 2]
    list.remove(2)
    list

    output:[1, 3, 2]


    del removes the item at a specific index

    list = [3, 2, 2, 1]
    del list[1]
    list

    output:[3, 2, 1]


    pop removes the item at a specific index and returns it

    list = [4, 3, 5]
    list.pop(1)

    3

    list

    output:[4, 5]
     
    #3
  4. veda vasavi erukulla

    Joined:
    May 3, 2019
    Messages:
    2
    Likes Received:
    0
    Assignment difference between del,pop and reove
    del and pop deletes the elements at specified indeces of the list
    pop returns the deleted element
    del is uses if multiple elements are to be deleted at a time at specified indeces
    remove deletes the first occurance of the element specified,it searches for the element and deletes the first occurance of it.
    Example shown below:

    upload_2019-7-4_13-54-46.png
     
    #4
  5. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    difference between remove ; delete and pop
    remove
    : this operation directly removes the particular element of list.
    for example
    x = ["raj" ,"kumar","amit","gupta",33,44,44,55,66,88,66]
    if we want to directly remove the element 33 mentioned in list then we can directly use below
    x.remove(33)
    Delete: if we want to remove any element by its index then we use delete,
    example: same example mentioned above via delete
    x.delete[4]
    but if there are multiple occurences of any element then delete will remove only the element addressed in list
    POP: removes the last element of list
    e.g. x.pop()
    will remove last element 66 in the list
     
    #5
  6. Abhishek Suman

    Joined:
    Jun 17, 2019
    Messages:
    2
    Likes Received:
    0
    # Assignment 1 Dictionary Fruit Color inversion
    # Program to inverting the dictionary

    # Attempting to create a list of dictionaries
    dict_c2f = dict()
    for fruit in d1:
    color = d1[fruit]
    if(color not in dict_c2f):
    dict_c2f[color] = [fruit]
    else:
    dict_c2f[color].append(fruit)
    list_c2f = []
    for x in dict_c2f:
    d = {}
    d[x] = dict_c2f[x]
    list_c2f.append(d)
    for x in list_c2f:
    print(x)

    [output]
    {'Red': ['Apple', 'Pomogranate']}
    {'Green': ['Guava', 'Grapes']}
    {'Yellow': ['Carror', 'Banana']}

    # Assignment 2 - Difference between pop, del and remove
    l = [2,3,4,5,6,7,8]
    print(l.pop(4))
    # Popped out Last element by default if no parameter is passed. or else
    # Poped out the the element from the list at an index value which is passed as parameter. Ex l[4] = 6 popped out
    [Output] 6

    del(l[3])
    # del function deletes the parameter. Whereas Parameter could be a list or a list element.
    # Ex l[3] = 5 deleted from the list
    print(l)

    [Output]
    [2, 3, 4, 7, 8]

    l.remove(3)
    # removes 3 from the list. i.e remove will remove the element if it founds in the list.
     
    #6
  7. veda vasavi erukulla

    Joined:
    May 3, 2019
    Messages:
    2
    Likes Received:
    0
    Assignment-2: Inverting the dictionary twice

    upload_2019-7-4_16-19-48.png
     
    #7
  8. _39901

    _39901 New Member

    Joined:
    Sep 10, 2018
    Messages:
    1
    Likes Received:
    0
    @support Team:
    Share the URL to download Recorded sessions - as my ID was registered at the backend.

    Thanks
     
    #8
  9. Karthikeyan Sekizhar

    Joined:
    Jun 12, 2019
    Messages:
    2
    Likes Received:
    0
    Assignment 1: del, pop and remove for lists.

    Screenshot 2019-07-06 at 10.12.01 AM.png

    Assignment 2: Given 'fruit to color', make 'color to fruit', then back to 'fruit to color'.

    Screenshot 2019-07-06 at 10.12.14 AM.png
     
    #9
  10. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    Can anybody let me know where i can see the assignments? i would also like to do hands on to it.
     
    #10
  11. Simon_35

    Simon_35 Member

    Joined:
    Jun 19, 2019
    Messages:
    3
    Likes Received:
    0
    Hi, I could not download or access the Google drive python files, it always gives me an error on something about Python. I installed Ananconda a few times, the problem are still there.
    Any ideas what to do?

    Thanks
    Simon
     
    #11
  12. Simon_35

    Simon_35 Member

    Joined:
    Jun 19, 2019
    Messages:
    3
    Likes Received:
    0
    Hi, I could not download or access the Google drive python files, it always gives me an error on something about Python. I installed Ananconda a few times, the problem are still there.
    Any ideas what to do?

    Thanks
    Simon
     
    #12
  13. _40206

    _40206 Member

    Joined:
    Sep 13, 2018
    Messages:
    2
    Likes Received:
    0
    Hi Samridhi,

    Can you please provide your Google drive link that was shared in Class1.


    Thanks,
    Amulya
     
    #13
  14. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    Can you please provide your Google drive link that was shared in Class1 before. In addition, could you please share your *.ipynb files that you reviewed on Sat and Sun (July 6 and July 7).

    thank you so much in advance,
    Irene
     
    #14
  15. SRIDHAR MOTHE

    SRIDHAR MOTHE Member

    Joined:
    Apr 24, 2019
    Messages:
    4
    Likes Received:
    0
    Hi Samridhi,

    I'm facing an issue, with Replacing NaN values, tried couple of options, to achieve the same. I didn't get any errors, but NaN's still remain. Enclosing a screenshot of the 2 methods i tried and also the NaNs.

    Please let me know, what am i doing wrong

    Thanks
    Sridhar DS_Python_Project_NYC_Issue_SM_12-07-2019.png
     
    #15
  16. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    I run into the issue with accessing UserID=2696, is this the correct way to access the information on the specific user? Please see below:



    upload_2019-7-12_18-1-55.png
     
    #16
  17. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    #17
  18. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi,

    2696 is an integer, why you have enclosed in single quotes? Please remove the quotes and try.

    Regards,
    Samridhi
     
    #18
  19. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi Sridhar,

    Replace function returns a value. This value has to be captured in a variable.
    Hi Sridhar,

    Please add argument inplace = True in the replace function for the changes to be reflected inplace in the dataframe.

    Regards,
    Samridhi
     
    #19
  20. Moulay Hicham El idrissi

    Joined:
    May 13, 2019
    Messages:
    1
    Likes Received:
    0
    Hi Samridhi,

    Could you please assist me with this?
    i'm getting this error while i'm testing the following code with applying ANOVA (see Screenshots) :

    ValueError: shape mismatch: value array of shape (1,6) could not be broadcast to indexing result of shape (1,2)

    it keeps giving the error no matter what Explanatory variable i'm choosing with the Response "Rating"; i tried with only one single independent and with multiple independents variable at same time but getting the same error!!!

    upload_2019-7-13_17-11-25.png
    upload_2019-7-13_17-12-19.png

    Thx
    Moulay ( AKA Emh.nawl)
     

    Attached Files:

    #20
  21. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20

    Hi,

    Please convert Gender into numerical using either map or replace function in order to apply any model on this.

    Regards,
    Samridhi
     
    #21
  22. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    For the Movielens project, Machine Learning section, I plotted the features (MovieID, Age, Occupation) and the response (Rating) using scatter plots and I did not get a linear regression. Could you please look at the graphs below and help identify what would this be? Logistical Regression or something else? This definitely does not look like a linear regression:

    upload_2019-7-16_22-24-12.png

    thank you so much in advance,
    Irene
     
    #22
  23. SRIDHAR MOTHE

    SRIDHAR MOTHE Member

    Joined:
    Apr 24, 2019
    Messages:
    4
    Likes Received:
    0
    NYC Project:

    We have a Hypothesis test to perform:

    5. Perform a statistical test for the following:
    Please note: For the below statements you need to state the Null and Alternate and then provide a statistical test to accept or reject the Null Hypothesis along with the corresponding ‘p-value’.

    • Whether the average response time across complaint types is similar or not (overall)
    • Are the type of complaint or service requested and location related?

    for the second test, we need location information, which is available in the Dataset as "Location", which is a collection of Longitude and Latitude. But this has around 3000 missing values.

    How do we impute this column, before going for Hypothesis testing?

    Please let me know

    Thanks
    Sridhar
     
    #23
  24. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi Irene,
    Linear Regression here gives very poor results. Please use multi-class classification techniques (LoRe, KNN) as discussed in the class (in Iris dataset)

    Regards,
    Samridhi
     
    #24
  25. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi Sridhar,

    Please use Location Type variable for this.

    Regards,
    Samridhi
     
    #25
  26. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0

    Thank you Samridhi. I used both Logistic Regression and KNN to see which of them will give best results. It seems that KNN gives better results than LogReg.
     
    #26
  27. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    Last question in the Movielen project asks to create train and test datasets, which I did. They are also asking to create a histogram for movie, age and occupation. Could you please clarify WHAT the ask is to do
    1. ONE (1) histogram with all of the parameters - movie, age and occupation? OR, a SEPARATE histogram for each of the attributes - movie histogram, age histogram and occupation histogram.
    2. Also, please confirm whether the histogram should use a Master_Data (combined data) dataset to plot OR data of 500 observations. And, should test data OR train data should be plotted or a histogram for test and train data?

    Could you please clarify the question as the ask is not very clear. I tried already plotting histogram with two attributes (using a Master_Data and data500) - age and occupation and it works OK. As soon as you add another attribute (i.e. MovieID), the histogram does not look good anymore. Thus, I wonder what really they meant by asking in this case. See below the different versions of histograms and graphs. Can you please clarify.

    Please see the histograms below:

    upload_2019-7-18_10-47-9.png

    upload_2019-7-18_10-48-31.png

    upload_2019-7-18_10-48-55.png
     
    #27
  28. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    1. Seperate histograms along with an analysis about the distribution shape
    2. You can use any data either movie or combined data.

    Regards,
    Samridhi
     
    #28
  29. VIKASH ORAON

    VIKASH ORAON New Member

    Joined:
    Feb 27, 2019
    Messages:
    1
    Likes Received:
    0
    Hi Samridhi,

    Could you please provide us feedback on our projects. What we did well and what we need to improve on.

    Thank you,
    Vikash
     
    #29
  30. Yeravelli Santhosh

    Yeravelli Santhosh New Member

    Joined:
    May 25, 2019
    Messages:
    1
    Likes Received:
    0
    Hi,

    In movie data set while applying the linear regression it giving the -1.15 value.
    Could you please help me
    Features500 contains col=age,occupation
    label500 contains col=rating

    upload_2019-7-20_20-50-47.png
     
    #30
  31. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    As discussed in the class you can create subplots. Also, you can make 3 kde curves in the same plot for comparison purpose.

    Regards,
    Samridhi
     
    #31
  32. Kapil Goyal_1

    Kapil Goyal_1 Member

    Joined:
    Jun 3, 2019
    Messages:
    2
    Likes Received:
    0
    Hello Mam ,

    Can you please me in the first point of Movies Project .
    • Import the three datasets
    • Create a new dataset [Master_Data] with the following columns MovieID Title UserID Age Gender Occupation Rating. (Hint: (i) Merge two tables at a time. (ii) Merge the tables using two primary keys MovieID & UserId)

    While loading the data getting below error .

    Below is the command I used .

    movies = pd.read_csv("C:\\Users\kapil goyal\\Downloads\Projects for Submission-20190720T132631Z-001.zip\\Projects for Submission\\movies.dat")
     
    #32
  33. Kapil Goyal_1

    Kapil Goyal_1 Member

    Joined:
    Jun 3, 2019
    Messages:
    2
    Likes Received:
    0
    Hello Mam ,

    Need to clear one more thing which project we have to submit either which shown in LMS Tool or you have shared with the name project4 .

    Thanks and Regards
    Kapil Goyal
     
    #33
  34. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    In LMS Assessment Project Tab, the requirements of the Project Movielen questions are different from the requirements of the Self Learning Course Course Resources Project in LMS. The LMS Assessment Project Tab requirements are exactly the same as you had in your version originally.

    Please confirm which version of the project requirements in LMS should I pick - Assessment Project Tab requirements or Self Learning Course Resources Project.

    thank you
    Irene
     
    #34
  35. Irene Boudarov

    Irene Boudarov Active Member
    Alumni

    Joined:
    Jan 24, 2019
    Messages:
    28
    Likes Received:
    0
    Hi Samridhi,

    I used two scaling techniques (minmax scaler and Robust scaler) for Movielen project. After the scaling techniques in both cases (minmax scaler and Robust scaler), I implemented logistic regression model and the Deviance came up as NaN and AIC came up as NaN. Could you please help me understand how did it came up this way? What does this mean when Deviance is NaN and AIC is NaN? Here are images below.

    thank you so much in advance,
    irene

    upload_2019-7-25_1-30-53.png
     
    #35
  36. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    Hi Samridhi,

    I listened to recordings can you please clarify the following questions:
    1.difference between the merging, concatenation and join operations in pandas?
    2. Enumerate concept in python?
     
    #36
    Last edited: Jul 25, 2019
  37. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi Rajkumar,

    Merge is used to merge 2 dataframes (inner join / outer joins based on some primary key), while concatenate is used to attach 2 dataframes row-wise or col wise.

    Enumerate is used to give index to each component of an object.

    Regards,
    Samridhi
     
    #37
  38. Samridhi Dutta

    Samridhi Dutta Well-Known Member
    Trainer

    Joined:
    Aug 16, 2017
    Messages:
    157
    Likes Received:
    20
    Hi Irene,

    You are using Binomial Regression to address a multi-nomial problem. Please use logistic regression from sklearn library, the way we used for iris multi-class classification in the class.

    Regards,
    Samridhi
     
    #38
  39. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
     
    #39
  40. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    Hi Samridhi,
    I was doing hands on in tuples while doing the practical examples i have the following clarification mentioned below:
    i created a tuple (its a nested tuple) example mentioned below:

    Clarification1:
    test =(["hello","rajkumar",33,45,66],("rajesh","atul",33,45.8),[55.6,66.3,99.85],(15,10,3,6,9))
    how would i print output like
    first element of tuple is: List
    second element of tuple is: Tuple
    third element of tuple is: List
    fourth element of tuple is: Tuple

    Clarification 2:
    test =(["hello","rajkumar",33,45,66],("rajesh","atul",33,45.8),[55.6,66.3,99.85],(15,10,3,6,9))
    how would i get the output printed as:
    This tuple contain 2 lists and 2 tuples

    Clarification3:
    test =(["hello","rajkumar",33,45,66],("rajesh","atul",33,45.8),[55.6,66.3,99.85],(15,10,3,6,9))
    How would i get output as below:
    The length of first element in tuple (list) is: 5
    The length of second element in tuple is: 4
    The length of third element in tuple is:3
    The lenght of fourth element in tuple is:5

    it would be great if you please provide me solution on this.
     
    #40
  41. SUNNY BHAVEEN CHANDRA

    SUNNY BHAVEEN CHANDRA Well-Known Member

    Joined:
    Feb 4, 2019
    Messages:
    56
    Likes Received:
    8
    Hi Rajkumar,

    Following is the solution to your query -

    upload_2019-8-9_2-26-20.png

    I hope this helps. And let me know if you have any further query.

    Regards,
    Sunny,
    Teaching assistant,
    Team Simplilearn
     
    #41
  42. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    thanks for helping me
     
    #42
  43. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    hi sir,
    i was trying to understand the concept of deep copy and shallow copy
    both are performing the same things
    Shallow copy
    i create a list and create copy of it using copy module
    if i change in anyways then the changes are not reflecting -

    deep copy
    i performed the same operation as mention above
    i noticed the same behaviour there also,

    then how do deep copy differ from shallow copy?
    I am not clear on this can you please help me on this and clarify this doubt?
     
    #43
  44. Rajkumar Tripathi

    Joined:
    May 20, 2019
    Messages:
    10
    Likes Received:
    0
    any update on my above query? i did not recieve any answer.
     
    #44
  45. Nishant_Singh

    Nishant_Singh Well-Known Member
    Simplilearn Support

    Joined:
    Aug 1, 2018
    Messages:
    222
    Likes Received:
    31
    Hi Rajkumar,

    In Python, there are two ways to create copies :
    1.) Deep copy
    2.) Shallow copy

    In order to make these copy, we use copy module. We use copy module for shallow and deep copy operations.

    # Deep Copy :
    Deep copy is a process in which the copying process occurs recursively. It means first constructing a new collection object and then recursively populating it with copies of the child objects found in the original. In case of deep copy, a copy of object is copied in other object. It means that any changes made to a copy of object do not reflect in the original object. In python, this is implemented using “deepcopy()” function.

    # Shallow Copy :
    A shallow copy means constructing a new collection object and then populating it with references to the child objects found in the original. The copying process does not recurse and therefore won’t create copies of the child objects themselves. In case of shallow copy, a reference of object is copied in other object. It means that any changes made to a copy of object do reflect in the original object. In python, this is implemented using “copy()” function.

    # Reference Image :
    [​IMG]

    # Note :

    The difference between shallow and deep copying is only relevant for compound objects (objects that contain other objects, like lists or class instances):

    * A shallow copy constructs a new compound object and then (to the extent possible) inserts references into it to the objects found in the original.

    * A deep copy constructs a new compound object and then, recursively, inserts copies into it of the objects found in the original.

    I hope that this will help you.

    Regards,
    Team Simplilearn
     
    #45

Share This Page