Welcome to the Simplilearn Community

Want to join the rest of our members? Sign up right away!

Sign Up

Big data on aws| Oct 24 -dec 5 | Shajee Pulukkool

Hi Shajee,

When I am running the below python code within an EC2 Instance I am getting the below mentioned error. Request your help.



import json
import datetime
import random
#import testdata
#from boto3 import kinesis
import boto3

#kinesis = kinesis.connect_to_region("us-east-1")
kinesis_client = boto3.client('kinesis', region_name='us-east-1')

def getData(iotName, lowVal, highVal):
data = {}
data["iotName"] = iotName
data["iotValue"] = random.randint(lowVal, highVal)
return data

while 1:
rnd = random.random()
if (rnd < 0.01):
data = json.dumps(getData("DemoSensor", 100, 120))
kinesis_client.put_record(StreamName="san-project-stream", Data=data, PartitionKey="DemoSensor")
print ('***************************** anomaly ************************* {data}')
else:
data = json.dumps(getData("DemoSensor", 10, 20))
kinesis_client.put_record(StreamName="san-project-stream", Data=data, PartitionKey="DemoSensor")
print (data)



File "datainject.py", line 25, in <module>
kinesis_client.put_record(StreamName="san-project-stream", Data=data, PartitionKey="DemoSensor")
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/client.py", line 357, in _api_call
return self._make_api_call(operation_name, kwargs)
File "/home/ec2-user/.local/lib/python2.7/site-packages/botocore/client.py", line 676, in _make_api_call
raise error_class(parsed_response, operation_name)
botocore.exceptions.ClientError: An error occurred (UnrecognizedClientException) when calling the PutRecord operation: The security token included in the request is invalid.


Please help me with this.

Thanks,
Sandip Sinha
9433749132
 

Zachary Nielsen

Administrator
Alumni
Customer
For those executing the "Build a Log Analytics Solution on AWS, July 2020" here are some modifications that will help the lab be successful.

1) Do step #4 first. This takes AWS an extremely long time to provision and could run pass the lab timer. I tried (3) times.
- In Step 4 - Item #7d - pg 18
- Reference the number next to your login on the "top right" of the AWS console .... Corestack_Role/username @ 1234-5678-9012 for the JSON file ... find/replace 123456789012
- Use whatsmyip.org to get your ip address for the JSON file ... replace ##.##.##.##
- Make sure 'web-log-summary' matches the name you used for the domain
{
"Version": "2012-10-17",
"Statement": [
{
"Effect": "Allow",
"Principal": {
"AWS": "*"
},
"Action": [
"es:*"
],
"Condition": {
"IpAddress": {
"aws:SourceIp": [
"##.##.##.##/32"
]
}
},
"Resource": "arn:aws:es:us-east-1:123456789012:domain/web-log-summary/*"
}
]
}

2) Start an EC2 Instance - Item #9 - pg 11 -- pip does not get installed correctly. Use this script instead:
#!/bin/bash
sudo yum update -y
sudo yum install git -y
sudo curl -O https://bootstrap.pypa.io/pip/2.7/get-pip.py
sudo python get-pip.py
sudo pip install pytz
sudo pip install numpy
sudo pip install faker
sudo pip install tzlocal
git clone https://github.com/kiritbasu/Fake-Apache-Log-Generator.git
mkdir /tmp/logs
sudo cp ./Fake-Apache-Log-Generator/apache-fake-log-gen.py /tmp/logs/
 

Attachments

  • 1614360338020.png
    1614360338020.png
    16.8 KB · Views: 3
  • 1614360677140.png
    1614360677140.png
    15.3 KB · Views: 3
  • 1614360740211.png
    1614360740211.png
    15.3 KB · Views: 3
Last edited:
Top