CloudWatch logs stream from AWS EC2 instance

In today IT world, if we want to find any abnormal activities within the network then you can easily find from the logs which have generated from the infrastructure. SO the logs are most important part of IT infrastructure.

In AWS , we can stream all logs from compute services by installing Cloudwatch agents.So in this blog you will learn how to install Cloudwatch Agents into instances and stream to the Cloudwatch dashboard.

The CloudWatch Logs agent installer on an existing EC2 instance to install and configure the CloudWatch Logs agent. After installation is complete, the agent confirms that it has started and it stays running until you disable it.

To configure your IAM role or user for CloudWatch Logs:

  • Open the IAM console at https://console.aws.amazon.com/iam/.
  • In the navigation pane, choose Roles.
  • Choose the role by selecting the role name (do not select the check box next to the name).
  • On the Permissions tab, expand Inline Policies and choose the link to create an inline policy.
  • On the Set Permissions page, choose Custom Policy, Select.
  • For more information about creating custom policies, see IAM Policies for Amazon EC2 .
  • On the Review Policy page, for Policy Name, type a name for the policy.
  • For Policy Document, paste in the following policy:

Install and Configure CloudWatch Logs on an Existing Amazon EC2 Instance

To install and configure CloudWatch Logs on an existing Amazon Linux instance

  1. Connect to your Amazon Linux instance. For more information, see Connect to Your Instance in the Amazon EC2 User Guide for Linux Instances.
  2. Update your Amazon Linux instance to pick up the latest changes in the package repositories.

                      sudo yum update –

Install the awslogs package.

                 sudo yum install -y awslogs

  1. Edit the /etc/awslogs/awscli.conf file

To install and configure CloudWatch Logs on ubuntu/centos instance

To run it directly from the Internet, use the following commands and follow the prompts:

curl https://s3.amazonaws.com//aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O

sudo python ./awslogs-agent-setup.py –region us-east-1

To download and run it standalone, use the following commands and follow the prompts:

curl https://s3.amazonaws.com//aws-cloudwatch/downloads/latest/awslogs-agent-setup.py -O

Curl https://s3.amazonaws.com//awscloudwatch/downloads/latest/AgentDependencies.tar.gz -O

tar xvf AgentDependencies.tar.gz -C /tmp/

Edit /etc/awslogs/awscli.conf file as per requirement

[/var/log/syslog]
datetime_format = %d/%b/%Y:%H:%M:%S
file = /var/log/syslog
buffer_duration = 5000
log_stream_name = system-logs
initial_position = end_of_file
log_group_name = web-server

You will find the log-group in cloudwatch dashboard

Once you click on log-group , you will find the streams for the logs which you have stream from instance.

Log-stream would be like below.

You have just streamed the logs from your server to cloudwatch.

 

Leave a Reply

Your email address will not be published. Required fields are marked *

*