Monitoring and Configuring AWS Agent

The section introduces you to the Foglight Hybrid Cloud Manager for Amazon Web Services (AWS) environment and provides you with essential information.

This section covers the following key areas:

Minimum application privileges

Each AWS Agent monitors the assets inside the selected region. To monitor an AWS environment, AWS Identity and Access (IAM) users need to use an Access Keys to secure REST or HTTP query protocol requests. Create an IAM user with the following privileges to use the Foglight Hybrid Cloud Manager for AWS:

  • AmazonSSMFullAccess
  • AmazonEC2ReadOnlyAccess
  • CloudWatchFullAccess
  • IAMReadOnlyAccess
  • AWSHealthFullAccess

To collect EC2 Memory metrics and Linux Volume metrics, make sure to assign the following privilege when creating the EC2 instance that will be launched and monitored:

  • AmazonEC2RoleforSSM To use Optimizer Reclaim action, it is recommended to create a custom policy to assign the following privileges to the user:
  • Write access level including the following actions for EC2 service is required:
    • ModifyInstanceAttribute
    • StartInstances
    • StopInstances
  • Write access level including the following actions for EC2 Auto Scaling service is required:
    • ResumeProcesses
    • SuspendProcesses

API used to collect Cost metrics

Foglight Hybrid Cloud Manager for AWS uses the AWS Cost and Usage Report to tracks your AWS usage and provides the estimated charges associated with your AWS account. AWS delivers the AWS Cost & Usage Report (in CSV format) for the Amazon Simple Storage Service (S3) bucket you specified, and updates the reports at least once a day. AWS Agent retrieves the reports programmatically using the Amazon S3 APIs. If you use the consolidated billing feature in AWS Organizations, this report is available only to the master account and includes activity for all the member accounts that are associated with the master account. For more information refer to the AWS Cost and Usage Report.

To get Account ID (12-digit number):

  1. Log in to the AWS Management Console: https://console.aws.amazon.com.

  2. Locate your Account ID.

    a. Click Support on the navigation bar on the upper-right.
    b. Select Support Center. Your currently signed-in account number (ID) appears in the Support Center title bar.

To create an AWS Cost and Usage Report:

  1. Sign in to the AWS Management Console and open the Billing and Cost Management console at https://console.aws.amazon.com/billing/.

  2. In the navigation pane, click Reports.

  3. Click Create report.

  4. Enter the following required information, and then click Next.
    a. Report name: enter the name of report.
    b. Additional report details: select the Include resource IDs checkbox.

  5. S3 bucket: Enter the name of the Amazon S3 bucket where you want the reports to be delivered and then select Verify. The bucket must have appropriate permissions.
    a. Click Sample Policy link and copy and paste the text in this sample policy into the permissions associated with your Amazon S3 bucket.
    b. Open a new Page to access your S3 bucket, click Permissions and then Bucket policy. Paste the text in this sample policy into the permissions associated with your Amazon S3 bucket.
    c. Below is an example for the S3 bucket policy. Update the following descriptions in bold according to your AWS Account and S3 bucket.

    • AWS monitoring user ARN: json "arn:aws:iam::88888888:user/exampleAWSUserTest":
      Format: “arn:aws:iam::your AWS Account ID:user/your monitoring AWS username”

      To get the AWS user ARN from AWS Console, select IAM, and then click the AWS user which is configured under the Foglight AWS Agent.

    • S3 bucket ARN: json "arn:aws:s3:::exampleBucketNameTest":
      Format: arn:aws:s3:::your bucket name Change the exampleBucketNameTest to your S3 bucket name.

      {
  "Version": "2008-10-17",
  "Id": "PolicyForFoglightAWSCostReport",
  "Statement": [
     {
         "Sid": "StmtForAWSBillingReportGet",
         "Effect": "Allow",
         "Principal": {
             "Service": "billingreports.amazonaws.com"
             },
             "Action": [
                 "s3:GetBucketAcl",
                 "s3:GetBucketPolicy"
                 ],
                 "Resource":"arn:aws:s3:::exampleBucketNameTest"
                 },
                 {
                     "Sid": "StmtForAWSBillingReportPut",
                     "Effect": "Allow",
                     "Principal": {
                         "Service": "billingreports.amazonaws.com"
                         },
                         "Action": [
                             "s3:PutObject"
                             ],
                             "Resource":"arn:aws:s3:::exampleBucketNameTest/*"
                             },
                             {
                                 "Sid": "StmtForAWSUserGet",
                                 "Effect": "Allow",
                                 "Principal": {
                                     "AWS":"arn:aws:iam::88888888:user/exampleAWSUserTest"
                                     },
                                     "Action": "s3:GetObject",
                                     "Resource": "arn:aws:s3:::exampleBucketNameTest/*"
                                     }

                                     ]   
                                 }

  6. Report path prefix - (Optional): Enter the report path prefix that you want to name of your report.

  7. Time granularity: Select Daily.

  8. Report versioning: Select Overwrite existing report.

  9. Enable report data integration for: Leave blank.

  10. Compression type: Select GZIP or ZIP.

  11. Click Next, after you have reviewed the settings for your report, and then click Review and Complete.

AWS monitoring setup

A complete setup includes the following steps:

  1. Get the authentication information through AWS Management Console. For more information, refer to Getting authentication information through console.
  2. Create an AWS Agent on the Foglight Management Server. For more information, refer to Creating an AWS Agent.
  3. (Optional)- Configure the interval of data collection. For more information, refer to Configuring data collection interval.

Getting authentication information through console

To create and retrieve Access Keys of a user through the AWS IAM console:

  1. Log in to the AWS IAM console at: https://console.aws.amazon.com/iam/.
  2. Click IAM under the Security, Identity & Compliance column.
  3. On the left navigation panel, click Users. The Resource Groups view opens on the right.
  4. In the Resource Groups view, click the user which Access Key is to be retrieved. The User Summary view opens.
  5. In the User Summary view, click Security credentials, then the Sign-in credentials view opens.
  6. In the Access keys area, click Create access key. The Create access key dialog box appears and shows the access key and Secret access key.
  7. Click Download .csv file to keep the access key and secret access key somewhere safe.
  8. (Optional) - If you see the Limit exceed message, click the close button next to the Status column to delete an access key that is not being used. Then repeat Step 6 to create and retrieve a new access key.

Configuring firewall settings

If your AWS Performance Agent is installed behind the firewall, ensure the following URL addresses and ports are open:

  • URL address:
    • *.amazon.com
  • TCP/UDP port:
    • 80 and 443

Creating an AWS Agent

To create an AWS agent:

  1. Log in to the Foglight browser interface.
  2. In the navigation panel, click Cloud Manager. The Cloud Manager dashboard opens.
  3. In the Cloud Manager dashboard, click AWS and then click on the Administration sub tab.
  4. Click Create AWS Agent. The Agent Setup Wizard dialog box opens.
  5. In the Select Agent Host view, select the agent manager on which the new agent is to be deployed, and then click Next.
  6. In the AWS Credential view, specify the following values, as needed, and then click Finish.
    • Account Alias: The display name of this account.
    • Access Key ID: The access key retrieved in Getting authentication information through console.
    • Secret Access Key: The secret access key retrieved in Getting authentication information through console.
    • Collect Memory Metric: Select this option to enable the collection of instance memory metrics. The default value is disabled. Foglight supports to collect memory metrics for both Windows and Linux OS. To enable Linux OS Memory Metrics collection, upload the private key and assign the credential to the required instance. No additional tasks are required when enabling Windows OS memory metrics collection.
    • Collect Linux Volume Utilization: Select this option to enable the collection of Linux volume utilization. The default value is disabled. Foglight only supports to collect volume utilization for Linux OS. To enable Linux OS Memory Metrics collection, upload the private key and assign the credential to the required instance.
    • Specify an agent name (Optional): Specify the name of agent.
    • Configure regions to be monitored (Optional): Select AWS regions for monitoring. All regions will be monitored if this field is not configured.
    • Configure Account Cost to Monitor: Configure the Cost Metrics collection. Collections will start only after the AWS Cost and Usage Report are created on the AWS Console.
    • Configure Proxy (Optional): Configure the proxy setting when the Agent Host requires a proxy connection to the Internet.

The new AWS Agent is created, and its data is to be displayed on the Monitoring tab after a few minutes.

Configuring data collection interval

Foglight Hybrid Cloud Manager enables you to configure the interval for data collection using the Agent Status dashboard. To configure the data collection interval:

  1. On the navigation panel, click Administration > Agents > Agent Status.
  2. On the Agent Status dashboard, select the AWS agent that you want to monitor, and then click Edit Properties. The Edit Properties view opens.
  3. Select True for Collect Memory Metric and Collect Volume Metric, and then specify a value for Collection Interval Offset. Quest highly recommends setting the Collector Config (also knowns as Collection interval) to a value greater than 10 minutes. If the Collection interval is less than 10 minutes, AWS agent cannot collect metrics from AWS Cloud Watch as AWS Cloud Watch has a 10-minute delay. If you insist on setting this interval less than 10 minutes, ensure the following:
  • Collection Interval Offset must be set to a non-negative integer.
  • The configuration should follow comply with the formula: (n+1) x I >= 10 minutes.
    • n represents the value of Collection Interval Offset.
    • I represents the value of Collector Config (in minutes).