AWS RDS Support

Starting with version 1.9.0, pgmetrics can fetch metrics exposed by AWS RDS instances via AWS CloudWatch. Both PostgreSQL and Aurora engines of AWS RDS are supported.

Currently, pgmetrics can collect:

  • RDS Metrics: These are a basic set of hypervisor-, OS- and Postgres-level metrics exposed by default for all RDS instances. More information is available here and the list of metrics themselves is available here.
  • Enhanced Monitoring: If you have enabled Enhanced Monitoring for your RDS instances, more metrics are available. These metrics are reported by an agent running inside the instances that host your RDS Postgres/Aurora databases, and include information like memory usage, disk usage and per-process resource usage. More information is available here and the list of metrics can be found at the end of this AWS documentation page.

pgmetrics collects these metrics using the AWS CloudWatch API for RDS Metrics and the AWS CloudWatch Logs API for Enhanced Monitoring metrics.

Invoking

To collect the information, pgmetrics has to know the Database Instance Identifier of your database. This identifier, chosen by you, uniquely identifies the database instance within a region for your AWS account. This identifier has to be passed to pgmetrics using the command-line option --aws-rds-dbid.

The AWS region and credential information must be passed via environment variables or other standard mechanisms, see the next section.

Here is an example invocation, directly using hard-coded credentials for the database:

PGPASSWORD=mysecretpass ./pgmetrics -h database-1.abcdefghijkl.us-east-1.rds.amazonaws.com \
    -U postgres --aws-rds-dbid=database-1 -w -fjson bench

In this case, the database identifier is “database-1”, the user is “postgres”, the database name is “bench”, the hostname (set by AWS) is “database-1.abcdefghijkl.us-east-1.rds.amazonaws.com” and the password is “mysecretpass”. This produces a JSON output similar to the following:

{
  "meta": {
    "version": "1.8",
    "at": 1586671270,
    "collected_dbs": [
      "bench"
    ],
    "local": false,
    "user_agent": "pgmetrics/v1.9.0"
  },
  "start_time": 1584703107,
  "system_identifier": "6776375427100147225",
  "checkpoint_lsn": "66/74000028",
  "prior_lsn": "",
  "redo_lsn": "66/74000028",
  "timeline_id": 1,
  "next_xid": 7242,
  "oldest_xid": 562,
  "oldest_active_xid": 0,
  "checkpoint_time": 1586671105,
  "wal_flush_lsn": "66/74000650",

  [... elided ...]

  "rds": {
    "basic": {
      "CPUUtilization": 3.83333333333818,
      "DatabaseConnections": 0,
      "DiskQueueDepth": 0.00006666555557407377,
      "FreeStorageSpace": 18220265472,
      "FreeableMemory": 477421568,
      "MaximumUsedTransactionIDs": 6681,
      "NetworkReceiveThroughput": 592.1401309978168,
      "NetworkTransmitThroughput": 3428.526191230146,
      "OldestReplicationSlotLag": -1,
      "ReadIOPS": 0,
      "ReadLatency": 0,
      "ReadThroughput": 0,
      "ReplicationSlotDiskUsage": 4096,
      "SwapUsage": 0,
      "TransactionLogsDiskUsage": 2281709568,
      "TransactionLogsGeneration": 0,
      "WriteIOPS": 1.2666455559074015,
      "WriteLatency": 0.000014652014652014653,
      "WriteThroughput": 18636.1787940402
    }
  },

  [... elided ...]

pgmetrics will detect if Enhanced Monitoring is enabled. If it is, it will also collect additional information, which will look like this:

  "rds": {
    "basic": {

  [... elided ...]

    },
    "enhanced": {
      "cpuUtilization": {
        "guest": 0,
        "idle": 95.8,
        "irq": 0,
        "nice": 0.2,
        "steal": 0.1,
        "system": 1.5,
        "total": 4.3,
        "user": 2.2,
        "wait": 0.3
      },
      "diskIO": [
        {
          "avgQueueLen": 0.33,
          "avgReqSz": 172.05,
          "await": 25.46,
          "device": "rdsdev",
          "readIOsPS": 0,
          "readKb": 0,
          "readKbPS": 0,
          "rrqmPS": 0,
          "tps": 12.9,
          "util": 0.23,
          "writeIOsPS": 12.9,
          "writeKb": 66584,
          "writeKbPS": 1109.55,
          "wrqmPS": 0
        },
        {
          "avgQueueLen": 0,
          "avgReqSz": 18.39,
          "await": 0.23,
          "device": "filesystem",
          "readIOsPS": 1,
          "readKb": 464,
          "readKbPS": 7.73,
          "rrqmPS": 0,
          "tps": 4.9,
          "util": 0.11,
          "writeIOsPS": 3.9,
          "writeKb": 2240,
          "writeKbPS": 37.33,
          "wrqmPS": 2.67
        }
      ],
      "engine": "POSTGRES",

  [... elided ...]

Supplying Region and Credentials

pgmetrics also requires the AWS region name as well as the credentials to invoke the AWS APIs to fetch the metrics.These can be supplied in various quasi-standard ways. As a best practice, AWS recommends that you specify credentials in the following order:

  • Use IAM roles for Amazon EC2 (if your application is running on an Amazon EC2 instance)
  • Use a shared credentials file (~/.aws/credentials)
  • Use environment variables (AWS_REGION, AWS_ACCESS_KEY_ID, AWS_SECRET_ACCESS_KEY and others)

You can find more detailed information here.

A Note About Billing

Usage of the AWS CloudWatch and AWS CloudWatch Logs APIs will incur costs, based on their pricing plans. This means that invoking pgmetrics to collect these metrics will incur charges on your AWS account.