AWS S3 External Storage Bucket Setup

Overview

Amazon Simple Storage Service (AWS S3) is a scalable cloud storage solution that enables organizations to securely store and manage data at any scale. CHEQ supports AWS S3 as an external destination for both one-time and scheduled reports.

Create your S3 Bucket & Access Key Pair

1. Create an S3 Bucket
   1. Select your desired storage region and proceed with the default bucket settings.
2. Create the Access/Secret Key
   1. Remember to copy or download the secret key before closing the window.
   2. Ensure the User account has permissions to add data to your bucket.

To grant a User access to the S3 bucket, you can also set that user's policy similar to the one below (replace <bucket-name> with your bucket name):

{
  "Version": "2012-10-17",
  "Statement": [
    {
      "Sid": "VisualEditor0",
      "Effect": "Allow",
      "Action": [
        "s3:ListBucket"
      ],
      "Resource": "arn:aws:s3:::<bucket-name>"
    },
    {
      "Sid": "VisualEditor1",
      "Effect": "Allow",
      "Action": [
        "s3:GetObject",
        "s3:PutObject"
      ],
      "Resource": "arn:aws:s3:::<bucket-name>/*"
    }
  ]
}

Send CHEQ Data to AWS S3 Bucket

In the platform, select the relevant product and navigate to Data Exports from the left menu. From there, select the Reports or Report Scheduler tab at the top of the page and click Create Report or Create Report Scheduler, depending on whether you need a one-time report or a recurring scheduled report.

The interface will look something like this, where External Storage has been selected:

Populate the fields with the information you just generated from your Amazon S3 bucket and click Test Connection to verify that the connection succeeded. You can also check Set as default connection to automatically apply this setup to all future reports. 

Access CHEQ Schedule Reports using Amazon S3

Log into your Amazon S3 account and go to Buckets ><bucket-name>. Once the report has been sent to S3, look for a folder named after your Scheduled Name - like 'Daily Analytics Export' from the example above.

The bucket folder path follows the pattern: <bucket-name>/<report_name>/<report_start_date>/

The filename follows the pattern: <report_start_date>_<file_num>.<format>

Daily generated reports will be split into multiple files in case there is a large scale of data, the max file size is 1M rows.