S3 Configuration

# Gong Data Cloud: S3 destination

How the S3 destination works

Once you setup your S3 destination in Gong Data Cloud, all of your available Gong data will be pushed to your destination & will be kept in sync via regular updates (daily).

Data transferred to S3 will be loaded as Apache Parquet files in Apache Hive style partitions. This means data will appear in the following folder structure:

Path template

<bucket_name>/<folder_name>/<model_name>/dt=<transfer_date>/<file_part>_<transfer_timestamp>.parquet

Path example

gong_data_cloud_bucket/gong_data_cloud_folder/conversations/dt=2025-01-01/0.2025010

Where:

<bucket_name> and <folder_name> are provided during destination configuration.
<model_name> is the name of the data model being transferred (this is equivalent to a table name in relational data destinations).
<transfer_date> and <transfer_timestamp> are generated at transfer time and based on the transfer's start time. <transfer_date> is of the form 2025-01-01, while <transfer_timestamp> is of the form 20250101000000.
<file_part> is a monotonically increasing integer for a given timestamp, and does not carry any special meaning.

📘 What are Apache Hive style partitions and Apache Parquet file format?
Apache Hive style partitions are compatible with most popular query engines, and should make data easily queryable and transportable. Apache Parquet file format is an open source, column-oriented data file format that offers efficient data compression and data integrity.

Prerequisites

By default, S3 authentication uses role-based access. You will need the trust policy prepopulated with the Gong AWS ARN and your ExternalId to grant access.

* The Custom trust policy is:

{
	"Version": "2012-10-17",
	"Statement": [
		{
			"Effect": "Allow",
			"Principal": {
				"AWS": "arn:aws:iam::655275192472:role/gpe-us-01-1-c1-app-prequel"
			},
			"Action": "sts:AssumeRole",
			"Condition": {
"StringEquals": {
     "sts:ExternalId": "<gong_external_id>" 
}
}
		}
	]
}

Where:

gong_external_id is a unique ID assigned to every Gong account. You can find your unique ID in the S3 connection form in the Gong app.

Step 1: Set up destination S3 bucket

Create bucket

Navigate to the S3 service page.
Click Create bucket.
Enter a Bucket name and modify any of the default settings as desired. Note: Object Ownership can be set to "ACLs disabled" and Block Public Access settings for this bucket can be set to "Block all public access" as recommended by AWS. Make note of the Bucket name and AWS Region.
Click Create bucket.

Step 2: Create policy and IAM role

Create policy

Navigate to the IAM service page.
Navigate to the Policies navigation tab, and click Create policy.
Click the JSON tab, and paste the following policy, being sure to replace BUCKET_NAME with the name of the bucket chosen in Step 1.

{
    "Version": "2012-10-17",
    "Statement": [
        {
            "Effect": "Allow",
            "Action": ["s3:PutObject", "s3:DeleteObject"],
            "Resource": "arn:aws:s3:::BUCKET_NAME/*"
        }
    ]
}

📘 Understanding the s3:DeleteObject requirement
By default, a connection test is performed against the destination during initial configuration and `s3:DeleteObject` is required to clean up test artifacts. Once the test has been performed successfully and the destination added, this action can be safely removed, as S3 destinations are append-only by default.

Click Next: Tags, click Next: Review.
Name the policy, add a description, and click Create policy.

Create role

Navigate to the IAM service page.
Navigate to the Roles navigation tab, and click Create role.
Select Custom trust policy and paste the provided trust policy to allow AssumeRole access to the new role. Click Next.
Add the permissions policy created above, and click Next.
Enter a Role name, for example, transfer-role, and click Create role.
Once successfully created, search for the created role in the Roles list, click the role name, and make a note of the ARN value.

📘 Alternative authentication method: AWS User with HMAC Access Key ID & Secret Access Key

📘 Alternative authentication method: AWS User with HMAC Access Key ID & Secret Access Key
Role based authentication is the preferred authentication mode for S3 based on AWS recommendations, however, HMAC Access Key ID & Secret Access Key is an alternative authentication method that can be used if preferred. Navigate to the IAM service page. Navigate to the Users navigation tab, and click Add users. Enter a User name for the service, for example, `transfer-service`, click Next. Under Select AWS access type, select the Access key - Programmatic access option. Click Next: Permissions. Click the Attach existing policies directly option, and search for the name of the policy created in the previous step. Select the policy, and click Next: Tags. Click Next: Review and click Create user. In the Success screen, record the Access key ID and the Secret access key.

Role based authentication is the preferred authentication mode for S3 based on AWS recommendations, however, HMAC Access Key ID & Secret Access Key is an alternative authentication method that can be used if preferred. Navigate to the IAM service page. Navigate to the Users navigation tab, and click Add users. Enter a User name for the service, for example, transfer-service, click Next. Under Select AWS access type, select the Access key - Programmatic access option. Click Next: Permissions. Click the Attach existing policies directly option, and search for the name of the policy created in the previous step. Select the policy, and click Next: Tags. Click Next: Review and click Create user. In the Success screen, record the Access key ID and the Secret access key.

Step 3: Add your destination

Securely share your bucket name, bucket region, and role ARN with us to complete the connection.