Skip to main content

​
Overview

The Google Cloud Platform (GCP) integration allows administrators to connect GCP projects to Engineering Metrics. Once configured, Engineering Metrics can securely access supported GCP services and collect cloud infrastructure, operational, and usage metrics for reporting and analysis. Using this integration, you can create connections, authenticate with a Service Account, enable individual GCP services as data scopes, and monitor cloud usage and infrastructure metrics.

​
Prerequisites

Before configuring the integration, ensure you have:
  • Access to the Engineering Metrics Dashboard
  • A Google Cloud Platform project
  • Permission to create Service Accounts in that project (the Editor or Owner role, or a custom role that includes iam.serviceAccounts.create and iam.serviceAccountKeys.create)
  • Permission to enable APIs and assign IAM roles at the project level, and at the organization level if you plan to enable the Carbon or Governance scopes
If you don’t have Owner, Editor, or IAM Admin access on the GCP project, ask your organization’s GCP administrator to perform these steps or grant you the required access.

​
Access GCP Connections

  1. Open the Engineering Metrics Dashboard.
  2. Navigate to Integration → Cloud Provider Metrics.
  3. Choose GCP Connections.

​
Prepare GCP credentials

Engineering Metrics authenticates to your GCP project using a Service Account and a downloaded JSON key file. You’ll need the project’s Project ID and the Service Account’s JSON key before creating a connection.
  1. Sign in to the Google Cloud Console.
  2. Click the project selector dropdown next to the Google Cloud logo and find your project — the Project ID column next to it is what you need (not the project name or number).
Alternatively, with the project selected, check the Project Info card on the Cloud Console dashboard. Copy the Project ID exactly as shown — it’s a lowercase string, for example my-app-prod-123456.
If your organization already has a dedicated Service Account for monitoring tools, reuse it and skip to step 4.
1

Create the Service Account

In the Cloud Console, go to IAM & Admin → Service Accounts and click + Create Service Account. Enter a name, for example engineering-metrics-monitoring, then Create and Continue.
2

Grant IAM roles

Under Grant this service account access to project, assign the roles listed in Required IAM roles below (you can also add these later from the Service Accounts page). Click Continue, then Done.
3

Generate the JSON key

Open the Service Account, select the Keys tab, click Add Key → Create new key, choose JSON, then Create. The key file downloads automatically.
4

Copy the credentials

Open the downloaded file in a text editor and copy its entire contents, including the opening and closing { }, to paste into the connection’s Service Account Credentials (JSON) field.
Google lets you download the private key only once, at creation time. If it’s lost, create a new key from the Keys tab and delete the old, unused one.

​
Create a connection

  1. Click New Connection. The Manage Connections: GCP dialog appears.
Use the Active toggle to control whether the connection is included in scheduled and manual syncs.

​
Required permissions and APIs

The Service Account needs specific APIs and IAM roles enabled depending on which data scopes you plan to use.
In the Cloud Console, go to APIs & Services → Library, search for the API by name (for example “Cloud Asset Inventory API”), then click Enable. Repeat for each API required by the scopes you plan to use.
Go to IAM & Admin → IAM, click + Grant Access, enter the Service Account’s email under New principals (format: name@project-id.iam.gserviceaccount.com), select the role, then Save. Repeat for each required role.
Click the project selector, switch to the Organization tab, then go to IAM & Admin → IAM and grant access the same way as a project-level role.
If you don’t see an Organization option in the project selector, your account lacks access to the organization resource — ask your GCP organization administrator to assign these roles instead.

​
Test and save

  1. Click Test Connection and wait for validation to complete. If it fails, verify the Project ID and Service Account credentials.
  2. Once the test succeeds, click Save Connection. The connection appears in the GCP Connections list.

​
Manage connections

Click the Edit icon under Actions, update the fields, then Test Connection and Save Connection. Changes take effect immediately.
Submitting new credentials, or changing the Project ID, permanently deletes all previously synced data for this connection before re-syncing under the new settings.
Click the Delete icon under Actions and confirm. This removes the integration for that project.If the connection is linked to a Project, you must unlink it first — deletion is blocked otherwise.
Deleting a connection immediately erases all previously collected data for that connection, not just future collection.
Active connections collect metrics on an ongoing basis. Inactive connections stay configured but collect nothing until re-enabled.

​
Configure data scopes

A data scope controls which GCP service Engineering Metrics collects data from. After creating a connection, select it and click Add Data Scope, then configure one of the scopes below.
Collects cloud spending, service usage costs, and project- or resource-level cost allocation.
Billing export is configured at the Cloud Billing account level and requires the Billing Account Administrator role.
  1. In BigQuery, select the destination project, then create a dataset (for example billing_export_dataset) in a multi-region location such as US or EU.
  2. Go to Billing → Cost management → Billing export, open the BigQuery export tab.
  3. Next to Standard usage cost (and Detailed usage cost for resource-level detail), click Edit settings and select the project and dataset you created.
The Billing Dataset is the dataset ID. Expand it in BigQuery to find the generated table, named gcp_billing_export_v1_<billing_account_id> — combine dataset and table name for the Billing Table value.
New export data can take up to 24 hours to appear; historical backfill takes up to 5 days and only covers the current and previous month.

​
Troubleshooting

  • Assign the BigQuery Job User and BigQuery Data Viewer roles
  • Verify Billing Export is enabled
  • Confirm the Dataset and Table values
  • Verify the required APIs are enabled
  • Confirm the Target Project IDs are correct
  • Confirm the Service Account has the scope’s required IAM roles
  • Re-copy the Project ID from the Cloud Console project selector (not the project name or number)
  • Re-open the downloaded JSON file and copy its full contents again
  • Generate a new Service Account key if the original was deleted or lost
  • Confirm the required APIs are enabled in APIs & Services → Library

​
Best practices

  • Use a dedicated Service Account for the Engineering Metrics integration
  • Assign only the required IAM permissions
  • Enable only the scopes you need
  • Verify Billing Export tables before enabling the Billing or Carbon scopes
  • Avoid sharing Service Account JSON credentials
  • Rotate Service Account keys periodically