Skip to main content

​
OpenAI Integration

​
Overview

The OpenAI integration in the Engineering Metrics Dashboard allows organizations to monitor OpenAI API usage, track model consumption, analyze user activity, and evaluate AI adoption metrics across teams. This integration is part of the LLM Metrics module.
Only one OpenAI admin key can be configured per organization. Multiple OpenAI connections are not supported.
A valid OpenAI admin key with the required permissions is necessary for successful integration.

​
Prerequisites

Before setting up the integration, ensure you have:
  • Access to the Engineering Metrics Dashboard
  • An OpenAI Admin API Key
  • Required permissions to configure integrations and projects

​
Step 1: Navigate to LLM Integrations

  1. Open the Engineering Metrics Dashboard.
  2. Navigate to Integrations.
  3. Select LLM Metrics Integrations.
  4. Choose OpenAI from the available providers.

​
Step 2: Create a New OpenAI Connection

  1. Click New Connection.
  2. Enter a unique Connection Name.
  3. Paste the OpenAI Admin Key into the provided field.
The admin key is generally managed by the organization administrator and is used to enable centralized AI usage tracking.

​
Step 3: Test the Connection

  1. Click Test Connection.
  2. Wait for validation to complete.

​
Expected Result

If the admin key is valid and configured correctly, the connection test succeeds. If the test fails, verify the API key and permissions before retrying.

​
Step 4: Save the Connection

  1. After a successful test, click Save Connection.
  2. The OpenAI connection will appear in the available connections list.
Only one OpenAI connection can exist within the same organization.

​
Step 5: Create an LLM Metrics Project

After creating the OpenAI connection:
  1. Navigate to Projects.
  2. Click Add Project.
  3. Select LLM Metrics as the project type.
  4. Enter the required project details.

​
Step 6: Select the OpenAI Data Source

Within the Data Source section:
  1. Select OpenAI.
  2. Choose the saved OpenAI admin key connection.
  3. Confirm the selected connection.
Once confirmed, the data scope is automatically added to the project.

​
Step 7: Configure Sync Behavior

The Sync Behavior section controls how frequently and for what duration data is collected.

​
Data Collection Range

Choose the duration for which historical data should be collected. Examples:
  • Last 10 Days
  • Last 30 Days
  • Last 6 Months
  • Last 1 Year

​
Sync Frequency

Select how often synchronization should run. Available options include:
  • Daily
  • Weekly
  • Monthly
  • Custom Schedule
Example:
  • Every 2 Hours

​
Schedule Status Options

StatusDescription
ActiveRuns synchronization automatically according to the configured schedule
InactiveDisables automatic synchronization
PauseTemporarily pauses synchronization for the configured connection

​
Scheduled Time

Users can configure a custom execution time for synchronization jobs.

​
Step 8: Complete Project Setup

  1. Review all configuration settings.
  2. Click Complete Setup.
The OpenAI integration project is now configured successfully.

​
Step 9: Run Initial Synchronization

  1. Open the configured project.
  2. Click Run Sync Now.
This action:
  • Collects the latest OpenAI usage data
  • Processes metrics and usage information
  • Updates the dashboard with the latest analytics

​
Understanding Sync Operations

​
Run Sync Now

The Run Sync Now option fetches the latest available usage data from OpenAI. Whenever new API activity occurs, synchronization updates the dashboard accordingly.

​
Retransform Data

Retransform Data does not fetch new information. Instead, it reprocesses previously collected data to:
  • Correct formatting inconsistencies
  • Standardize collected records
  • Recalculate processed metrics
In most cases, normal synchronization automatically performs the required transformation process.

​
Monitoring Integration Status

The Status section provides:
  • Synchronization history
  • Task execution details
  • Success and failure status
  • Processing duration
  • Completed operations overview
This helps users monitor synchronization health and activity.

​
Viewing OpenAI Metrics

After synchronization completes:
  1. Navigate to the LLM Metrics Dashboard.
  2. Search for the configured OpenAI project.
  3. Open the project to view analytics and usage metrics.
The dashboard provides visibility into:
  • User Activity
  • API Usage
  • Model Consumption
  • Organization-wide AI Adoption
  • Usage Trends and Patterns

​
Available Insights

​
User-Level Usage Data

The dashboard displays API usage information for users consuming OpenAI services within the organization.

​
API Key and Sub-Key Tracking

Projects may contain multiple API keys and sub-keys. The platform tracks usage across:
  • Projects
  • Sub-Keys
  • Individual API Keys

​
Model Usage Analytics

Organizations can monitor which OpenAI models are actively being used. Examples include:
  • GPT Models
  • Different Model Versions
  • Team-wise Usage Distribution

​
ROI and Business Impact Summary

The ROI and Business Impact section provides estimates related to:
  • Development Time Savings
  • Productivity Improvements
  • Operational Efficiency
  • Estimated Cost Optimization
These values are calculated using predefined internal metrics and AI usage patterns.

​
Executive Summary

The Executive Summary section provides a broader overview of AI adoption trends and organizational impact.
Certain Executive Summary features may still be under active development depending on the deployment version.

​
Troubleshooting

​
Connection Test Failed

​
Possible Causes

  • Invalid API Key
  • Missing Permissions
  • Expired Credentials

​
Resolution

  • Verify the admin key
  • Confirm permissions with the administrator
  • Retry the connection test

​
No Data Appearing After Sync

​
Possible Causes

  • Synchronization still in progress
  • Incorrect project configuration
  • Schedule disabled

​
Resolution

  • Check the Status section
  • Re-run synchronization manually
  • Verify sync schedule settings

​
Schedule Not Running

​
Possible Causes

  • Schedule set to Inactive
  • Schedule currently Paused

​
Resolution

  • Set the schedule status to Active
  • Verify the configured sync frequency and execution time

​
Best Practices

​
Use Clear and Unique Connection Names

Use descriptive connection names that clearly identify the environment.

​
Periodically Verify Admin Key Validity

Regularly confirm that the configured OpenAI Admin Key remains valid.

​
Configure Regular Synchronization Schedules

Maintain consistent synchronization intervals to ensure accurate reporting.

​
Monitor Synchronization Status Regularly

Review synchronization health and execution history to identify issues early.

​
Review Usage and ROI Metrics Periodically

Regularly evaluate adoption, consumption, and business impact metrics.

​
Summary

The OpenAI integration enables organizations to centralize and monitor AI usage directly within the Engineering Metrics Dashboard. By configuring the admin key connection, scheduling synchronization, and monitoring usage analytics, teams gain visibility into:
  • AI Adoption
  • Model Usage
  • User Activity
  • ROI Metrics
  • Business Impact