Vantage integrates with your Databricks account through the use of Billable Usage Logs. Vantage provides an S3 bucket for Databricks to deliver usage logs to on a periodic basis. Databricks delivers cost-only data to the bucket, outlining the Databricks service used, usage in Databricks Units (DBU), and metadata related to workspace, cluster, and any related tags.
Connect Your Databricks Account
Create a free Vantage account, then follow the steps below to integrate Databricks costs.
Create the Connection
- From the Vantage console, navigate to the Databricks Settings page.
- At the top, select the Connect tab, then click Setup Account.
- Enter your Databricks Account ID.tip
You can find your Databricks Account ID in the user profile dropdown of the Databricks account console.
- Click Connect Account.
- On the Databricks Settings page, you will see your account listed with a Status of
Pendingto complete the connection. Additional steps will be displayed on the Finalize Your Databricks Account Setup page. Keep this page open.
Finalize Your Databricks Account Setup
Vantage will prepare an S3 bucket to store your Databricks billable usage logs. You will need to configure Databricks to deliver your usage logs to that bucket.
Install or update the Databricks CLI.
Configure authentication for the Databricks CLI:info
The below steps are based on the OAuth user-to-machine (U2M) authentication Databricks documentation.
<ACCOUNT_ID>with the value of your Databricks account. (See the Databricks documentation for how to locate your account ID.) Replace the
<ADMIN_PASSWORD>with your account administrator user's credentials.
host = accounts.cloud.databricks.com
account_id = <ACCOUNT_ID>
username = <ADMIN_USERNAME>
password = <ADMIN_PASSWORD>
Initiate OAuth token management with the following command, replacing
ACCOUNT_IDwith the account ID you used in the last step:
databricks auth login --host https://accounts.cloud.databricks.com/ --account-id <ACCOUNT_ID>
Follow the remaining instructions provided on the Finalize Your Databricks Account Setup page in Vantage. These instructions will walk you step by step through creating a storage configuration, creating a credential configuration, and creating a log delivery configuration to finalize the integration.
After completing all the steps above, your Databricks integration status should automatically update from
Importing. Once fully imported, the status will update to
Imported. You can view your connection status on the Databricks Settings page.
Databricks generally delivers usage logs once per day.
Databricks Reporting Dimensions
Databricks Cost Reports enable you to filter Databricks costs across several dimensions:
- Not tagged
The Tag filter contains values like
JobID, which can be used to view costs for specific Databricks jobs.