Databricks Integration (Using Microsoft Entra ID)

This section describes the expected user experience and workflow when integrating Posit Connect with an instance of Azure Databricks. When using Databricks with Connect, Databricks must be configured to authenticate with Azure. Microsoft Entra ID is responsible for issuing OAuth Tokens which Databricks accepts.

Overview

Connect supports integrating with Azure Databricks using OAuth tokens. These tokens are issued by a Microsoft Entra ID OAuth Application to content running in Connect. Issued tokens are accepted by Databricks to authorize protected resource access.

To fully configure a Connect / Databricks OAuth flow, the following personas are required:

  • An Azure Administrator who has the ability to create and configure Microsoft Entra ID OAuth Applications.
  • A Posit Connect Administrator who has the ability to configure OAuth Integrations in Posit Connect.
  • A Posit Connect Publisher who authors content which interacts with protected Databricks resources. For more information on authoring content from Connect using Databricks resources, please see the Databricks section of the User Guide.
  • A Posit Connect Viewer who has permission to view the Publisher’s content. The Viewer must grant permission for the content to authenticate to Databricks on their behalf. For more information on viewing content from Connect that uses Databricks resources, please see the Databricks section of the User Guide.

Workflow

The following sections describe the steps to integrate Posit Connect content with Azure Databricks.

When the content is authenticating to the third-party service as a viewer, then the access token is requested using the Authorization Code Grant Type. This allows Posit Connect to request a temporary user access token from the OAuth Application. The end user (the content viewer) is redirected to a consent screen after visiting the content for the first time. The user must explicitly grant Connect permission to access the protected resources on their behalf.

Once consent is given, the user is redirected to the content running in Connect. The content then impersonates the user (the content viewer) when making requests to the third-party service.

1. Register a new OAuth App in Microsoft EntraID

An Azure administrator registers a new OAuth Application in Microsoft Entra ID.

The Azure administrator adds a redirect_uri for the OAuth Application as seen in the screenshot below. This redirect is where Azure sends the user credentials at the end of the OAuth handshake. This allows Posit Connect to obtain a temporary access token and refresh token from Azure, which is then used to access protected Databricks resources on behalf of the user.

The redirect_uri should be an FQDN of the Posit Connect server:

  • https://${Server.Address}/__oauth__/integrations/callback

The Register OAuth App page of the Azure Console.

In the API permissions section, add a permission for AzureDatabricks/user_impersonation as seen below.

The API Permissions section of the Azure Console with an added permission for `AzureDatabricks/user_impersonation`.

In the Certificates & secrets section, add a new client secret for the OAuth Application.

The Azure administrator makes note of the tenant_id, client_id, and client_secret from the registered OAuth Application and shares this information to the Posit Connect administrator.

2. Create a new OAuth Integration in Posit Connect

A Posit Connect administrator configures a new OAuth Integration in Posit Connect. The example below shows how to create a new Databricks OAuth Integration using curl. The following fields must be provided to enable the integration:

The OAuth scope 2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default is a reference to the AzureDatabricks/user_impersonation permission that allows Posit Connect content to impersonate the content viewer. Do not change this value.

curl -H "Authorization: Key ${CONNECT_API_KEY}" -XPOST https://connect.example.org/__api__/v1/oauth/integrations \
--data '{
  "client_id": "<my-databricks-oauth-app-client-id>",
  "client_secret": "<my-client-secret>",
  "tenant_id": "<my-azure-tenant-id>",
  "description": "A helpful description for Publishers to use when choosing an OAuth integration for their content.",
  "default_scopes": ["2ff814a6-3304-4ab8-85cb-cd0e6f879c1d/.default", "offline_access"]
}'

3. Deploy content to Posit Connect

See the Databricks section of the Posit Connect User Guide for an overview of how content publishers interact with this feature.

Glossary

  • Identity Provider (Azure; Microsoft Entra ID) - The identity provider maintains the identities of users and is responsible for authenticating users.

  • Third-Party Service (Databricks) - The third-party service that has protected resources.

  • OAuth Application - OAuth Applications are provisioned by a Microsoft Entra ID Administrator. An OAuth Application issues tokens that are used to authenticate to Databricks.

  • OAuth Client - OAuth Clients request tokens from OAuth Applications.

  • OAuth Integration - In Posit Connect, an OAuth Integration defines the capabilities of an OAuth Client. This includes all of the information required to initialize an instance of an OAuth Client.