Databricks

Posit Connect can integrate with an instance of Azure Databricks, allowing you to author content that accesses Databricks resources. For more information on integrating Connect with Databricks, see the Databricks section of the Admin Guide.

Author

Authored content that requires access to Databricks resources should use the Posit SDK or call the /v1/oauth/integrations/credentials endpoint directly to obtain Databricks credentials. It is up to the content to ensure that the provided credentials are used correctly when making subsequent requests to the protected resources. When viewing content with Databricks resources, there may be additional complexity to ensure that the correct token is injected. This complexity depends on the stateful-ness of the deployed content.

The following is a simple example of a streamlit app which connects to Databricks as the content viewer using the databricks-python-sdk. For more examples using different frameworks see the Connect examples for the Posit SDK. Many of these examples use the databricks-sql-python library. For an example that uses the databricks-connect library, see this example:

import os

from posit.connect.external.databricks import viewer_credentials_provider

from databricks import sql
from databricks.sdk.service.iam import CurrentUserAPI
from databricks.sdk.core import ApiClient, Config

import pandas as pd
import streamlit as st
from streamlit.web.server.websocket_headers import _get_websocket_headers

DATABRICKS_HOST = os.getenv("DATABRICKS_HOST")
DATABRICKS_HOST_URL = f"https://{DATABRICKS_HOST}"
SQL_HTTP_PATH = os.getenv("DATABRICKS_PATH")

session_token = _get_websocket_headers().get("Posit-Connect-User-Session-Token")

credentials_provider = viewer_credentials_provider(user_session_token=session_token)

cfg = Config(host=DATABRICKS_HOST_URL, credentials_provider=credentials_provider)
databricks_user = CurrentUserAPI(ApiClient(cfg)).me()
st.write(f"Hello, {databricks_user.display_name}!")

with sql.connect(
    server_hostname=DATABRICKS_HOST,
    http_path=SQL_HTTP_PATH,
    auth_type="databricks-oauth",
    credentials_provider=credentials_provider,
) as connection:
    with connection.cursor() as cursor:
        cursor.execute("SELECT * FROM samples.nyctaxi.trips LIMIT 10;")
        result = cursor.fetchall()
        st.table(pd.DataFrame(result))

Deploy

  1. The Posit Connect Publisher (content author) updates their application code to obtain OAuth tokens from the OAuth Integration (via the Posit Python SDK), then deploys the content to Posit Connect.

  2. Once deployed, the publisher enables the OAuth Integration for their content using the guid of the Databricks OAuth Integration that was created by the Posit Connect administrator. The example below shows how to associate the new OAuth Integration with the deployed content using curl.

Publishers can see all available OAuth Integrations using the following curl command: curl -H "Authorization: Key ${CONNECT_API_KEY}" https://connect.example.org/__api__/v1/oauth/integrations

curl -H "Authorization: Key ${CONNECT_API_KEY}" -XPUT \
https://connect.example.org/__api__/v1/content/<content-guid>/oauth/integrations/associations \
--data '[{"oauth_integration_guid": "<oauth-integration-guid>"}]'

View

Posit Connect users who have the Viewer permission on the published content can now visit the content to receive a personalized view of the published application.

When interacting with the content for the first time from the Posit Connect dashboard, the content viewer is prompted to login to the OAuth Integration. After clicking Login, they are redirected to Azure and must complete the login flow. Once complete, they are directed back to the content on Posit Connect.

When interacting with the content from the open-solo view, the viewer is not prompted to login. They must visit the login URL of the OAuth Integration before interacting with the deployed content in order to initialize their OAuth Session. The login URL has the form: https://connect.example.org/__oauth__/integrations/<oauth-integration-guid>/login

The Login modal displayed on the viewer's first visit to the content.

The content now uses the viewer’s personal Databricks credentials when interacting with Databricks APIs providing the viewer with a personalized view of the published content in Posit Connect.