Databricks connection patterns on Connect using viewer OAuth authentication

These access patterns enable you to connect to and read from Databricks databases or schemas in Posit Connect using Connect Viewer-level OAuth Authentication. They include both Python and R code examples for connecting to Databricks using Spark and SQL.

Choose a different pattern by referring to the table in Choose a connection for Databricks.

For an overview of credential types and scope in different environments, see Credential types and Managing data access from development to deployment.

About Connect viewer-level OAuth authentication

ImportantAdministrator configuration required

To use this type of connection, your Connect administrator must first configure a viewer type Databricks OAuth integration in Connect. Your Databricks Workspace Administrator must also grant viewers appropriate permissions to access Databricks clusters and SQL warehouses based on their individual credentials.

For detailed setup instructions, see the Databricks OAuth Integrations page in the Connect Admin Guide.

Viewer integrations allow published content to use the content viewer’s credentials when accessing protected resources, providing the viewer with a personalized view of the content. Viewers can only see the data the external system grants them access to.

Viewer integrations only support interactive content types. Viewer integrations do not support statically rendered content, content with access type Anyone - no login required, or Share links.

Required steps when publishing

You must enable the OAuth integration for your specified content using the Access tab in the Content Settings pane in Connect, or by selecting the integration upon deployment.

For more details, refer to the Adding Integrations section of the Connect User Guide.

Spark cluster connection

ImportantBuild the connection per session, not at global scope

The viewer’s session token only exists at request time, so the connection must be created inside per-request code, for example, within server() in a Shiny app — not at global scope.

A connection created at global scope would either fail at startup with no token available, or be shared across all viewers, which contradicts the per-viewer access that viewer OAuth provides.

To authenticate with Databricks using viewer credentials, your code must retrieve the viewer’s session token from the Posit-Connect-User-Session-Token HTTP header and exchange it for a Databricks OAuth access token. The posit.connect.external.databricks module from the posit-sdk package simplifies this token exchange process, with ConnectStrategy() handling the exchange when deployed to Connect.

The code example below uses different authentication strategies depending on the environment:

  • ConnectStrategy retrieves and exchanges the content session token when deployed to Connect
  • WorkbenchStrategy uses Workbench managed credentials in Workbench
  • databricks_cli provides a fallback for local development.

This approach allows the same code to run seamlessly across environments.

import os

# ...
import pandas as pd
from databricks.connect import DatabricksSession
from databricks.sdk.core import databricks_cli
from posit.connect.external.databricks import ConnectStrategy, databricks_config
from posit.workbench.external.databricks import WorkbenchStrategy

# ...
# Retrieve user session token
# The token is stored on the http header for the interactive user session.
# Depending on what framework you are using (e.g., Shiny, Streamlit, etc.)
# the framework will have a different retrieval method for the header.
# This example shows retrieval of the token when using Shiny.
from shiny.express import session

session_token = session.http_conn.headers.get("Posit-Connect-User-Session-Token")

# Configure Databricks connection using posit external databricks
config = databricks_config(
    posit_default_strategy=databricks_cli,
    posit_workbench_strategy=WorkbenchStrategy(),
    posit_connect_strategy=ConnectStrategy(user_session_token=session_token),
    host=f"https://{os.environ['DATABRICKS_HOST']}",
    cluster_id=os.environ["DATABRICKS_CLUSTER_ID"],
)

# Create Databricks Spark session
spark = DatabricksSession.builder.sdkConfig(config).getOrCreate()

# Read a table located at "samples.nyctaxi.trips"
df = spark.read.table("samples.nyctaxi.trips")
df.show(5)

# Convert the Spark table to a Pandas DataFrame
pandas_df = df.limit(5).toPandas()

pandas_df

To authenticate with Databricks using viewer credentials, your code must retrieve the viewer’s session token and exchange it for a Databricks OAuth access token. The connectcreds package simplifies this process, with the connect_viewer_token() function handling the token retrieval and exchange when deployed to Connect.

This code example uses conditional logic to authenticate depending on the environment: when deployed to Connect, connect_viewer_token() retrieves and exchanges the viewer’s session token, while in development environments like Workbench, you can use an alternative authentication method. This approach allows the same code to run across different environments.

library(shiny)
library(sparklyr)
library(pysparklyr)
library(connectcreds)
library(dplyr)
library(dbplyr)

ui <- fluidPage(
  # ... your UI elements
)

server <- function(input, output, session) {
  # if running on Connect
  if (Sys.getenv("POSIT_PRODUCT") == "CONNECT") {
    sc <- spark_connect(
      cluster_id = "<your-cluster-id>",
      method = "databricks_connect",
      token = connectcreds::connect_viewer_token()$access_token
    )

    # if running outside of Connect (in development)
  } else {
    # <retrieve the access token using your preferred development method>
    access_token <- "<your-access-token>"
    # <make spark connection to Databricks using example access patterns in this document>
    sc <- spark_connect(
      cluster_id = "<your-cluster-id>",
      method = "databricks_connect",
      token = access_token
    )
  }

  # Interact with the Spark table at "my_catalog.my_schema.my_table" as a dplyr data frame
  data <- tbl(sc, dbplyr::in_catalog("samples", "nyctaxi", "trips"))

  # ... render to UI
}

shinyApp(ui, server)

For more in-depth information and complete examples, see:

SQL warehouse connection

ImportantBuild the connection per session, not at global scope

The viewer’s session token only exists at request time, so the connection must be created inside per-request code, for example, within server() in a Shiny app — not at global scope.

A connection created at global scope would either fail at startup with no token available, or be shared across all viewers, which contradicts the per-viewer access that viewer OAuth provides.

To authenticate with Databricks using viewer credentials, your code must retrieve the viewer’s session token from the Posit-Connect-User-Session-Token HTTP header and exchange it for a Databricks OAuth access token. The posit.connect.external.databricks module from the posit-sdk package simplifies this token exchange process, with ConnectStrategy() handling the exchange when deployed to Connect.

The code example below uses different authentication strategies depending on the environment:

  • ConnectStrategy retrieves and exchanges the content session token when deployed to Connect
  • WorkbenchStrategy uses Workbench managed credentials in Workbench
  • databricks_cli provides a fallback for local development.

This approach allows the same code to run seamlessly across environments.

import os

# ...
import pandas as pd
from databricks.sdk.core import (
    databricks_cli,  # this can be removed if not using `posit_default_strategy`
)
from posit.connect.external.databricks import ConnectStrategy, databricks_config
from posit.workbench.external.databricks import WorkbenchStrategy

# ...
# Retrieve user session token
# The token is stored on the http header for the interactive user session.
# Depending on what framework you are using (e.g., Shiny, Streamlit, etc.)
# the framework will have a different retrieval method for the header.
# This example shows retrieval of the token when using Shiny.
from shiny.express import session
from sqlalchemy import create_engine, text

session_token = session.http_conn.headers.get("Posit-Connect-User-Session-Token")

# Configure Databricks connection using environment variables
config = databricks_config(
    posit_default_strategy=databricks_cli,
    posit_workbench_strategy=WorkbenchStrategy(),
    posit_connect_strategy=ConnectStrategy(user_session_token=session_token),
    host=f"https://{os.environ['DATABRICKS_HOST']}",
    warehouse_id=os.environ["DATABRICKS_HTTP_PATH"],
)

# Create SQLAlchemy engine
connection_string = (
    f"databricks://token:{config.token}@{os.environ['DATABRICKS_HOST']}:443"
)
engine = create_engine(
    connection_string, connect_args={"http_path": os.environ["DATABRICKS_HTTP_PATH"]}
)

# Query data
query = "SELECT * FROM samples.nyctaxi.trips LIMIT 10"
df = pd.read_sql(text(query), engine)

df

The odbc package provides a helper function odbc::databricks() to allow for seamless connections to Databricks using user-based Workbench managed credentials and Connect viewer OAuth credentials without needing to adjust your code.

This is the recommended approach for connecting to Databricks SQL warehouses from R. The only difference between this code and what is shown in the Posit Workbench Managed Credentials section is the addition of library(connectcreds), which automatically detects the environment and retrieves the viewer’s credentials when running on Connect.

See the databricks() helper function in the odbc package documentation for more information.

library(shiny)
library(dplyr)
library(dbplyr)
library(connectcreds) # this is required for the code to run using viewer credentials on Posit Connect

ui <- fluidPage(
  # ... your UI elements
)

server <- function(input, output, session) {
  con <- DBI::dbConnect(
    odbc::databricks(),
    httpPath = Sys.getenv("DATABRICKS_HTTP_PATH")
  )

  # Use dplyr to query data at "my_catalog.my_schema.my_table"
  data <- tbl(con, DBI::Id("samples", "nyctaxi", "trips"))

  # ... render to UI
}

shinyApp(ui, server)

These code samples are intentionally minimal. For more in-depth information and complete examples, see: