Databricks Integration

​

Prerequisites

• Unity Catalog enabled, with at least one catalog you want Qualia to read.
• A SQL warehouse. Serverless is recommended — it starts in seconds, where a classic warehouse can take several minutes on the first query.
• A Genie Space covering the tables Qualia should answer questions about. Selected during setup and checked by the access preflight.
• Partner-powered AI features enabled in workspace admin settings. Genie does not answer without it.
• An OAuth application registered in your Databricks account console. Databricks does not support automatic app registration, so this step cannot be skipped.

​

Required privileges

​

Registering the OAuth application

1. Note the client ID. Create a client secret as well if you want a confidential client; a public client works without one.
2. Add the redirect URL shown in Qualia’s Databricks settings. It must match exactly, including the path and, on desktop, the port. Databricks does not accept the arbitrary loopback port that RFC 8252 allows, which is why the desktop URL names a fixed one — http://127.0.0.1:8020/api/databricks/callback by default. Set DATABRICKS_CALLBACK_PORT if something else on your machine already holds that port, then register the URL Qualia shows afterward.
3. Grant the scopes sql, genie, unity-catalog, offline_access, openid, email, and profile. unity-catalog is what lets the access preflight list your catalogs; offline_access lets Qualia refresh access without sending you back through the browser every hour; and openid email profile identify who authorized, so Qualia can bind the authorization to your Qualia account.

​

Connecting

1. Go to Settings → Integrations → Databricks.
2. Enter your workspace URL (for example dbc-a1b2c3d4-e5f6.cloud.databricks.com) and pick the cloud it runs on.
3. Paste the OAuth client ID, and the client secret if you created one.
4. Confirm the redirect URL matches what you registered.
5. Click Save and continue. Qualia verifies the workspace responds before storing anything.
6. Under Your access, click Sign in with Databricks and authorize.

​

Authentication methods

​

Setting up a service principal (M2M)

1. In your Databricks account console, go to User management → Service principals and create one.
2. On the service principal, generate an OAuth secret. Record the client ID and secret; the secret is shown once.
3. Grant it the same privileges a person needs — CAN USE on the SQL warehouse, USE CATALOG and USE SCHEMA on the parents, and SELECT on the tables.
4. Provide the client ID and secret through the Databricks Partner Connect tile. Nothing in Qualia’s settings page accepts an M2M secret, so a workspace you connected by hand should use token federation below instead — it needs no secret at all.

​

Setting up token federation

1. Create a service principal as above — steps 1 and 3, skipping the OAuth secret.
2. In Qualia, open Settings → Integrations → Databricks. Under Authentication method, set Query as to Service principal, then enter the service principal’s application ID. Leave Federation audience blank unless your policy needs a specific one. Save. Note what this changes: every query then runs as that one service principal, so its Unity Catalog grants apply to everyone in your organization and no answer is attributable to an individual. Leave Query as on Each person if you want per-user governance.
3. A Federation policy section appears with the exact values Qualia will sign. They are specific to your organization, and each one has a copy button.
4. In your Databricks account console, open the service principal and add a federation policy:

​

Partner Connect

​

Querying from a notebook

import qualia_databricks

df = qualia_databricks.query("SELECT * FROM main.sales.orders LIMIT 100")

df <- databricks_query("SELECT * FROM main.sales.orders LIMIT 100")

using DataFrames
df = DataFrame(Qualia.Databricks.query("SELECT * FROM main.sales.orders LIMIT 100"))

qualia_databricks.query(
    "SELECT * FROM main.sales.orders WHERE total > :floor",
    {"floor": 100},
)

databricks_query(
  "SELECT * FROM main.sales.orders WHERE total > :floor",
  list(floor = 100)
)

Qualia.Databricks.query(
    "SELECT * FROM main.sales.orders WHERE total > :floor";
    parameters = Dict("floor" => 100),
)

with qualia_databricks.connect() as connection:
    with connection.cursor() as cursor:
        cursor.execute("SELECT count(*) FROM main.sales.orders")
        total = cursor.fetchone()[0]

​

Asking Genie

​

Reconnecting

​

Rotating the client secret

​

Disconnecting

​

Where credentials are stored

​

Troubleshooting

​

Attribution

• Renders Genie’s content blocks unmodified rather than summarized by the model.
• Keeps every URI Genie returns a working link.
• Cites the Genie Space that produced it.