Call Databricks Jobs from a Python Service

• Install dependencies
• Example job trigger
• Wrap it in a small API
• Why teams use this pattern
• Notes

Databricks jobs are often only one part of a larger delivery flow. A Python service may still need to receive a request, validate inputs, trigger a notebook or workflow, and return a run identifier to the caller.

Install dependencies

pip install databricks-sdk fastapi uvicorn

Example job trigger

import os

from databricks.sdk import WorkspaceClient


def trigger_job(customer_id: str, job_id: int) -> str:
    client = WorkspaceClient(
        host=os.environ["DATABRICKS_HOST"],
        token=os.environ["DATABRICKS_TOKEN"],
    )

    run = client.jobs.run_now(
        job_id=job_id,
        job_parameters={"customer_id": customer_id},
    )

    return str(run.run_id)

Wrap it in a small API

from fastapi import FastAPI
from databricks.sdk import WorkspaceClient

app = FastAPI()
client = WorkspaceClient()


@app.post("/runs/customer-refresh")
def run_customer_refresh(customer_id: str):
    run = client.jobs.run_now(
        job_id=123456789,
        job_parameters={"customer_id": customer_id},
    )
    return {"run_id": run.run_id}

Why teams use this pattern

• external apps should not need notebook-specific logic
• request validation can happen before cluster work starts
• authentication and auditing are easier to centralize
• one service can route to several jobs without exposing workspace details

Notes

• keep workspace credentials outside source control
• validate input before starting the run
• return the run id so callers can check status elsewhere
• separate quick API responses from long-running compute