Accessing Azure Databricks Data Table from SAS 9.4 Environment

In order to connect to a SPARK cluster from an external application, a Databricks workspace user credential is required. This credential serves as the entry point for external applications to access objects and data within Databricks.

Before accessing the data table from Azure Databricks using SAS 9.4, it is essential to have or create an Azure Databricks workspace. The following steps outline the necessary configuration:

1. SPARK Data-frame Saved as a Table: Save your SPARK data-frame as a table in order to share it with SAS applications.

2. Configure SPARK Cluster for Storage Account ADLS2 Storage: Configure your SPARK cluster to access storage account ADLS2 storage.

3. Access Token from Azure Databricks Workspace: Obtain an access token from the Azure Databricks workspace.

4. JDBC Jar File Available at SAS Compute Server: Ensure that the JDBC jar file is available at the SAS compute server.

5. SAS/ACCESS Interface to JDBC at SAS Compute Server: Install and configure the SAS/ACCESS interface to JDBC at the SAS compute server.

Currently, users can only read data from Azure Databricks using the JDBC Access engine. However, a new SAS/ACCESS engine for Databricks is scheduled for release in August-September 2022. At present, there is no dedicated SAS/ACCESS Interface to the Databricks database.

To access the Azure Databricks workspace and data tables using SAS 9.4, users can utilize the JDBC Driver provided by Azure Databricks. The SPARK engine is used to process data within Azure Databricks, which is optimized for Azure data lakes and provides an interactive workspace for setting up environments and collaborating with data scientists.

Azure Databricks is a jointly developed data and AI service from Databricks and Microsoft, designed specifically for data analytics users. With its robust features and scalable architecture, Azure Databricks enables users to efficiently process large datasets and gain valuable insights from their data.

In summary, accessing the Azure Databricks data table from SAS 9.4 requires a Databricks workspace user credential, configuration of SPARK cluster settings, and utilization of the JDBC Access engine or the new SAS/ACCESS engine for Databricks scheduled for release in August-September 2022.

Azure data studio databricks