Connect Databricks

-- Unity Catalog must be enabled for metadata ingestion
-- 
-- To verify Unity Catalog is enabled:
-- 1. Go to your Databricks workspace
-- 2. Navigate to Catalog Explorer
-- 3. If you see catalogs, schemas, and tables, Unity Catalog is enabled
-- 
-- Note: Unity Catalog is required for Unity Layer metadata ingestion

-- To create an access token:
-- 1. Click on your user icon (top right)
-- 2. Select "User Settings"
-- 3. Go to "Access Tokens" tab
-- 4. Click "Generate New Token"
-- 5. Add a comment (e.g., "Unity Layer Integration")
-- 6. Set expiration (optional)
-- 7. Click "Generate"
-- 8. COPY THE TOKEN IMMEDIATELY (you won't see it again!)
-- 
-- Save the token securely

-- Grant the following permissions in Unity Catalog:
-- 
-- 1. USE CATALOG on target catalog
-- 2. USE SCHEMA on target schemas
-- 3. SELECT on schemas/tables
-- 
-- Example SQL:
-- GRANT USE CATALOG ON CATALOG your_catalog TO unitylayer_user;
-- GRANT USE SCHEMA ON SCHEMA your_catalog.your_schema TO unitylayer_user;
-- GRANT SELECT ON SCHEMA your_catalog.your_schema TO unitylayer_user;

-- Validate the connection:
-- 1. Use workspace URL, token, and catalog in Unity Layer
-- 2. Check that object metadata appears in Unity Layer
-- 3. Verify schemas and tables are accessible
-- 
-- Required information:
-- - Workspace URL (e.g., https://your-workspace.cloud.databricks.com)
-- - Token (personal access token)
-- - Catalog (your Unity Catalog name)