Refactor Unity Catalog to fetch catalog/schema/table metadata from System tables

[mars-lan]

🤔 Why?

Fetching metadata from system.information_schema is preferred over individual catalog's information_schema, as the former doesn't require granting SELECT permission to all tables.

Note that SELECT permission is still required to retrieve a table's properties & last refresh date as they're not available from various system tables.

🤓 What?

• Replace all REST API calls with queries against system.information_schema. The only exception is IAM-related metadata, which is only available via REST APIs.
• Consolidate all queries into queries.py to make testing easier and better organization.
• Update databricks dependencies to the latest version

🧪 Tested?

Tested end-to-end against a product instance. Verified that the before & after MCEs are the same except:

Columns

• precision for date-related types is set.
• nullable & tag are set.
• Use upper case for nativeType.

Table properties

• Missing Spark-related properties as they're not available via the SHOW TBLPROPERTIES command.
• Use the property's raw values instead of JSON-encoded strings.

Hierarchy:

• Set createdAtSource, createdBy, lastUpdated, updatedBy in sourceInfo
• Set systemContacts & systemDescription

☑️ Checks

• My PR contains actual code changes, and I have updated the version number in pyproject.toml.

[github-actions]

☂️ Python Coverage

current status: ✅

Overall Coverage

Lines	Covered	Coverage	Threshold	Status
13282	11863	89%	85%	🟢

New Files

File	Coverage	Status
metaphor/unity_catalog/queries.py	97%	🟢
TOTAL	97%	🟢

Modified Files

File	Coverage	Status
metaphor/unity_catalog/extractor.py	95%	🟢
metaphor/unity_catalog/models.py	100%	🟢
metaphor/unity_catalog/profile/extractor.py	93%	🟢
metaphor/unity_catalog/utils.py	87%	🟢
TOTAL	94%	🟢

updated for commit: 51ebe03 by action🐍

[codecov]

Codecov Report

Attention: Patch coverage is 96.75516% with 11 lines in your changes missing coverage. Please review.

Project coverage is 89.31%. Comparing base (db18615) to head (51ebe03).
Report is 1 commits behind head on main.

Files with missing lines	Patch %	Lines
metaphor/unity_catalog/extractor.py	95.83%	4 Missing ⚠️
metaphor/unity_catalog/queries.py	97.41%	4 Missing ⚠️
metaphor/unity_catalog/utils.py	89.28%	3 Missing ⚠️

Additional details and impacted files

@@            Coverage Diff             @@
##             main    #1022      +/-   ##
==========================================
+ Coverage   89.26%   89.31%   +0.05%     
==========================================
  Files         202      203       +1     
  Lines       13194    13282      +88     
==========================================
+ Hits        11777    11863      +86     
- Misses       1417     1419       +2     

☔ View full report in Codecov by Sentry.
📢 Have feedback on the report? Share it here.

[usefulalgorithm]

update pyproject.toml and it's good