feat(rust-pyo3): add POC PyO3 Python bindings for the kernel

[vikrantpuppala]

Summary

This is a proof-of-concept / RFC. Not for merge as-is.

Adds a new satellite crate rust-pyo3/ that exposes the Databricks ADBC Rust kernel to Python via PyO3 + abi3. Validates the multi-language strategy in docs/kernel-strategy-final-recommendation.md — Decision 2: first-party PyO3 binding over routing through adbc_driver_manager.

What works

• End-to-end smoke against a live SEA warehouse (PAT auth).
• Inline + CloudFetch result paths.
• pyarrow.Table return via Arrow C Data Interface (zero-copy where the schema permits).
• abi3-py39 wheel — one wheel covers Python 3.9+ versions.
• Performance at parity with or better than the existing Thrift-based path on most query sizes (see numbers in perf(rust): reduce per-query overhead and coalesce small batches #422).

Public surface

import databricks_adbc_pyo3 as dbx

conn  = dbx.Connection(host, http_path, access_token, *, catalog=None, schema=None)
rs    = conn.execute(sql)            # → ResultSet (returns when schema is known)
rs.num_columns(); rs.column_names(); rs.arrow_schema()
batch = rs.fetch_next_batch()        # → pyarrow.RecordBatch | None  (streaming)
table = rs.fetch_all_arrow()         # → pyarrow.Table              (drains rest)

Explicitly out of scope (POC)

• PAT only — no OAuth M2M, U2M, Azure SP, or external credential providers.
• No metadata methods (get_objects, get_table_schema, get_table_types).
• No async execute, no cancel(), no Ctrl-C signal handling, no logging bridge into Python logging.
• No prepared statements / parameter binding from Python.
• No tests, no CI integration, not packaged for PyPI.

Design notes for reviewers

• Wraps the kernel's ADBC Optionable layer (string-keyed config). The kernel-strategy doc proposes typed Rust config structs (DatabricksConfig, AuthConfig, …) as Phase 0a. Once that lands, this binding should switch to typed construction. Currently this layering choice is a deliberate POC compromise — Phase 0a is a separate workstream and didn't seem worth blocking the binding spike on.

• Uses a new Statement::execute_owned inherent method (in perf(rust): reduce per-query overhead and coalesce small batches #422) that returns Box<dyn RecordBatchReader + Send + 'static>, decoupling the reader's lifetime from the Statement. This lets the PyO3 wrapper hold the reader past the Statement's drop, which is needed for streaming.

• Releases the GIL during all kernel-side work (execute_owned, batch drain) and reacquires once for the pyarrow conversion phase.

Dependency

Depends on #422 for the execute_owned API and the server_side_closed cleanup short-circuit. This branch is based on that one; if #422 lands first, this rebases trivially onto main.

Open questions

1. Do we want to keep this satellite under databricks-adbc/rust-pyo3/ long-term, or move it to its own repo (databricks-adbc-python or similar) once it grows beyond a POC?
2. Phase 0a (typed config structs) is a precondition for a production-quality binding. Should this PR land as-is and we do Phase 0a + a typed-API rewrite as a follow-up, or is it cleaner to hold this until Phase 0a is done?
3. The kernel-strategy doc commits to PyO3 + abi3 — but the existing Python design doc (python-driver-rust-adbc-sea-design.md §4.2-4.3) still describes the adbc_driver_manager integration. The newer doc supersedes that, but the older sections need a rewrite.

Test plan

• maturin develop --release produces an abi3-py39 wheel.
• import databricks_adbc_pyo3 works in a fresh venv.
• examples/e2e_smoke.py round-trips small (SELECT 1) and large (1M rows via range()) queries against a dogfood warehouse.
• Production tests, CI integration, real auth, real packaging — all deferred.

This pull request and its description were written by Isaac.