Infer Docker registry host from workspace profile - #6782
Merged
pietern merged 1 commit intoSep 24, 2026
Merged
Conversation
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
from
September 21, 2026 17:30
d4fef20 to
416ca6a
Compare
Collaborator
Integration test reportCommit: 3a91fa6
Top 3 slowest tests (at least 2 minutes):
|
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
from
September 21, 2026 19:09
416ca6a to
319effe
Compare
yolocs
marked this pull request as ready for review
September 21, 2026 19:38
Contributor
Approval status: pending
|
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
from
September 22, 2026 06:07
319effe to
62668a9
Compare
pietern
reviewed
Sep 22, 2026
pietern
left a comment
pietern
left a comment
Contributor
There was a problem hiding this comment.
The tests in this PR mock pretty much everything and then go through e2e flows.
I recommend testing the units independently and reserving e2e for acc tests.
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
from
September 22, 2026 18:43
62668a9 to
e07b06f
Compare
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
2 times, most recently
from
September 23, 2026 05:36
fc154fa to
2e32231
Compare
Collaborator
Author
I restructured the unit tests to exercise each component directly, covering host derivation, helper status, output formats, errors, and config immutability. I couldn’t add a successful host acceptance test without a much larger change because the logic depends on the profile’s workspace domain. I believe the existing configure acceptance test still provides solid end-to-end coverage of the Docker helper flow. |
Co-authored-by: Isaac <no-reply@databricks.com>
yolocs
force-pushed
the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
from
September 23, 2026 17:00
2e32231 to
3a91fa6
Compare
pietern
approved these changes
Sep 24, 2026
pietern
deleted the
chen-shou_data/stack/auth-docker-infer-region-hosts
branch
September 24, 2026 08:41
deco-sdk-tagging Bot
added a commit
that referenced
this pull request
Sep 24, 2026
## Release v1.18.0 ### CLI * The AI Runtime commands have moved to `databricks air`. The previous `databricks experimental air` path now directs users to the new command. ([#6722](#6722)) * Write local state, cache, and config files atomically so an interrupted or concurrent write cannot corrupt them. ([#6708](#6708)) * Deprecate `--region` in `databricks auth docker configure` ahead of its removal in the next release, infer the Artifact Registry region when it is omitted, and add `databricks auth docker host --profile <name>` to show the profile's registry host and credential-helper status. ([#6782](#6782)) * Return `UNAUTHENTICATED` instead of `INVALID_REFRESH_TOKEN` when `databricks auth token --output json` cannot refresh a cached U2M token. ([#6731](#6731)) * Retry the current-user (SCIM `Me`) lookup on transient HTTP 500 responses so a temporarily-unavailable backend no longer fails bundle commands outright. ([#6766](#6766)) * Preserve workspace-file and volume access for SSH server descendants when the bootstrap notebook exits and the server survives. ([#6645](#6645)) * `ssh connect` and `ssh setup` now accept a `--keep-detached-processes` flag to keep processes detached from the SSH session (`tmux`, `setsid`, `nohup`) running after the tunnel shuts down. Teardown then terminates only the tunnel's own process group, and the bootstrap job run is held open while any detached process is still running, so the survivors keep their `/Workspace` and `/Volumes` access. A held-open run also suppresses cluster autotermination, so the flag is off by default, is bounded by `--server-timeout`, and is dedicated-cluster only. Without it, the server now logs a warning naming the detached processes it is about to destroy, instead of sweeping them silently. ([#6387](#6387)) ### Bundles * direct: Allow clearing a catalog's or schema's `custom_max_retention_hours` by removing it from configuration. ([#6792](#6792)) * direct: Allow clearing a genie space's `description` and a secret's `comment` by removing them from configuration. ([#6789](#6789)) * Fix direct-engine deploy recreating an MLflow experiment on every deploy when its `trace_location` was set out-of-band. ([#6787](#6787)) * direct: Store a Genie space's `serialized_space` in state as a content hash instead of its full contents. ([#6707](#6707)) * Fix `bundle deploy` failing with "Invalid python file reference" for jobs that use `git_source` with a `spark_python_task` on the direct engine. ([#6751](#6751)) * Fixed the direct engine mishandling UC grants that combine `ALL_PRIVILEGES` with a privilege it does not imply (`MANAGE`, `READ_METADATA`, `EXTERNAL_USE_SCHEMA`, `EXTERNAL_USE_LOCATION`): such privileges were dropped when granted and left behind when revoked, so the deployment never converged. ([#6733](#6733), [#6743](#6743)) * Don't fail migration if clean up actions fail. ([#6772](#6772)) * Ignore the backend-provided `spark.sql.ansi.enabled: "true"` pipeline configuration default when detecting direct-engine drift. ([#6816](#6816)) * Fix recreating a postgres synced table sometimes failing with a 409 ALREADY_EXISTS error while the previous table is still being deleted. ([#6728](#6728)) * Direct engine no longer recreates a resource when an immutable field the config omits was populated by the backend. ([#6790](#6790)) ### Dependency Updates * Bump dependencies with known vulnerabilities. ([#6723](#6723)) * Bump `github.com/databricks/databricks-sdk-go` from v0.178.0 to v0.182.0. ([#6817](#6817)) * Bump the Databricks Terraform provider from v1.132.0 to v1.134.0. ([#6818](#6818))
Collaborator
Integration test reportCommit: 92335ba
13 interesting tests: 10 flaky, 3 FAIL
Top 50 slowest tests (at least 2 minutes):
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
🥞 Stacked PR (generated by git ns)
Use this link to review incremental changes.
Changes
Docker setup now needs only a workspace profile. The CLI resolves the workspace ID, obtains the workspace region from its metastore summary, derives the full Artifact Registry hostname, and configures Docker for that host:
Users no longer need to look up and supply the workspace home region. The existing
--regionflag remains available for compatibility, emits a deprecation warning when used, and is scheduled for removal in the next release.The new read-only
hostcommand returns both the derived registry hostname and whether Docker is already configured to use the Databricks credential helper:Structured output makes the result available to scripts without reconstructing the hostname:
Both commands now turn an invalid OAuth refresh token into a direct recovery step instead of exposing the underlying token request:
Profiles whose host is not a Databricks workspace domain fail before any workspace request or Docker configuration change:
To run these flows end to end against the acceptance test server, a workspace served from
http://127.0.0.1maps to a.localhostregistry DNS zone (e.g.900800700600.container.us-west-2.localhost). This is the same local-testing exception OAuth login already makes forhttp://127.0.0.1hosts. The credential helper returns tokens for.localhostregistries only to profiles onhttp://127.0.0.1, so a real workspace's token never reaches a local registry. The test server also gains a defaultmetastore_summaryhandler.Why
Workspace profiles do not expose the home region directly, and asking users to discover it separately made Docker setup unnecessarily error-prone. The workspace metastore already provides the matching region, so the CLI can derive the correct registry host from the selected profile and expose that same result for later commands and automation.
The error handling keeps authentication and profile-selection failures actionable, especially when a saved OAuth session has expired or an account profile is selected accidentally.
Tests
Acceptance tests in
acceptance/cmd/auth/docker/now run the real flow against the test server, without--regionand without rewriting the profile mid-test:configure:hostreports the helper as not configured;configureresolves the workspace ID (SCIM) and region (metastore summary), savesworkspace_idto the profile, and writes the exact Docker config;host --output jsonreports it as configured;docker-credential-databricks getreturns a token. Recorded requests show which API calls each step makes. A rerun leaves the Docker config unchanged, and the deprecated--regionflag makes no API calls.configure-errors: unsupported flags, conflicting profile selectors, a non-Databricks host, and a duplicateworkspace_idall fail without creating the Docker config or changing profiles.host: text and JSON output,credsStoredetection, and the profile is never modified.token: cached vs. refreshed token, unknown workspace, non-registry host,--profilerejected, a test-server registry never matching a remote profile, duplicate profiles.invalid-refresh-token:host,configure, andtokenall print the re-login command, and no Docker config is created.Unit tests that walked these flows through mocks were removed. The remaining ones cover what the test server can't reach: shim install ordering, the PATH warning, metastore failures, and the
workspace_id = nonesentinel.go test ./acceptance -run 'TestAccept/cmd/auth' -count=1(129 passed)go test ./cmd/auth/... ./libs/dockercredentials/... ./libs/testserver/..../task lint-q,./task fmt-q,./task wsThis pull request and its description were written by Isaac.