Skip to content

Remove fog-aws and fog-core, complete fog removal#5288

Open
kathap wants to merge 24 commits into
mainfrom
remove-fog-aws
Open

Remove fog-aws and fog-core, complete fog removal#5288
kathap wants to merge 24 commits into
mainfrom
remove-fog-aws

Conversation

@kathap

@kathap kathap commented Jul 14, 2026

Copy link
Copy Markdown
Contributor

Remove the last remaining fog gems (fog-aws, fog-core) and all fog blobstore code. All fog gems (fog-azure-rm, fog-google, fog-aliyun) have now been fully removed. storage-cli is the default blobstore backend and has fully replaced fog. Removing fog-aws and fog-core eliminates their transitive dependencies (fog-json, fog-xml) as well, completing the fog removal and reducing ongoing Dependabot maintenance burden.

Changes:

  • Remove fog-aws and fog-core from Gemfile
  • Delete lib/cloud_controller/blobstore/fog/ entirely (FogClient, FogBlob, Cdn, providers)
  • Remove fog branch and require from blobstore ClientProvider
  • Remove fog_connection and fog_aws_storage_options config keys from all schemas (api, worker, clock, deployment_updater, blobstore_benchmarks) and YAML fixtures (cloud_controller.yml, bosh-lite.yml,
    port_8181_config.yml)
  • Remove AWS legacy provider mapping from StorageCliClient
  • Replace Fog.mock! and AWS fog_connection credentials in test infrastructure with local blobstore
  • Delete all fog specs under spec/unit/lib/cloud_controller/blobstore/fog/
  • Fix LocalBlob#content_length to return Integer instead of String
  • Fix LocalClient#delete_all to respect root_dir
  • Add abstract interface methods (internal_download_url, public_download_url, local_path) to Blob base class
  • Remove fog blobstore from README and update ADR-0014 to reflect completed fog removal

@kathap
kathap force-pushed the remove-fog-aws branch 2 times, most recently from 2a2cc09 to ff727cd Compare July 14, 2026 11:41
Replace obsolete FogClient/fog_connection test setup with LocalClient and
blobstore_type: 'local'. Job and controller specs that used FogClient as a
test double now use LocalClient directly; config-based specs switch from
fog_connection to blobstore_type/local_blobstore_path. Remove Fog::Mock.reset
calls that are no longer needed. Delete fog_client_spec.rb — it tested the
deprecated FogClient which requires fog provider gems no longer bundled.

  * Tests were broken because fog-aws removal left no registered fog provider,
    so Fog::Storage.new(provider: ...) raised ArgumentError at runtime.
  * Blobstore behavior under test is the job/controller logic, not fog internals,
    so LocalClient is the right substitute.
kathap added 2 commits July 14, 2026 13:52
fog_client_spec.rb restored — rewritten to stub Fog::Storage.new with a FakeStorage in-memory backend defined in the spec itself. All provider-independent FogClient behavior (uploads, downloads, copies,
deletion, CDN, file permissions, bucket management, root dir scoping) is covered. The AWS-specific contexts (encryption, delete_multiple_objects via AWS bulk delete) are not included since those were the
AWS-only parts removed in this PR. No fog provider gem is needed.

Three job specs — blobstore_path extracted into a named let, after { FileUtils.rm_rf(blobstore_path) } added to each.
kathap added 3 commits July 14, 2026 14:49
Each call to the non-memoized method returned a different proxy object,
so stubs and expect().to receive() set on one proxy were never observed
by FogClient which called directories on a different instance.
kathap added 9 commits July 15, 2026 09:16
…tance

allow_any_instance_of(LocalClient) patches the class itself, leaving RSpec
mock state that fires when LocalClient at_exit hooks run after the suite ends
(triggered by use_temp_storage: true instances from test_config defaults).
Stubbing the specific package_blobstore instance avoids this entirely.
…ectations

OutsideOfExampleError was already fixed by 92a3cca (instance-specific stub).
The BLOBSTORE_PATH constant and local blobstore_type were unnecessary — revert
to local-temp-storage. Restore packages_spec download expectations to 302 +
Location header, matching actual blobstore URL generator behavior.
@kathap kathap changed the title Remove fog-aws gem and all fog AWS blobstore support Remove fog gems and all fog blobstore support Jul 16, 2026
@kathap kathap changed the title Remove fog gems and all fog blobstore support Remove fog-aws and fog-core, complete fog removal Jul 17, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant