[core] Fix copyWithLatestSchema option precedence#8550
Conversation
Generated-by: Claude Code
|
After reversing the merge order of schema options, 'copyWithLatestStyle()' will lose the runtime override of 'table. copy (dynamicOptions)'. Temporary regression test stable failure: Expected snapshot. num-retained. max=200, but actually recovered to persistent value of 100. This method is called multiple times in the Flink CDC schema refresh and retry link, which belongs to actual regression; It is necessary to distinguish between persistent options and dynamic options, and it is not possible to simply reverse the overall merging order. |
The previous fix reversed the merge order so the latest committed schema options win, which fixed altered persistent values not being picked up (apache#4957) but dropped runtime overrides applied via copy(dynamicOptions): those live only in the current table copy's schema, so letting the committed schema win clobbered them (e.g. snapshot.num-retained.max fell back from a runtime override to the committed value). copyWithLatestSchema is called on the Flink CDC schema refresh/retry path, so this was a real regression. Rebase onto the latest committed options, then re-apply only the dynamic overrides -- the entries whose value differs from the committed schema this copy is based on. Both altered persistent values and runtime overrides now survive a refresh. Added a test covering the dynamic-option case alongside the existing altered-value test. Generated-by: Claude Code
|
You're right — real regression. Dynamic options from |
|
|
Purpose
fix #4957
copyWithLatestSchema()now lets the latest committed schema options override stale options from the current table copy, so altered option values survive a refresh.Tests
AppendOnlySimpleTableTest#testCopyWithLatestSchemaPicksUpAlteredOptionValues: altersnapshot.num-retained.maxto 10, then assertcopyWithLatestSchema()reflects it.expected: 10 but was: 100; passes after.mvn -pl paimon-core -am clean installran the normal checks (checkstyle/spotless/rat); onlyPostgresqlCatalogTestwas skipped for local Docker absence.