Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Feat ignore binlog events #1506

Open
wants to merge 2 commits into
base: master
Choose a base branch
from

Conversation

xiehaopeng
Copy link

A Pull Request should be associated with an Issue.

Related issue: #1484

Further notes in https://github.com/github/gh-ost/blob/master/.github/CONTRIBUTING.md
Thank you! We are open to PRs, but please understand if for technical reasons we are unable to accept each and any PR

Description

Support for Ignoring Binlog Events Beyond Chunk Boundary Values
If the unique key value in a binlog event surpasses the maximum value defined for chunk iteration but remains below the overall maximum boundary for copying, the event is ignored. This approach is inspired by the design concept from shaohk's feat binlog apply optimization. We have implemented the functionality to ignore such binlog events based on the code from release v1.1.7.

Dynamic Expansion of Maximum Copy Boundary Values
To prevent issues arising from the interchangeable use of "copy rows" and "apply events" during gh-ost processes, which can lead to Table Space Bloat Due to Gh-ost #1484, we have introduced a mechanism to dynamically expand the maximum boundary value for copying. By ignoring large unique key insert events, this solution potentially mitigates the problem.

In case this PR introduced Go code changes:

  • contributed code is using same conventions as original code
  • script/cibuild returns with no formatting errors, build errors or unit test errors.

@xiehaopeng xiehaopeng force-pushed the feat-ignore-binlog-events branch from f8d881d to 5903ffc Compare March 13, 2025 12:02
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

1 participant