Skip to content

Per-tenant retention periods#3879

Merged
pstibrany merged 9 commits into
cortexproject:masterfrom
stevesg:per-user-retention
Mar 1, 2021
Merged

Per-tenant retention periods#3879
pstibrany merged 9 commits into
cortexproject:masterfrom
stevesg:per-user-retention

Conversation

@stevesg

@stevesg stevesg commented Feb 25, 2021

Copy link
Copy Markdown
Contributor

What this PR does:
Implement per-tenant retention periods by performing block deletion in the Compactor.

Note that I've split the change into three commits to make reviewing it easier:

  1. Add the new configuration
  2. Add the new metrics
  3. The change + unit tests

Which issue(s) this PR fixes:
Fixes #3622

Checklist

  • Tests updated
  • Documentation added
  • CHANGELOG.md updated - the order of entries should be [CHANGE], [FEATURE], [ENHANCEMENT], [BUGFIX]

@stevesg stevesg changed the title Per user retention Per-tenant retention periods Feb 25, 2021
@stevesg
stevesg marked this pull request as ready for review February 25, 2021 12:50
@pracucci
pracucci requested review from pracucci and pstibrany and removed request for pstibrany February 25, 2021 15:09

@pracucci pracucci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Very good job 👏 👏 👏 The PR LGTM! I left few nits, but everything should be quick to address 🚀

Comment thread CHANGELOG.md Outdated
Comment thread pkg/util/validation/limits.go Outdated
Comment thread pkg/util/validation/limits.go Outdated
Comment thread pkg/compactor/compactor.go Outdated
Comment thread pkg/compactor/blocks_cleaner_test.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/blocks_cleaner_test.go Outdated
Comment thread pkg/compactor/blocks_cleaner_test.go Outdated
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>
@stevesg

stevesg commented Feb 26, 2021

Copy link
Copy Markdown
Contributor Author

Rebase

@pracucci pracucci left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Good job, LGTM! (modulo a couple of nits)

Comment thread CHANGELOG.md Outdated
Comment thread pkg/util/validation/limits.go Outdated
Comment thread pkg/util/validation/limits.go Outdated
Comment thread pkg/util/validation/limits.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I see, but the log below will be misleading ("applying retention"). I think we should also check if retention <= 0 to circuit break.

Comment thread pkg/compactor/blocks_cleaner.go Outdated
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>

@pstibrany pstibrany left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, nice job. Thank you!

Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/blocks_cleaner.go Outdated
Comment thread pkg/compactor/compactor.go
Signed-off-by: Steve Simpson <steve.simpson@grafana.com>

@pstibrany pstibrany left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Thanks.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Feature request: be able to specify the retention time for blocks

3 participants