Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Add CompactionScheduler #151

Closed
wants to merge 32 commits into from
Closed

Add CompactionScheduler #151

wants to merge 32 commits into from

Conversation

kfaraz
Copy link
Owner

@kfaraz kfaraz commented Jul 16, 2024

Pending items

  • callback on taskqueue
  • handle success and failure inside DatasourceQueue
  • make policy serializable
  • add another policy
  • enable segments polling if overlord is standalone
  • test on cluster - standalone, coordinator-overlord
  • unit tests
  • integration tests


public void updateConfig(DataSourceCompactionConfig latestConfig)
{
final DataSourceCompactionConfig previousConfig = config.getAndSet(latestConfig);

Check notice

Code scanning / CodeQL

Unread local variable

Variable 'DataSourceCompactionConfig previousConfig' is never read.

AutoCompactionSnapshot getAutoCompactionSnapshotForDataSource(String dataSource);

Long getTotalSizeOfSegmentsAwaitingCompaction(String dataSource);

Check notice

Code scanning / CodeQL

Useless parameter

The parameter 'dataSource' is never used.
{
Map<Object, AutoCompactionSnapshot> getAutoCompactionSnapshot();

AutoCompactionSnapshot getAutoCompactionSnapshotForDataSource(String dataSource);

Check notice

Code scanning / CodeQL

Useless parameter

The parameter 'dataSource' is never used.
@kfaraz kfaraz changed the base branch from master to refactor_task_master July 18, 2024 09:46
@kfaraz kfaraz changed the base branch from refactor_task_master to master July 21, 2024 05:56
kfaraz and others added 7 commits July 21, 2024 13:56
Fixes apache#16766

Change log level from INFO to DEBUG when processing an empty user map
during polling.  An empty user map is a normal situation for some
authenticators (e.g. LDAP) and polling is frequent (1 minute by
default.)
@kfaraz kfaraz closed this Jul 22, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Projects
None yet
Development

Successfully merging this pull request may close these issues.

2 participants