Skip to content

NUTCH-3201 Fetcher launches only a single reduce task, ignoring the configuration - #949

Open
sebastian-nagel wants to merge 1 commit into
apache:masterfrom
sebastian-nagel:NUTCH-3201
Open

NUTCH-3201 Fetcher launches only a single reduce task, ignoring the configuration#949
sebastian-nagel wants to merge 1 commit into
apache:masterfrom
sebastian-nagel:NUTCH-3201

Conversation

@sebastian-nagel

Copy link
Copy Markdown
Contributor

Remove the instruction which configures the single reduce task.

@sonarqubecloud

sonarqubecloud Bot commented Aug 5, 2026

Copy link
Copy Markdown

@lewismc

lewismc commented Aug 5, 2026

Copy link
Copy Markdown
Member

Thanks @sebastian-nagel damn I remember adding this to profile locally and neglected to clean it up. Fantastic catch. This is just a bad bug. Using something like bats to unit test nutch and/or crawl script wouldn't have caught it either. I kinda wish MRUnit wasn't sent to the attic as it could have been used to test such cases. Anyway +1 from me.

@lewismc lewismc left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

+1

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants