Skip to content

perf(jira,product): fix finding-group push N+1 and add case-insensitive product-name index#15122

Open
Maffooch wants to merge 4 commits into
devfrom
perf/sentry-nplus1-remediation
Open

perf(jira,product): fix finding-group push N+1 and add case-insensitive product-name index#15122
Maffooch wants to merge 4 commits into
devfrom
perf/sentry-nplus1-remediation

Conversation

@Maffooch

@Maffooch Maffooch commented Jul 1, 2026

Copy link
Copy Markdown
Contributor

Shortcut: sc-13412

Two focused fixes for Sentry `db_query` issues.

Fixes

  • DJANGO-42P8 — `push_finding_group_to_jira` loaded the group without prefetching its findings, so the JIRA helpers it fans out to (jira_description, jira_priority, get_sla_deadline, get_labels, get_tags, ...) each re-ran `finding_group.findings.all()`, producing an N+1 on `dojo_finding_group_findings`. Load the group with `prefetch_related('findings')`.
  • DJANGO-D2M — filtering findings by product name issues `WHERE UPPER(dojo_product.name) = UPPER(%s)`, which the plain unique btree can't serve (slow scan). Add a functional `Upper(name)` index (created CONCURRENTLY, migration 0273).

Testing

  • jira: query-count regression test (5 → 0 `dojo_finding_group_findings` reads); existing jira push behavior unchanged.
  • index: schema regression test asserting the index exists; `makemigrations --check` clean.

🤖 Generated with Claude Code

Maffooch and others added 2 commits June 30, 2026 23:29
Fixes DJANGO-42P8.

push_finding_group_to_jira loaded the group without prefetching its findings,
so the JIRA helpers it fans out to (jira_description, jira_priority,
get_sla_deadline, get_labels, get_tags, ...) each re-ran
finding_group.findings.all(), producing an N+1 on dojo_finding_group_findings.
Load the group with prefetch_related('findings') so those reads hit the cache.

Guarded by a query-count regression test.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lookup

Fixes DJANGO-D2M.

Filtering findings by product name issues WHERE UPPER(dojo_product.name) =
UPPER(%s), which the plain unique btree on name can't serve, causing a slow
sequential scan. Add a functional Upper(name) index (created CONCURRENTLY).

Guarded by a schema regression test asserting the index exists.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@Maffooch Maffooch requested a review from mtesauro as a code owner July 1, 2026 06:01
@github-actions github-actions Bot added New Migration Adding a new migration file. Take care when merging. unittests labels Jul 1, 2026
Fix docstring summary placement (D213), drop unused noqa directives, and move the
jira helper import to top-level (the Django test runner has apps loaded, so the
lazy import is unnecessary). No behavior change; both tests still pass.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Comment on lines +33 to +35
The real helpers read finding_group.findings.all() several times; if the
group is prefetched those reads hit the prefetch (0 queries). Without the
fix each read is a fresh dojo_finding_group_findings query (N+1).

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Does/Did it really execute a new query everytime .all() is called? Usually django caches it after the first time is what I thought?

Copy link
Copy Markdown
Contributor Author

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I was under that impression as well. Maybe it is different for m2m relationships? This performance issue was raised by sentry that also looks at SQL queries

@valentijnscholten valentijnscholten added this to the 3.1.0 milestone Jul 1, 2026
dojo_testdata.json can't load under V3_FEATURE_LOCATIONS (Endpoint model
deprecated), which failed setUpClass on the V3 CI matrix variant. Apply the
standard @versioned_fixtures decorator so it loads dojo_testdata_locations.json
under V3. Verified passing under both V3_FEATURE_LOCATIONS true and false.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

New Migration Adding a new migration file. Take care when merging. unittests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants