Skip to content

Implemented Golden EurostatData_Education_Attainment#2086

Open
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:Implemented_goldens_education_attainment
Open

Implemented Golden EurostatData_Education_Attainment#2086
niveditasing wants to merge 2 commits into
datacommonsorg:masterfrom
niveditasing:Implemented_goldens_education_attainment

Conversation

@niveditasing

@niveditasing niveditasing commented Jul 7, 2026

Copy link
Copy Markdown
Contributor

This import has recurring deletion failures less than 0.01%. To tackle these small deletions, the threshold has been increased, but we also added two new golden checks. These validation checks will verify that the data contains all sets in a golden set, these are defined within the validation_config.json file, which is situated in the same directory as manifest.json. The threshold has been raised to 0.05, a value determined by historical and current deletion percentages, with a small buffer to account for these minor deletions.

PR checklist: https://docs.google.com/spreadsheets/d/1BzweR9Sj58j0H2_BweGTmfE4Z1lrjPZL8u1FS0kzCeg/edit?pli=1&gid=0#gid=0

@gemini-code-assist gemini-code-assist Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request introduces a golden summary report and configures validation rules for the Eurostat regional education attainment statistics. The feedback highlights an incorrect relative path for the golden data file in the validation configuration, which needs to be corrected to prevent validation failures.

"rule_id": "check_goldens_summary_report",
"validator": "GOLDENS_CHECK",
"params": {
"golden_files": "../../../../golden_data/golden_summary_report.csv"

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

high

The relative path specified for golden_files is incorrect. The golden file golden_summary_report.csv is located in the golden_data directory within the same folder as validation_config.json (i.e., scripts/eurostat/regional_statistics_by_nuts/education_attainment/golden_data/golden_summary_report.csv).

Using ../../../../golden_data/golden_summary_report.csv resolves to the root of the repository, which will cause the validation check to fail because the file cannot be found. It should be updated to golden_data/golden_summary_report.csv.

Suggested change
"golden_files": "../../../../golden_data/golden_summary_report.csv"
"golden_files": "golden_data/golden_summary_report.csv"

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant