Adding Golden data to avoid missing reference failure#2085
Adding Golden data to avoid missing reference failure#2085pravnkumar-cloudsufi wants to merge 1 commit into
Conversation
There was a problem hiding this comment.
Code Review
This pull request introduces a validation configuration and golden summary data for the US USDA QuickStats script. Specifically, it adds a new golden summary report CSV, references the validation configuration file in the manifest, and defines validation rules for deleted records and golden data checks. However, the relative path specified for the golden file in validation_config.json is incorrect and would lead to a file-not-found error; it should be updated to point directly to golden_data/golden_summary_report.csv.
|
Hello! Adding golden data for validation is a great practice for maintaining the integrity of your data pipeline and preventing regressions. This approach ensures that any unexpected changes in the summary report output are caught early. Your implementation in the manifest and the validation configuration follows the standard patterns used in this repository. If you have any specific questions about the validation logic or need further assistance, feel free to ask! |
PR Checklist: https://docs.google.com/spreadsheets/d/1QB65wmyxUqEK6dPQILe7MQ57mr0sTgs8-AUH1hXAvFk/edit?gid=268834378#gid=268834378
Import Name: USDA_AgricultureSurvey