ai_data_detection
Differences
This shows you the differences between two versions of the page.
| Both sides previous revisionPrevious revision | |||
| ai_data_detection [2025/05/25 15:07] – [Extending the Data Detection] eagleeyenebula | ai_data_detection [2025/05/25 15:09] (current) – [Best Practices] eagleeyenebula | ||
|---|---|---|---|
| Line 233: | Line 233: | ||
| ===== Best Practices ===== | ===== Best Practices ===== | ||
| 1. **Use Incremental Checks:** Perform quality checks at different stages of the pipeline (e.g., after loading raw data and after preprocessing steps). | 1. **Use Incremental Checks:** Perform quality checks at different stages of the pipeline (e.g., after loading raw data and after preprocessing steps). | ||
| + | |||
| 2. **Automate Logging:** Set up centralized logging for tracking data issues across multiple datasets. | 2. **Automate Logging:** Set up centralized logging for tracking data issues across multiple datasets. | ||
| + | |||
| 3. **Adapt Custom Methods:** Extend the module for domain-specific checks, such as outlier detection, range checks, or invalid category detection. | 3. **Adapt Custom Methods:** Extend the module for domain-specific checks, such as outlier detection, range checks, or invalid category detection. | ||
| + | |||
| 4. **Handle Issues Early:** Address identified data issues before training machine learning models. | 4. **Handle Issues Early:** Address identified data issues before training machine learning models. | ||
ai_data_detection.1748185640.txt.gz · Last modified: 2025/05/25 15:07 by eagleeyenebula