User Tools

Site Tools


ai_data_monitoring_reporing

Differences

This shows you the differences between two versions of the page.

Link to this comparison view

Both sides previous revisionPrevious revision
Next revision
Previous revision
ai_data_monitoring_reporing [2025/05/25 16:47] – [How It Works] eagleeyenebulaai_data_monitoring_reporing [2025/05/25 16:50] (current) – [1. Monitoring Data Quality] eagleeyenebula
Line 49: Line 49:
  
   * **Data Monitoring Tools:**     * **Data Monitoring Tools:**  
-    Detect missing values and calculate dataset coverage (% completeness).+    1. Detect missing values and calculate dataset coverage (% completeness).
  
   * **Flexible Report Generation:**     * **Flexible Report Generation:**  
-    Automated string-based summary reports for processed datasets or workflows.+    2. Automated string-based summary reports for processed datasets or workflows.
  
   * **Detailed Logging:**     * **Detailed Logging:**  
-    Logs all actions, including data quality checks and report generation results, for thorough traceability.+    3. Logs all actions, including data quality checks and report generation results, for thorough traceability.
  
   * **Integration-Ready:**     * **Integration-Ready:**  
-    Easily integrates into existing pipelines as a monitoring or reporting component.+    4. Easily integrates into existing pipelines as a monitoring or reporting component.
  
   * **Customizable Reporting Templates:**     * **Customizable Reporting Templates:**  
-    Can be extended to generate reports in various formats like JSON, HTML, or Markdown.+    5. Can be extended to generate reports in various formats like JSON, HTML, or Markdown.
  
 ---- ----
Line 75: Line 75:
     Generates a textual summary of the processed dataset.     Generates a textual summary of the processed dataset.
  
-The workflow is as follows:+**The workflow is as follows:**
  
-  Pass data into **monitor_data_quality** to receive a structured dictionary containing monitored results   +  Pass data into **monitor_data_quality** to receive a structured dictionary containing monitored results  (e.g., missing value count, coverage percentage). 
-    (e.g., missing value count, coverage percentage). +  Use **generate_report** to create a human-readable string report based on the findings or processed data.
-  Use **generate_report** to create a human-readable string report based on the findings or processed data.+
  
 ==== 1. Monitoring Data Quality ==== ==== 1. Monitoring Data Quality ====
Line 85: Line 84:
   * **Missing Data:** Identifies **None** or **NaN** values in the dataset.   * **Missing Data:** Identifies **None** or **NaN** values in the dataset.
   * **Total Data Points:** Counts the overall size of the dataset.   * **Total Data Points:** Counts the overall size of the dataset.
-  * **Coverage Percentage:** Calculates the completeness of the dataset as **(Total Values - Missing Values) / Total Values 100**.+  * **Coverage Percentage:** Calculates the completeness of the dataset as **(Total Values - Missing Values) / Total Values 100**.
  
 The output is a dictionary summarizing quality statistics: The output is a dictionary summarizing quality statistics:
ai_data_monitoring_reporing.1748191669.txt.gz · Last modified: 2025/05/25 16:47 by eagleeyenebula