Quality Score
Anomalies Detected
Schema Changes
Records Processed
Z-score threshold exceeded (4.2)
Unusual spike in transaction amounts detected in the last 24 hours. Values exceed 3 standard deviations from the mean.
Uniqueness constraint failed
3 duplicate customer_id values found in recent transactions. This violates the primary key constraint.
Added field: customer_segment (STRING)
Unusual pattern in transaction_fee column
All 24 data quality rules passed successfully
| transaction_amount | transaction_fee | customer_age | items_count | |
|---|---|---|---|---|
| transaction_amount | 1.00 | 0.78 | 0.12 | 0.92 |
| transaction_fee | 0.78 | 1.00 | 0.08 | 0.65 |
| customer_age | 0.12 | 0.08 | 1.00 | 0.21 |
| items_count | 0.92 | 0.65 | 0.21 | 1.00 |
Values detected that exceed 3 standard deviations from the mean.
Values detected that fall outside 1.5 times the interquartile range.
Sudden change detected in the number of daily transactions.
| Field | Type | Change | Date | Transaction |
|---|---|---|---|---|
| customer_segment | STRING | Added | May 18, 2025 | 00000128ab42f9 |
| transaction_fee | DECIMAL(10,2) | Modified | May 15, 2025 | 00000127de31c8 |
| promotion_code | STRING | Removed | May 10, 2025 | 00000126fc29b7 |
| payment_method | STRING | Added | May 5, 2025 | 00000125ba18a6 |
| customer_location | STRUCT | Modified | May 1, 2025 | 00000124a907b5 |
| Field | Type | Nullable | Description |
|---|---|---|---|
| transaction_id | STRING | No | Unique identifier for the transaction |
| customer_id | STRING | No | Customer identifier |
| transaction_date | TIMESTAMP | No | Date and time of the transaction |
| transaction_amount | DECIMAL(10,2) | No | Amount of the transaction |
| transaction_fee | DECIMAL(10,2) | Yes | Fee associated with the transaction |
| product_id | STRING | No | Product identifier |
| store_id | STRING | No | Store identifier |
| payment_method | STRING | Yes | Method of payment |
| customer_segment | STRING | Yes | Customer segment classification |
| customer_location | STRUCT | Yes | Customer location information |
-- SQL Transformation
SELECT
rt.transaction_id,
rt.customer_id,
rt.transaction_date,
rt.transaction_amount,
rt.transaction_fee,
rt.product_id,
rt.store_id,
rt.payment_method,
cd.customer_segment,
cd.customer_location
FROM
raw_transactions rt
JOIN
customer_data cd ON rt.customer_id = cd.customer_id
WHERE
rt.transaction_date >= current_date() - INTERVAL 30 DAYS-- SQL Transformation
SELECT
store_id,
DATE(transaction_date) as transaction_day,
COUNT(*) as daily_transactions,
SUM(transaction_amount) as daily_amount,
AVG(transaction_amount) as avg_transaction_value,
MIN(transaction_amount) as min_transaction_value,
MAX(transaction_amount) as max_transaction_value
FROM
customer_transactions
GROUP BY
store_id, DATE(transaction_date)
ORDER BY
store_id, transaction_dayThe following downstream tables and processes are affected by this schema change:
This table references the new column and requires updates to its transformation logic.
This table does not reference the new column and requires no changes.
This report depends on the new column and requires updates to its visualization logic.
Passed Rules
Warning Rules
Failed Rules
Total Rules
| Status | Rule Name | Type | Description | Last Run | Actions |
|---|---|---|---|---|---|
| unique_customer_id | SQL | Customer ID must be unique across all transactions | May 20, 2025 | ||
| transaction_amount_range | Python | Transaction amount should be between $0.01 and $10,000.00 | May 20, 2025 | ||
| not_null_transaction_id | YAML | Transaction ID must not be null | May 20, 2025 | ||
| valid_transaction_date | SQL | Transaction date must not be in the future | May 20, 2025 |
-- SQL Rule Definition
SELECT
customer_id,
COUNT(*) as occurrence_count
FROM
customer_transactions
GROUP BY
customer_id
HAVING
COUNT(*) > 13 records found that violate this rule:
| customer_id | occurrence_count |
|---|---|
| CID-78342 | 12 |
| CID-65219 | 3 |
| CID-91087 | 2 |