Strategies and Technologies for Effective Data Quality Management

The data quality challenge is not new. “There has always been a problem with data: It is only as good as the last time it was checked,” says Industry 4.0 expert Jane Arnold, COO at APERIO and an independent board director at Houston AI Institute. “Assuming existing data is ‘good enough’ is a mistake. I see data quality as the foundation of trust in digital systems.”

The AI hype has some senior executives expecting it will magically solve their data quality problems, but what AI actually does is expose the data quality flaws at scale, she explains. 

Capitalizing on AI requires solving the underlying data quality problem first, but companies may not realize they have a problem because of the massive scale of data involved. “If your industrial data systems have 7 million tags, how can you possibly know they're all correct? Only about 45-80% of the time-series data our customers bring in is good data,” Arnold observes.

“As AI becomes embedded in operational decisions, data quality becomes a governance issue because AI won’t work if there are problems with the underlying data,” says Arnold. “It is also a risk management issue because industrial data quality is a leading indicator of operational risk, and AI without validated data introduces systemic exposure.”

AI-driven data quality improvement solutions are fast emerging 

Purpose-built software platforms and tools designed to tackle the data quality challenge are continuously developing and improving, including the following four examples:

• Anomalo’s Data Quality Monitoring is an AI-based software that automatically detects and alerts teams to quality issues in structured, semi-structured and unstructured enterprise data. The industry-tailored augmented data quality solution continuously monitors data patterns and deviations from the norm, enabling informed mitigation. An agentic interface, Anomalo’s Intelligent Data Analyst (AIDA), supports interaction with the data using natural language.
• Databricks’ Data Quality Monitoring, integrated with the Databricks Data Intelligence Platform, offers continuous monitoring of the data estate at scale with Agentic AI. Anomaly detection monitors all critical tables in a schema with one click, learning patterns and seasonal behavior to identify issues. At the table level, data profiling captures and tracks summary statistics for historical context. Root causes are surfaced, and issues are prioritized so teams can ensure timely resolution.
• DataWise from APERIO is specifically built to continuously validate industrial data streams across the enterprise at scale, to detect signal degradation and quality and to quantify the economic impact before AI systems consume flawed inputs. The software can identify, prioritize and help teams resolve operational time-series data quality problems such as bad, missing or stale data based on root cause analysis findings and proposed remediation steps.
• Stibo Systems MDM is a master data management platform that provides tools for data quality validation, including automatic data profiling and data quality metrics; automatic data cleansing and enrichment; continuous data monitoring and validation; and a framework for exposing and extracting data to streamline auditing and facilitate root cause analysis. 

Seek out expert recommendations when developing your data quality management strategy 

Start now on your journey to data quality improvement by incorporating advice from specialists. Your digital transformation and AI success depend on these universal essentials — especially the need to keep humans in the equation.

From Georgi Chonkov:

1. Define measurements for good quality data. Is the data you're using complete? Is it unique? Is it up to date? Is it accurate? Is it consistent?
2. Keep reference data up to date and establish clear responsibilities for maintaining it. Properly managing internal and external reference data — the fixed (not changing) data used to classify or categorize master data, such as product codes or measurement units — can reduce errors substantially.
3. Define clear error management processes. An effective data error management strategy involves dividing responsibilities, defining bug reporting procedures, selecting appropriate tools, classifying errors and prioritizing urgent issues.

From Jane Arnold:

1. Industrials need to treat data quality the same way they treat safety or compliance, where it is measurable and non-negotiable, because of its importance to uptime, analytics and everything you do after collecting the data. Those ignoring the fact that you have to deal with data quality first will face a lot of wasted money and confusion.
2. Regardless of the tool used, the ability to consistently identify and correct data quality problems, from the largest to the smallest, is an imperative for continuous improvement.
3. With AI, you still need to keep the human in the loop; otherwise, there will be no living, breathing, thinking being reviewing the data and saying, "Yes, this is real," or "No, this is a hallucination," or "No, this is absolutely ridiculous, and I'm not going to do that."