It’s a boring topic! Really dry, and the basic premise hasn’t changed in decades: rules, rules, rules, and more rules! So why are we at Talenode tackling something that’s not only dull but also unlikely to grab eyeballs?
Because it’s a topic that governs so much of HR tech, and there is no solution in the market that adequately solves for it!
Ask anyone who works with HR data, and they’ll tell you about their love-hate relationship with it. Data cleaning teaches a lot but is also a source of ongoing frustration. Updating, correcting, replacing, and endlessly reworking data can make anyone boil over. Yet, when that pristine, clean dataset is finally ready, it feels like something worth protecting. Unfortunately, as soon as it’s out in the real world, it gets corrupted, changed, and modified all over again. In weeks or months, it’s often unrecognizable.
Our Journey with Data Cleaning
Ankit Abrol and I have spent years navigating the highs and lows of data cleaning. From manually managing data with Excel macros and formulas during consulting projects to addressing the never-ending chaos of line HR operations, we’ve seen it all.
In consulting, cleaning data for a specific project is still manageable; there is a clear start and endpoint. But in HR operations, clean data is a constant battle! Major re-orgs, RIFs (reductions in force), M&A activity, or even unforeseen events such as COVID-19 can disrupt the system overnight.
We’ve tried countless approaches to manage this chaos, from point-in-time fixes to time-over-time data strategies. Some solutions worked well for event-based management, while others helped us apply rules faster. Over time, we realized that keeping data updated in real-time across multiple systems is an uphill climb, especially when resources, experience, and tech support are limited. No single solution addressed everything, so we had to evolve.
HRMS: The Source of Truth and Challenges
To illustrate the complexities of data cleaning, let’s look at HRMS platforms such as SuccessFactors and Workday, which serve as the source of truth for employee information. Even in a single system, there are a ton of challenges. For simplicity, this article will focus on point-in-time data across an HRMS (future articles will dive into time-over-time and multi-system data).
Here are the key questions we’ve faced—and how we’re tackling them:
1. What Columns to Review?
Not everyone needs to see every column of data:
- HRBPs focus on job relationships (manager, leader tagging) and org structure.
- Total Rewards teams care about compensation and benefits data.
- HR Operations review lifecycle data (onboarding, offboarding) and personal data.
Our Approach: Customize error visibility by stakeholder role so each group sees only what’s relevant to them.
2. What Rows to Review?
Some roles require access only to their specific data, while others need a hierarchy-based view.
Our Approach: Enable tailored access to rows based on role and scope of responsibility.
3. What Are the Errors?
Users often don’t know all the rules applied to their data, making it hard to identify and fix errors. A few common issues include:
- Duplicates and blanks: Which fields can have blanks, and where are duplicates prohibited?
- Drop-down lists: Are department names and locations consistent? Why are “Senior” and “Sr.” tagged inconsistently for the same designation?
- Associations: Are SBUs correctly linked to BUs? Is the grade-to-level mapping accurate?
Our Approach: A centralized, role-specific repository of rules for easy reference helps simplify error identification and correction.
4. How to Review and Correct Errors?
Even when errors are identified, fixing them can feel like a monumental task.
Our Approach:
- Error Count Dashboards to give users clear visibility into error metrics.
- Correction Prompts to suggest fixes, so users don’t have to remember all possible values.
- Exceptions to allow users to flag rule deviations and document why they apply.
- Bulk Updates to correct multiple records simultaneously (e.g., BU-SBU tagging).
These challenges only scratch the surface. We are yet to discuss the hassles faced by platform administrators or the complexities of aligning data across systems and time periods.
Why It Matters
Data cleaning often takes a back seat to analytics and reporting. It’s treated as just another “feature” rather than a critical component of decision-making. But without clean data, analytics lose their value. Compliance, operational efficiency, and strategic insights all hinge on the accuracy and integrity of the underlying data.
That’s why Talenode is focused on building HR’s own data quality platform. Our MVP tackles these challenges head-on, simplifying the process and helping organizations maintain clean, reliable data over the long haul.