Hanzala

Enhancing Data Governance Is the Top Most Priority for OrganisationsPhoto by Alexander Awerin on UnsplashData Observability is all the rage in the industry right now.Vast amounts of data and endless Data Quality issues have led us to this stage. Gartner sees it in the "innovation trigger" phase and has at least 5–10 years' worth of growth left before it plateaus. This means there are plenty of untapped use cases for Data Observability.So, let's look at seven of those use cases specifically to improve Data Governance.1.…

Level up Your Data Game by Mastering These 4 SkillsPhoto by Miguel A Amutio on UnsplashA colleague recently asked how they can level up their data skills this year.Data skills can come in many forms. From Data Engineering, Analysis to Data Science & Visualisation. However, they all have a common goal: to drive business value. This means the answer to the skills-related question is more value-driven than technical.So, let's explore four ways to improve data skills this year.1. Finding the Root Cause of Data…

Modernise Your Data Quality Framework by Including These ChangesPhoto by Valentin Gautier on UnsplashData Quality (DQ) continues to be a big challenge for many organisations, especially those trying to modernise their data stack. Years of underinvestment in data programmes are now costing companies nail-biting millions of pounds in regulatory fines.But that’s the stick — what is the carrot? How can you convince your senior leadership teams about investment in DQ before the regulator waves its stick? Propose a framework…

Lessons I wish I knew about before embarking on data migration journeysPhoto by Sigmund on Unsplash"Life is like a box of chocolates; you never know what you're gonna get" — if only Tom Hanks knew about the perils of Data Migration (DM). On a late Friday afternoon, receiving an urgent ping with the dreaded "reconciliation has failed again" will get your heart racing faster than your morning cardio.If you have been involved in DM, you will know that this is one of the most complex projects undertaken by data teams. DM in…

In a rush for the modern data stack, don’t forget about legacyPhoto by Yassine Khalfalli on UnsplashI have read and written many Modern Data Stack (MDS) articles, and I am so glad that, as an industry, we are taking the challenge of Big Data with its even bigger problems head-on. But one thing we seem to be missing is understanding how we at least “keep the lights on” the Legacy Data Stack (LDS) whilst migrating to the MDS.Whilst moving to the MDS, most of your customers continue to be served by your legacy data stack. In…

Core Lessons That Will Ultimately Help You in Your Learning JourneyPhoto by Harrison Kugler on UnsplashAt the start of any new project/venture, you should reflect on the past and the lessons learned. Having delivered multiple complex data warehouses, data lakes and analytics projects, I learned a thing or two about what to do and what to avoid.The key to any successful project is the people contributing to it. But the top reasons for failure are also people, along with unrealistic timelines, changing goalposts, and tired…

Is It All Just Hype, or Are There Learnings Too?Photo by Stephen Dawson on UnsplashI remember developing a new data management strategy for a client four years ago, focusing on their existing governance, hierarchies and on-premise data stack. Although the strategy has aged like a fine wine, some elements and assumptions have been proven incorrect.The key to robust strategies is to avoid running after the next shiny object. The compounding effect of a good strategy will only materialise after the monotonous repetition of…

Steps That Will Ultimately Help You Improve the Health of Your DataPhoto by Markus Spiske on UnsplashRecently, I have been asked the question: what is even more important than the data quality? And I realised I might be providing an incomplete narrative in my Data Quality (DQ) blogs.DQ is undoubtedly an essential aspect of data; however, there are many more facets than simply improving its quality. And merely improving the data quality will not give you all the benefits. I have distilled these down to five things we ought…

Tips and Tricks in Dealing with Common Data Quality IssuesPhoto by Miikka Airikkala on UnsplashImagine having clean, good-quality data for all your analytics, machine learning and decision-making. Yes — I can’t imagine it either!The inherent characteristic of data is its quality, which will deteriorate even with the most robust controls. 100% accuracy and completeness don’t exist, which is also not the point. Instead, the point is to pick your battles and improve quality to an acceptable threshold.Let’s look at 15 common…

Improving Data Quality Is a Balancing Act between Time and CostPhoto by Photoholgic on UnsplashWhen I was running a data quality training session, an attendee asked about the frequency of Data Quality (DQ) checks. I realised it was not a straightforward answer.All data is not created equal; hence, DQ checks vary based on criticality, time, and cost. Although it would be excellent to check every attribute in your data lake or warehouse after each update or insert, such an operation is usually not feasible.For this reason,…