Got it

Difficulties and concerns with Big data quality

Latest reply: Mar 5, 2022 13:23:15 1398 105 50 0 0

bdq



Differences in data preparation


To standardize and purify data for new use cases, a number of data preparation approaches are frequently required. This is a manual, monotonous, and time-consuming job. When data prep teams working with data from separate silos calculate similar sounding data items in different ways, data quality issues might develop. 


For instance, one team may compute overall customer revenues by deducting returns from sales, but another team may calculate it solely on the basis of sales. As a result, metrics in separate data pipelines are inaccurate.



Bringing diverse data taxonomies together


Business units inside a corporation or merged companies may have developed and fine-tuned their respective data taxonomies and semantics to represent how they operate. Private equity investments, for instance, can hasten the speed of acquisitions and mergers, resulting in the consolidation of several businesses into a single giant corporation. 


Each acquired company's CRM, marketing automation, marketing content management, customer database, and lead qualifying technique data was often unique. Integrating various technologies into a single data structure in order to manage unified campaigns poses significant hurdles in terms of big data quality.



Consistency is important


Cleaning, verifying, and normalizing data can all pose problems with big data quality. For instance, one telephone firm developed models that connected network failure data, outage reports, and consumer complaints to see if problems could be traced to a specific place. However, some of the addresses were inconsistent, with "456 Second Street" appearing in one system and "456 2ND STREET WEST" appearing in another.



There isn't a plan in place to govern data


Lack of data governance and communication can result in a slew of quality difficulties. A solid data governance program that defines, manages, and communicates data policies, guidelines, and standards for successful data utilization and to build data literacy should be backed up by a big data quality plan. The principles and specifics of the data are understood and respected by the data community once it has been isolated from its source contexts.



Obtaining the ideal equilibrium


There seems to be a natural conflict between the desire to acquire all accessible data and the need to ensure that the data collected is of the greatest quality. 


It's also crucial to comprehend the goal of gathering certain data, the techniques utilized to gather big data, and the organization's intended downstream analytics applications. Proprietary practices can often develop that are subject to mistakes, unstable, and unrepeatable.



Excessive data collection


Data management teams can become preoccupied on gathering ever-increasing amounts of data. However, more isn't always the best strategy. The more collected data, the higher the danger of data inaccuracies. 


Before training the data model, irrelevant or faulty data must be removed, however even cleaning approaches can have a negative impact on the outcome.


  • x
  • convention:

faysalji
Moderator Author Created Feb 8, 2022 05:16:33

Thanks for sharing
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:39:55 (0) (0)
Thank you for taking the time to review this post.  
zaheernew
MVE Author Created Feb 8, 2022 06:08:13

Useful info
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:02 (0) (0)
Thank you for taking the time to review this post.  
Unicef
MVE Created Feb 8, 2022 06:24:33

Very good and helpful post
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:08 (0) (0)
Thank you for taking the time to review this post.  
SamB
Moderator Created Feb 8, 2022 10:00:00

thank you for sharing
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:14 (0) (0)
Thank you for taking the time to review this post.  
amr_rashedy
MVE Author Created Feb 8, 2022 12:08:23

Thanks for sharing
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:21 (0) (0)
Thank you for taking the time to review this post.  
MahMush
Moderator Author Created Feb 8, 2022 14:16:41

Very detailed information, well written 

View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:28 (0) (0)
Thank you for taking the time to review this post.  
MahMush
Moderator Author Created Feb 8, 2022 14:27:04

completely amazing

View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:35 (0) (0)
Thank you for taking the time to review this post.  
Anno7
Moderator Author Created Feb 9, 2022 04:36:40

well done
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:42 (0) (0)
Thank you for taking the time to review this post.  
Vien
Created Feb 9, 2022 04:59:57

Good share
View more
  • x
  • convention:

user_4358465
user_4358465 Created Feb 27, 2022 04:40:49 (0) (0)
Thank you for taking the time to review this post.  

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.