Got it

HCIA-Big Data | Introduction to Big data era

Latest reply: Jul 27, 2022 16:30:12 685 21 8 0 0

Hello, everyone!

In this post, I will describe what is big data and big data 4V's.

Big Data Era

Ushering in the 4th Industrial Revolution and Embracing the Intelligent Era


Ushering in the 4th Industrial Revolution and Embr


In the 1760s, the UK took the lead in launching the first industrial revolution bringing the world into the Age of Steam. As a result, the UK became the first industrialized country in the world and brought a series of social changes.

In the late 1860s, the second industrial revolution began. With the large-scale application of AC power, we entered the Age of Electricity.

The third industrial revolution is marked by the invention and application of atomic energy electronic computers space technology and bioengineering. It is a revolution of science and technology, involving many fields such as information technology, new energy technology, new material technology, biotechnology space technology, and marine technology. The United States has gradually gained its power during the third industrial revolution also known as the information revolution.

The fourth revolution of science and technology is unfolding with the convergence of new IT technologies such as cloud computing, big data, IoT, and AI. Those who can respond to the call of the era will become the trendsetter. 

Moving From Data Management to Data Operations


Moving From Data Management to Data Operations


In addition to the industrial revolution, operators need to transform, their mindsets to adapt to the digital and information era. Under such circumstances, we need to move from data management to data operations. Here we can see three sentences.

1. Data drive experience

Have you heard of the recommendation system? Actually, this is how Amazon makes money.

For example, I have been searching for cookers for a while on Amazon and next time I log in to it I would be most likely advertised with some frying pans or even some teapots right. So just like this simple example enterprises like Amazon or YouTube, label the users based on their behaviors and features and all the behaviors and features are nothing but data.

2. Data Drives Decision-Making

There is another example, as is known to all there are many oil fields in the Middle East. But how do they determine where is proper for digging oils. Again based on the data, but now they mainly rely on the historical data, like data for the past 50 years, and based on some technical methods like data mining and data visualization, we can make decisions more easily and more convincingly.

3. Data Drives Processes

We are currently in the big data era and every process involves many times of data collisions. Like we pay for our lunch by credit cards or by mobile phones, or like transferring money to our friends in the bank, almost everything now requires data flowing efficiently, and also we can use big data methods to helps us simplify all the processes as well.

Everything Is Data and Data Is Everything



Everything Is Data and Data Is Everything


Definition on Wikipedia:

  • Big data refers to data sets with sizes beyond the ability of commonly used software tools to capture, curate, manage, and process data within a tolerable elapsed time.


Definition on Wikipedia:


1. Volume

The volume of big data must be enormous. Does anyone know the units of the data measurement? Commonly we will use Byte, KB, MB, GB, TB, PB, EB, ZB, etc to describe the data volume right. And in general, we call the data as big data when the volume is about TB. And to give you a direct impression there is an example. Basically, an SD movie lasting for 1.5 hours is about 1GB, so if you download more than 1000 movies at the same time, you can say you are dealing with big data.

2. Variety

In the big data era, we need to process data of different types. In general, the data can be divided into 3 types structured data, semi-structured data, and unstructured data. 

Structured data usually resides in relational databases as 2-demensional tables. And in the databases, we have 'fields' that store length-delineated data like phone numbers social security numbers or ZIP codes, etc. 

Unstructured data includes pictures videos audio etc. Or namely unstructured data either does not have a pre-defined data model or is not organized in a pre-defined manner. 

Semi-structured data mostly refers to the web page files in HTML, XML, YAML, or JSON format.

3. Velocity

From the result of the research, the data we generated in the recent 5 years is much more than the data generated in the past 5000 years. So the explosive growth of data requires us to increase the velocity on dealing with this information.

4. Value

All of the data is valuable according to different applications, but the volume of data is huge. Maybe only a small part of it is useful for handling a specific problem. To some extent the original data is like fossil oil which needs to be extracted or processed just like the oil should be refined from petroleum. As we all know fossil oil is vital in our society but it just consists of a very small proportion of the raw petroleum. 

So we can use the four keywords to describe big data very much. And this is the most widely accepted definition of big data.

Big Data Processing VS Traditional Data Processing

From database to big data

  • "Fishing in the pond" VS "Fishing in the ocean”("fish" indicates the data to be processed).


Big Data Processing

Traditional Data Processing

Data scale

Large (in GB, TB, or PB)

Small (in MB)

Data type

Various data types (structured, semi-structured, and non-structured data)

Single data type (mostly structured data)

Relationship between mode and data

Modes are set after data is generated. Modes evolve when data increases.

Modes are set before data is generated.

Object to be processed


“Fish in the ocean”. "Some fishes" are used to determine whether other types of fish exist.

"Fish in the pond"

Processing tool

No size fits all.

One size fits all.

That's all, thanks!


  • x
  • convention:

user_4358465
Created Feb 18, 2022 05:26:53

Thanks for this detailed and well written post!
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Feb 21, 2022 01:24:42 (0) (0)
 
Saqibaz
Saqibaz Created Mar 27, 2022 15:49:45 (0) (0)
 
little_fish
Admin Created Feb 24, 2022 07:06:23

Cool
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Feb 24, 2022 07:27:24 (0) (0)
 
Saqibaz
Saqibaz Created Mar 27, 2022 15:49:59 (0) (0)
 
Saqibaz
Created Mar 27, 2022 15:49:33

Thanks for sharing
View more
  • x
  • convention:

Ayeshaali
Ayeshaali Created Mar 27, 2022 17:57:26 (0) (0)
 
user_3915171
Created Mar 27, 2022 17:54:58

thanks
View more
  • x
  • convention:

Ayeshaali
Created Mar 27, 2022 17:57:34

Thanks for Sharing
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Mar 28, 2022 09:07:45 (0) (0)
 
NTan33
Created Mar 28, 2022 08:46:04

A most fascinating concept.
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Mar 28, 2022 09:07:32 (0) (0)
Thanks!  
VinceD
Created May 6, 2022 05:35:16

interesting content.
View more
  • x
  • convention:

VinceD
VinceD Created May 6, 2022 05:35:58 (0) (0)
 
MahMush
Author Created May 6, 2022 10:16:59

The four dimensions Volume, Variety, Variation, and Visibility are used to categorize the main aspects of the processes that transform resources into outputs.
View more
  • x
  • convention:

RNT
Created May 6, 2022 10:24:50

Useful, thanks
View more
  • x
  • convention:

12
Back to list

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.