Got it

HCIA-Big Data | Huawei Big Data Solution

Latest reply: Mar 29, 2022 07:21:52 677 5 4 0 0

Hello, everyone!

In this post, I want to share with you the Huawei Big data solution.

Introduction to Huawei Big Data Solution

Kunpeng Big Data Solution

Huawei's secure and controllable Kunpeng Big Data Solution provides one-stop high-performance big data computing and data security capabilities. This solution aims to resolve basic problems such as data security, efficiency, and energy consumption during intelligent big data construction in the public safety industry.

Big Data Pro

This solution adopts the public cloud architecture with storage-compute decoupling, ultimate scalability, and the highest possible efficiency. The highly scalable Kunpeng computing power is used as the computing resource and Object Storage Service (OBS) that supports native multi-protocols is used as the storage pool. The resource utilization of big data clusters can be greatly improved, and the big data cost can be halved.

Drew on HUAWEI CLOUD's extensive experience, Huawei big data solution provides you with high-performance and highly-reliable infrastructure resources and AI training and inference platforms for big data services, witnessing your success in digitization and intelligentization.

Advantages of Huawei Big Data Solution

High security

  • Controllable servers and big data platforms.

  • Chip-level data encryption, ensuring data security.

High performance

  • 30% higher performance than common servers of the same level.

  • Ultimate computing power and high-concurrency application scenario optimization.

  • Support for big data clusters with 5000+ nodes.

High openness

  • Compatible with the Arm ecosystem and support for mainstream hardware and software.

  • OpenLabs for services such as software development, application migration, and compatibility certification.

advantages

HUAWEI CLOUD Big Data Services

One-stop service for data development, test, and application.

HUAWEI CLOUD Big Data Services

100% compatibility with open-source ecosystems, 3rd-party components managed as plug-ins, one-stop enterprise platform.

Storage-compute decoupling + Kunpeng optimization for better performance.

HUAWEI CLOUD MRS Overview

MRS is a HUAWEI CLOUD service that is used to deploy and manage the Hadoop system and enables one-click Hadoop cluster deployment.

MRS provides enterprise-level big data clusters on the cloud. Tenants can fully control clusters and easily run big data components such as Hadoop, Spark, HBase, Kafka, and Storm. MRS is fully compatible with open source APIs, and incorporates advantages of HUAWEI CLOUD computing and storage and big data industry experience to provide customers with a full-stack big data platform featuring high performance, low cost, flexibility, and ease-of-use. In addition, the platform can be customized based on service requirements to help enterprises quickly build a massive data processing system and discover new value points and business opportunities by analyzing and mining massive amounts of data in either real time or non-real time.

Advantages of MRS 

High performance

  • Leverages Huawei-developed CarbonData storage technology which allows one data set to apply to multiple scenarios.

  • Supports such features as multi-level indexing, dictionary encoding, pre-aggregation, dynamic partitioning, and quasi-real-time data query. This improves I/O scanning and computing performance and returns analysis results of tens of billions of data records in seconds.

  • Supports Huawei-developed enhanced scheduler Superior, which breaks the scale bottleneck of a single cluster and is capable of scheduling over 10,000 nodes in a cluster.

  • Optimizes software and hardware based on Kunpeng processors to fully release hardware computing power and achieve cost-effectiveness.

Easy O&M

  • Provides a visualized big data cluster management platform, improving O&M efficiency.

  • Supports rolling patch upgrade and provides visualized patch release information.

  • Supports one-click patch installation without manual intervention, ensuring long-term stability of user clusters.

  • Delivers high availability (HA) and real-time SMS and email notification on all nodes.

Easy O&M

High security

  • With Kerberos authentication, MRS provides role-based access control (RBAC) and sound audit functions.

  • MRS is a one-stop big data platform that allows different physical isolation modes to be set up for customers in the public resource area and dedicated resource area of HUAWEI CLOUD as well as HCS Online in the customer's equipment room.

  • A cluster supports multiple logical tenants. Permission isolation enables the computing, storage, and table resources of the cluster to be divided based on tenants.

high security

high security

Cost-effectiveness

  • Provides various computing and storage choices based on diverse cloud infrastructure.

  • Separates computing from storage, delivering low-cost massive data storage solutions.

  • Supports flexible configuration of node and disk specifications.

  • Supports on-demand scale up or down of cluster.

  • Supports temporary clusters, which are automatically deleted after job execution.

  • Supports custom policies and auto scaling of clusters, releasing idle resources on the big data platform for customers.

cost effectiveness

Application Scenarios of MRS

Offline analysis of massive volumes of data.

  • Low cost: OBS offers cost-effective storage.

  • Mass data analysis: Hive analyzes TB/PB-scale data.

  • Visualized data import and export tool: Loader exports data to Data Warehouse Service (DWS) for business intelligence (BI) analysis.

Offline analysis of massive volumes of data

Large-scale data storage.

  • Real time: Kafka accesses massive amounts of vehicle messages in real time.

  • Massive data storage: HBase stores massive volumes of data and supports data queries in milliseconds.

  • Distributed data query: Spark analyzes and queries massive volumes of data.

Large-scale data storage

Low-latency real-time data analysis.

  • Real-time data ingestion: Flume implements real-time data ingestion and provides various data collection and storage access methods.

  • Data source access: Kafka accesses data of tens of thousands of elevators and escalators in real time.

Low-latency real-time data analysis

That's all, thanks!

The post is synchronized to: HCIA-Big Data

  • x
  • convention:

Saqibaz
Created Mar 2, 2022 11:12:04

Thanks for sharing
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Apr 2, 2022 01:49:13 (0) (0)
 
NTan33
Created Mar 3, 2022 06:36:12

Good to know such an option is available.
View more
  • x
  • convention:

olive.zhao
olive.zhao Created Apr 2, 2022 01:49:07 (0) (0)
Thanks!  
little_fish
Admin Created Mar 29, 2022 07:21:52

Thanks, dear.
View more
  • x
  • convention:

Comment

You need to log in to comment to the post Login | Register
Comment

Notice: To protect the legitimate rights and interests of you, the community, and third parties, do not release content that may bring legal risks to all parties, including but are not limited to the following:
  • Politically sensitive content
  • Content concerning pornography, gambling, and drug abuse
  • Content that may disclose or infringe upon others ' commercial secrets, intellectual properties, including trade marks, copyrights, and patents, and personal privacy
Do not share your account and password with others. All operations performed using your account will be regarded as your own actions and all consequences arising therefrom will be borne by you. For details, see " User Agreement."

My Followers

Login and enjoy all the member benefits

Login

Block
Are you sure to block this user?
Users on your blacklist cannot comment on your post,cannot mention you, cannot send you private messages.
Reminder
Please bind your phone number to obtain invitation bonus.
Information Protection Guide
Thanks for using Huawei Enterprise Support Community! We will help you learn how we collect, use, store and share your personal information and the rights you have in accordance with Privacy Policy and User Agreement.