What are the steps to deploy a big data solution ?

    • There are three steps to deploy a Big Data Solution
Deploying Big Data solution

Data Ingestion

  • The first step for deploying a big data solution is the data ingestion i.e. extraction of data from various sources.
  • The data source may be a CRM like Salesforce, Enterprise Resource Planning System like SAP, RDBMS like MySQL or any other log files, documents, social media feeds etc.
  • The data can be ingested either through batch jobs or real-time streaming. The extracted data is then stored in HDFS.
Data Ingestion in Big Data

Data Storage

  • After the data ingestion, the next step is to store the extracted data. The data either be stored in HDFS or NoSQL database (i.e. HBase).
  • The HDFS storage works well for sequential access whereas HBase for random read/write access.
Data Storage

Data Processing

  • The final step in deploying a big data solution is the data processing.
  • The data is processed through one of the processing frameworks like Spark, MapReduce, Pig, etc.
Data Processing in Big Data

Categorized in:

Big Data

Tagged in:

, , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , , ,