Processing in Big Data

Types of Processing in Big Data


1. Batch Processing:

Batch Processing System is an efficient way of processing large volumes of data. where a group of transactions is collected over a period of time. Data is collected, entered, processed and then the batch results are produced. The main function of a batch processing system is to automatically keep executing the jobs in a batch. This is the important task of a batch processing system i.e. performed by the ‘Batch Monitor’ resided in the low end of main memory.

2. Real Time Processing:

Real Time Processing systems are very fast and quick respondent systems. These systems are used in an environment where a large number of events (generally external) must be accepted and processed in a short time. Real time processing requires quick transaction and characterized by supplying immediate response. For example, defense application systems like as RADAR, etc.


Following are the differences between Batch Processing System and Real Time Processing System:-



Comments

Post a Comment

If you have any doubts, Please let me know

Popular posts from this blog

Hadoop

Image
What is Hadoop ? Hadoop is a framework that allows us to store and process large data sets in parallel and distributed fashion. It was developed by Doug Cutting in 2005, was working in yahoo. It efficiently store and process large datasets ranging in size from gigabytes to petabytes of data. Instead of using one large computer to store and process the data, Hadoop allows clustering multiple computers to analyze massive datasets in parallel more quickly. Hadoop is used for Batch processing where, f irst data is collected, entered and then processed. Later, it produces batch result. It uses master-slave architecture where one master and multiple slave works in a cluster.. Hadoop uses commodity hardware. So, it is cost effective. It is scalable as per requirement. Hadoop deals with flat files in any format. Hadoop follows RPC communication model. By default, Communication happens in every 3 seconds where slave communicates to master. There are three components of Hadoop: HDFS  - Hado...
Read more

Q/A (Big Data) 2

Q - Hive Stores metadata in which type of database? A - MySQL Q - Why to analyze data? A - (i) Exponential growth in a machine data over the last years.      (ii) Growing no of machine and usage of IOT devices. Q - From where, large data is generated? Sensor data Machine data Business data Q - What is communication model in hadoop? Hadoop follows RPC communication. Communication happens in every 3 sec (by default). Slaves communicate to master. Q - What are important configuration files in hadoop? Hdfs-site.xml – Block replication Core-site.xml – I/O settings , setting of Hadoop cluster Hadoop-env.sh – Environment setup Master Slave Mapred-site.sh – For setting a map reduce Q - Big Data as an opportunity? Cost Reduction - Cost effective storage system for huge data sets. Next generation products - Automated car, Health care. Faster and better decision making - Provides ways to analyze information quickly and make decisions. Improved services or products - Evaluation of cu...
Read more

Linux Commands

  Basic Linux Commands ls                                  -                            List of files lsr                                    -                         List of files recursively mkdir                               -                         Make directory rmdir                                -                         Remove directory touchz               ...
Read more