Sunday 26 April 2015

Working of Hive

The following diagram depicts the workflow between Hive and Hadoop.
How Hive Works
The following table defines how Hive interacts with Hadoop framework:
Step No.Operation
1Execute Query
The Hive interface such as Command Line or Web UI sends query to Driver (any database driver such as JDBC, ODBC, etc.) to execute.
2Get Plan
The driver takes the help of query compiler that parses the query to check the syntax and query plan or the requirement of query.
3Get Metadata
The compiler sends metadata request to Metastore (any database).
4Send Metadata
Metastore sends metadata as a response to the compiler.
5Send Plan
The compiler checks the requirement and resends the plan to the driver. Up to here, the parsing and compiling of a query is complete.
6Execute Plan
The driver sends the execute plan to the execution engine.
7Execute Job
Internally, the process of execution job is a MapReduce job. The execution engine sends the job to JobTracker, which is in Name node and it assigns this job to TaskTracker, which is in Data node. Here, the query executes MapReduce job.
7.1Metadata Ops
Meanwhile in execution, the execution engine can execute metadata operations with Metastore.
8Fetch Result
The execution engine receives the results from Data nodes.
9Send Results
The execution engine sends those resultant values to the driver.
10Send Results
The driver sends the results to Hive Interfaces.

No comments:

Post a Comment

 A good reference for Shell scripting  https://linuxcommand.org/lc3_writing_shell_scripts.php