apache hive - Hbase Apache Hive- hive tutorial - hadoop hive - hadoop hive - hiveql

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As ETL Data Target - Hive to Hbase :

learn hive - hive tutorial - apache hive - data from hive to hbase - hive examples

HBase As Data Source - Hbase to Hive :

learn hive - hive tutorial - apache hive - data from hbase to hive - hive examples

HBase As Data Source - Hbase to Hive - Datawarehouse :

learn hive - hive tutorial - apache hive - data from hbase to hive - datawarehouse - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Hbase architecture :

learn hive - hive tutorial - apache hive - data from hbase to hive - hbase architecture - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Hive architecture :

learn hive - hive tutorial - apache hive - data from hbase to hive - hive architecture - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Hbase and Hive architecture :

learn hive - hive tutorial - apache hive - data from hbase to hive - hbase and hive architecture - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Hbase map reduce job for insert :

learn hive - hive tutorial - apache hive - data from hbase to hive - hbase map reduce job for insert - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Map-Only Job for INSERT :

learn hive - hive tutorial - apache hive - data from hbase to hive - hbase map only job for insert - hive examples

HBase As Data Source - Hbase to Hive - Query processing in Hbase :

Example Query : SELECT name, notes FROM users WHERE userid=‘xyz’;

Rows are read from HBase via

org.apache.hadoop.hbase.mapred.TableInputFormatBase

HBase determines the splits (one per table region)

HBaseSerDe produces lazy rows/maps for RowResults

Column selection is pushed down

Any SQL can be used (join, aggregation, union…)

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Meta Store Integration in Hbase :

DDL can be used to create metadata in Hive and HBase simultaneously and consistently

CREATE EXTERNAL TABLE: register existing Hbase table

DROP TABLE: will drop HBase table too unless it was created as EXTERNAL

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Bulk Load in Hbase :

Example Query : SET hive.hbase.bulk=true;
                 INSERT OVERWRITE TABLE users SELECT … ;

• But for now, you have to do some work and issue multiple Hive commands
  • Sample source data for range partitioning
  • Save sampling results to a file
  • Run CLUSTER BY query using HiveHFileOutputFormat and TotalOrderPartitioner (sorts data, producing a large number of region files)
  • Import HFiles into HBase
  • HBase can merge files if necessary

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Hbase Hive range partitioning during load :

learn hive - hive tutorial - apache hive - data from hbase to hive - hbase hive range partitioning during load - hive examples

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Sampling Query For Range Partitioning :

Given 5 million users in a table bucketed into 1000 buckets of 5000 users each, pick 9 user_ids which partition the set of all user_ids into 10 nearly-equal-sized ranges.

    select user_id from
          (select user_id
          from hive_user_table
          tablesample(bucket 1 out of 1000 on user_id) s
          order by user_id) sorted_user_5k_sample
          where (row_sequence() % 501)=0;

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

HBase As Data Source - Hbase to Hive - Sorting Query For Bulk Load :

•           set mapred.reduce.tasks=12;
            set hive.mapred.partitioner=
            org.apache.hadoop.mapred.lib.TotalOrderPartitioner;
            set total.order.partitioner.path=/tmp/hb_range_key_list;
            set hfile.compression=gz;
            create table hbsort(user_id string, user_type string, ...)
            stored as inputformat 'org.apache.hadoop.mapred.TextInputFormat’
            outputformat 'org.apache.hadoop.hive.hbase.HiveHFileOutputFormat’ tblproperties ('hfile.family.path' = '/tmp/hbsort/cf');
            insert overwrite table hbsort
            select user_id, user_type, createtime, …
  from hive_user_table
            cluster by user_id;
  

apache hive related article tags - hive tutorial - hadoop hive - hadoop hive - hiveql - hive hadoop - learnhive - hive sql

What is HBQL :

HBQL focuses on providing a convenient language layer for managing and accessing individual HBase tables, and is not intended for heavy-duty SQL processing such as joins and aggregations

HBQL is implemented via client-side calls, whereas Hive/HBase integration is implemented via map/reduce jobs