pig tutorial - apache pig tutorial - Apache Pig - SUBTRACT() Function - pig latin - apache pig - pig hadoop
What is SUBTRACT() Function in Apache Pig ?
- The SUBTRACT() function used in Apache Pig is used to subtract two bags.
- The SUBTRACT() function takes two bags as inputs and returns the bag contains the tuples of the first bag which are not there in the second bag.
- The SUBTRACT() function returns the difference between two numbers.
Syntax
grunt> SUBTRACT(expression, expression)
Example
We need to assume that we have two files namely wikitechy_employee_sales.txt and wikitechy_employee_bonus.txt in the HDFS directory /pig_data/ which is given below:
<b>wikitechy_employee_sales.txt</b>
1,RobinHood,22,25000,sales
2,BOB,23,30000,sales
3,Saya,23,25000,sales
4,Sarah,25,40000,sales
5,Joseph,23,45000,sales
6,Vanitha,22,35000,sales<b>wikitechy_employee_bonus.txt</b>
1,RobinHood,22,25000,sales
2,Abirami,23,20000,admin
3,Saya,23,25000,sales
4,Preethi,25,50000,admin
5,Joseph,23,45000,sales
6,Sruti,30,30000,admin
We have loaded the files into Pig, with the relation names called employee_sales and employee_bonus respectively.
employee_sales
grunt> employee_sales = LOAD 'hdfs://localhost:9000/pig_data/wikitechy_employee_sales.txt' USING PigStorage(',')
as (sno:int, name:chararray, age:int, salary:int, dept:chararray);
employee_bonus
grunt> employee_bonus = LOAD 'hdfs://localhost:9000/pig_data/wikitechy_employee_bonus.txt' USING PigStorage(',')
as (sno:int, name:chararray, age:int, salary:int, dept:chararray);
- We need to group the records and the tuples of the relation names employee_sales and employee_bonus with the key sno, by using the COGROUP operator which is given below:
grunt> cogroup_data = COGROUP employee_sales by sno, employee_bonus by sno;
- We need to verify the relation cogroup_data by using the DUMP operator which is given below:
<b>grunt> Dump cogroup_data;</b>
(1,{(1,RobinHood,22,25000,sales)},{(1,RobinHood,22,25000,sales)})
(2,{(2,BOB,23,30000,sales)},{(2,Abirami,23,30000,admin)})
(3,{(3,Saya,23,25000,sales)},{(3,Saya,23,25000,sales)})
(4,{(4,Sarah,25,40000,sales)},{(4,Preethi,25,50000,admin)})
(5,{(5,Joseph,23,45000,sales)},{(5,Joseph,23,45000,sales)})
(6,{(6,Vanitha,22,35000,sales)},{(6,Sruti,30,30000,admin)})
Subtracting One Relation from the Other
- We will now need to subtract the tuples of employee_bonus relation from the employee_sales relation.
grunt> sub_data = FOREACH cogroup_data GENERATE SUBTRACT(employee_sales, employee_bonus);
Verification
<b>grunt> Dump sub_data;</b>
({})
({(2,BOB,23,30000,sales)})
({})
({(4,Sarah,25,40000,sales)})
({})
({(6,Vanitha,22,35000,sales)})
- Now in the same way, we need to subtract the employee_sales relation from the relation employee_bonus relation which is given below:
grunt> sub_data = FOREACH cogroup_data GENERATE SUBTRACT(employee_bonus, employee_sales);
- Now we need to verify the contents of the sub_data relation by using the Dump operator which is given below:
<b>grunt> Dump sub_data;</b>
({})
({(2,Abirami,23,20000,admin)})
({})
({(4,Preethi,25,50000,admin)})
({})
({(6,Sruti,30,30000,admin)})