sqoop - Sqoop Job - apache sqoop - sqoop tutorial - sqoop hadoop
What is Sqoop job?
- Sqoop job creates and saves the import and export commands.
- Sqoop job command allows us to create a job.
- Job remembers the parameters used to create job, so they can be invoked any time with same arguments.
- It specifies parameters to identify and recall the saved job.
- This re-calling or re-executing is used in the incremental import, which can import the updated rows from RDBMS table to HDFS.
Learn sqoop - sqoop tutorial - sqoop job Re-Calling - sqoop examples - sqoop programs
Apache Sqoop Job Command With Example:
- The apache sqoop job defines how to create and maintain the Sqoop jobs.
- It specifies parameters to identify and recall the saved job. This re-calling or re-executing is used in the incremental import, which can import the updated rows from RDBMS table to HDFS.
Step 1 - Change the directory to /usr/local/hadoop/sbin
$ cd /usr/local/hadoop/sbinClick "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Step 2 - Start all hadoop daemons.
$ start-all.sh
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Step 3 - The JPS (Java Virtual Machine Process Status Tool) tool is limited to reporting information on JVMs for which it has the access permissions.
$ jps
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Step 4 - Change the directory to /usr/local/sqoop/bin
$ cd /usr/local/sqoop/binClick "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Sqoop options:
| Argument | Description |
|---|---|
| --create | Defines a new job withthe specified job-id(name).Actual sqoop import command should be seperated by "--" |
| --delete | Deletes a saved job. |
| --exec | Executes the saved job. |
| --show | show the save job configuration |
| --list | Lists all the saved jobs |
Create Job (--create)
- Here we are creating a job with the name myjob, that can import the table data from RDBMS table to HDFS.
- The below command is used to create a job that is importing data from the wikitechy_employee table in the db database to the HDFS file.
$ sqoop job --create myjob \
-- import \
--connect jdbc:mysql://localhost/userdb \
--username root \
--password root \
--table employee \
--m 1 \
--target-dir /targetfloder
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Verify Job (--list):
- The '--list' argument is used to verify the saved jobs.
- Use the below command to verify the list of saved Sqoop jobs.
$ sqoop job --list
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Learn sqoop - sqoop tutorial - Sqoop Job List - sqoop examples - sqoop programs
Inspect Job (--show):
- This '--show' argument is used to inspect or verify particular jobs and their details.
- The below command and the output is used to verify a job called myjob.
$ sqoop job --show myjob
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Learn sqoop - sqoop tutorial - Sqoop Job Show - sqoop examples - sqoop programs
Execute Job (--exec):
- This '--exec' option is used to execute a saved job.
- Use the below command to execute a saved job called myjob.
$ sqoop job --exec myjob
Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team
Learn sqoop - sqoop tutorial - Sqoop Job Exec - sqoop examples - sqoop programs
Verify after executing:
$ hdfs dfs -cat /user/hduser/targetfolder/part-m-00000
Learn sqoop - sqoop tutorial - Sqoop Job Verify - sqoop examples - sqoop programs
