sqoop - Sqoop saved Job - apache sqoop - sqoop tutorial - sqoop hadoop

The Saved Sqoop Job remembers the parameters used by a job so they can be re-executed by invoking the job several times.

Following command creates saved jobs:

sqoop job --create job_name --import --connect <connect-string>/dbname \ --table table_name

Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team

Copy Code

The command above just creates a job with the job name you specify.

It means that the job you created is now available in your saved jobs list which can be executed later.

Following command executes a saved job :

sqoop job --exec job_name --username uname –P

Click "Copy code" button to copy into clipboard - By wikitechy - sqoop tutorial - team

Copy Code

Sample Saved Job

Sqoop Metastore

A Sqoop metastore keeps track of all jobs.

By default, the metastore is contained in your home directory under .sqoop and is only used for your own jobs. If you want to share jobs, you would need to install a JDBC-compliant database and use the --meta-connect argument to specify its location when issuing job commands.

Important Sqoop commands:

• $ sqoop job –list – Lists all jobs available in metastore
• sqoop job --exec JOB1 – Executes JOB1
• sqoop job --show JOB1 – Displays metadata of JOB1

Sqoop Option File :

Certain arguments in import, export commands and saved jobs are to be written every time you execute them.

For instance following arguments are used repetitively in import and export commands as well as saved jobs :

learn sqoop - sqoop tutorial - sqoop2 tutorial - sqoop option text - sqoop job - sqoop code - sqoop programming - sqoop download - sqoop examples

So these arguments can be saved in a single text file say option.txt.

While executing the command just include this file for the argument --options-file.

Following command shows the use of –options-file argument:

Key points to note

Each argument in the option file should be on a new line.

-connect in option file cannot be written as --connect.

Same is the case for other arguments too.

Option file is generally used when large number of Sqoop jobs use a common set of parameters such as:

• Source RDBMS ID, Password
• Source database URL
• Field Separator
• Compression type