sqoop - sqoop2 - sqoop2 tutorials - apache sqoop - sqoop tutorial - sqoop hadoop
What is Sqoop2? - sqoop2 - sqoop2 tutorial
- Teradata, MySQL, PostgreSQL, Oracle, Netezza
- HDFS (text, sequence file), Hive, HBase, Avro And vice versa
sqoop-hadoop-hive :
![learn sqoop - sqoop tutorial - sqoop-hadoop-hive - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/sqoop-hadoop-hive.png)
![learn sqoop - sqoop tutorial - sqoop import export - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/sqoop-import-export.png)
Sqoop1 Architecture :
![learn sqoop - sqoop tutorial - what is sqoop - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/what-is-sqoop.png)
- CrypAc, contextual command line arguments
- Tight coupling between data transfer and output format
- Security concerns with openly shared credentials
- Not easy to manage installation/Configuration
- Connectors are forced to follow JDBC model
Sqoop2 Architecture :
![learn sqoop - sqoop tutorial - what is sqoop2 - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/what-is-sqoop2.png)
![learn sqoop - sqoop tutorial - what is sqoop2 - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/sqoop2.png)
Sqoop1: Client side Tool
- - Connectors are installed/configured locally
- – Local requires root privileges
- – JDBC drivers are needed locally
- – Database connecAvity is needed locally
Sqoop2: Sqoop as a Service - client side tools :
- – Connectors are installed/configured in one place
- – Managed by administrator and run by operator
- – JDBC drivers are needed in one place
- – Database connectivity is needed on the server
![learn sqoop - sqoop tutorial - what is sqoop2 - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/sqoop-as-a-service.png)
Client Interface
- – Command line interface (CLI) based
- – Can be automated via scripting
- – CLI based (in either interactive or script mode)
- – Web based (remotely accessible)
- – REST API is exposed for external tool integration
Sqoop 2: Connection vs Job metadata :
- Connection (distinct per database)
- Job (distinct per table)
![learn sqoop - sqoop tutorial - sqoop2 tutorial - database connection - sqoop code - sqoop programming - sqoop download - sqoop examples](https://wikitechy.com/tutorials/sqoop/img/sqoop-images/sqoop2-metadata-database-connection.png)
- Connectors Register Metadata
- Metadata enables creation of Connections and Jobs
- Connections and Jobs stored in Metadata Repository
- Operator runs Jobs that use appropriate connections
- Admins set policy for connection use
Sqoop 2: Security
- Administrators create/edit/delete connections
- Operators use connections