[Solved-2 Solutions] Copying file from s3:// to local file system ?
Problem:
How to Copying file from s3:// to local file system ?
Solution 1:
- We should use the s3n protocol (unless you stored the files on s3 using the s3 protocol) - s3 is used for block storage (i.e. similar to hdfs, only on s3) and s3n is for native s3 file system
Here is an example
raw_logs = LOAD 's3n://access:secret@XXXXX/input/access_log_1' USING TextLoader AS (line:chararray);
Solution 2:
- cp in unaware of s3
cp s3://my-bucket-name/path/to/file - We can also use the below one:
s3cmd get s3://some-s3-bucket/some-s3-folder/local_file.ext ~/local_dir/
Another example:
s3cmd get s3://BUCKET/OBJECT LOCAL_FILE
Get file from bucket