M15 — Archives and remote file transfer

Prerequisites#

• M5 (copy/move) and M14 (you can write a tiny script).
• Optional: SSH access to a second Linux host (cloud VM, second WSL distro, or a lab partner). If you only have one machine, do the local archive labs and read the remote sections for later.

Learning objectives#

1. Create and list tar archives; extract them safely into a chosen directory.
2. Compress with gzip (and recognize .tar.gz / .tgz).
3. Optionally use zip / unzip when collaborating with non-Unix tools.
4. Copy files with scp and browse with sftp when SSH is available.
5. Explain when rsync is better than a blind re-copy.
6. Complete a pack → transfer → unpack workflow.

Lab environment#

mkdir -p /tmp/linux-lab-pushpjeet/m15/project/{src,docs} \
         /tmp/linux-lab-pushpjeet/m15/extract \
         /tmp/linux-lab-pushpjeet/m15/remote-sim
cd /tmp/linux-lab-pushpjeet/m15
echo "src code v1" > project/src/app.py
echo "readme" > project/docs/readme.md
echo "config" > project/config.ini

1. Archive vs compress (do not mix them up)#

project-backup.tar.gz means: tar first, then gzip. People say “tarball” for that combo.

2. tar — the daily driver#

Common flags (GNU tar; remember them as a set):

Create#

cd /tmp/linux-lab-pushpjeet/m15
tar -czf project-backup.tar.gz project
ls -lh project-backup.tar.gz

Lab run: archive was about 252 bytes for this tiny tree (your size will grow with real projects).

List#

tar -tzf project-backup.tar.gz

Expected:

project/
project/config.ini
project/docs/
project/docs/readme.md
project/src/
project/src/app.py

Extract elsewhere (safer than dumping into .)#

tar -xzf project-backup.tar.gz -C extract
find extract -type f | sort

Expected:

extract/project/config.ini
extract/project/docs/readme.md
extract/project/src/app.py

Safety tip: before extracting an unknown archive, always -t list it. Avoid extracting as root into /.

Useful create pattern (like M14)#

tar -czf backup.tar.gz -C /path/to/parent dirname

-C keeps paths short inside the archive (dirname/... instead of absolute paths).

3. gzip on a single file#

cp project/docs/readme.md readme-copy.md
gzip -k readme-copy.md    # -k keeps the original (GNU gzip)
ls -l readme-copy.md*

Lab run:

-rw-r--r-- 1 box box  7 ... readme-copy.md
-rw-r--r-- 1 box box 42 ... readme-copy.md.gz

(Tiny files can get larger after gzip headers — normal. Compression wins on logs and source trees.)

Decompress: gunzip -k readme-copy.md.gz or gzip -dk readme-copy.md.gz.

4. zip / unzip (optional, cross-platform)#

When a teammate on Windows expects a .zip:

zip -r project.zip project
unzip -l project.zip
unzip project.zip -d /tmp/from-zip

If zip is missing on a minimal server, install it via your package manager (M16) or use tar.gz instead. On this drafting box zip was not preinstalled; unzip -l still lists archives created another way.

5. Remote copy over SSH#

All of these need network reachability and SSH credentials (key or password). Replace lab@server-b with your real user and host.

scp — secure copy#

# file → remote home
scp project-backup.tar.gz lab@server-b:~/

# remote → local
scp lab@server-b:~/project-backup.tar.gz ./from-remote.tar.gz

# recursive directory
scp -r project lab@server-b:~/incoming/

sftp — interactive session#

sftp lab@server-b
# inside: ls, cd, put file, get file, bye

rsync — smart sync (intro)#

rsync copies differences and can resume. Local mirror example:

rsync -av project/ remote-sim/project-mirror/

Remote form:

rsync -avz project/ lab@server-b:~/project/

Trailing slash matters: project/ means “contents of project”; project means “the directory named project”.

If scp / rsync are not installed on a minimal image, install openssh-client and rsync (M16). On the drafting Debian box they were absent — the workflow below uses a local pack → copy → unpack so you can still practice without a second host.

6. Workflow: pack → copy → unpack#

Even on one machine, rehearse the shape you will use across hosts:

cd /tmp/linux-lab-pushpjeet/m15
tar -czf project-backup.tar.gz project
mkdir -p remote-sim/inbox
cp project-backup.tar.gz remote-sim/inbox/   # stand-in for scp
mkdir -p remote-sim/unpacked
tar -xzf remote-sim/inbox/project-backup.tar.gz -C remote-sim/unpacked
find remote-sim/unpacked -type f

When you have SSH:

scp project-backup.tar.gz lab@server-b:~/inbox/
ssh lab@server-b 'mkdir -p ~/unpacked && tar -xzf ~/inbox/project-backup.tar.gz -C ~/unpacked'

Common mistakes#

Hands-on lab (45–60 minutes)#

Part A — Tar round-trip (20 min)#

1. Build the project/ tree above.
2. Create project-backup.tar.gz.
3. List with -tzf; extract into extract/.
4. diff -r project extract/project — should be silent (identical).

Part B — Gzip a log (10 min)#

1. Copy any text file; gzip -k it.
2. Compare sizes with ls -l.
3. gunzip -k and confirm content matches.

Part C — Remote or simulated remote (15 min)#

If you have server-b: scp the tarball across; extract on the far side.
If not: use the remote-sim/ workflow in section 6.

Part D — Stretch#

Write a script ship-project.sh that (1) tars project/, (2) prints scp command you would run, (3) optionally runs it if $REMOTE is set (REMOTE=lab@server-b ./ship-project.sh).

Practice: check your understanding#

Multiple choice#

Q1. What does tar -czf out.tar.gz dirname do?

• A. Only compresses a single file named dirname
• B. Creates a gzip-compressed archive of dirname
• C. Extracts out.tar.gz into dirname
• D. Starts an sftp session

Answer: B.

Q2. Why run tar -tzf file.tar.gz before extracting an unknown archive?

• A. It deletes unsafe files automatically
• B. It lists contents so you can see where files will land
• C. It converts the archive to zip
• D. It verifies SSH keys

Answer: B.

Q3. Which tool is best when you re-sync a large tree often and only a few files changed?

• A. Always full scp -r
• B. rsync
• C. gzip alone
• D. chmod

Answer: B.

Flashcards#

Match the columns — tool ↔ job#

Fill in the blank#

1. Create: tar ___ project-backup.tar.gz project
   Answer: -czf

2. List: tar ___ project-backup.tar.gz
   Answer: -tzf

3. Extract into extract/: tar -xzf project-backup.tar.gz ___ extract
   Answer: -C

4. Secure copy to remote home: ___ my.tgz lab@server-b:~/
   Answer: scp

Order the steps — pack → copy → unpack#

1. tar -czf project-backup.tar.gz project
2. scp project-backup.tar.gz lab@server-b:~/inbox/
3. On remote: mkdir -p ~/unpacked
4. On remote: tar -xzf ~/inbox/project-backup.tar.gz -C ~/unpacked
5. Spot-check with find ~/unpacked -type f (or diff against a known good tree)

Correct order: 1 → 2 → 3 → 4 → 5

Mini terminal challenges#

Challenge: Archive a project#

Goal: Produce project-backup.tar.gz and prove it extracts cleanly.

Setup: Use /tmp/linux-lab-pushpjeet/m15/project.

Tasks:

1. Create the archive in m15/.
2. Extract into a fresh roundtrip/ directory.
3. diff -r project roundtrip/project.

Verify: diff prints nothing.

Stretch: Exclude *.pyc or __pycache__ if you add some (tar --exclude).

Challenge: Simulated ship#

Goal: Complete section 6’s remote-sim workflow without SSH.

Verify: remote-sim/unpacked/project/src/app.py exists and matches the source.

Module wrap#

You can pack trees with tar/gzip, extract safely, and move archives with scp or sync with rsync. Next: install the missing tools with your distro’s package manager (M16).

Continue

← Linux from Scratch hub · Cheat sheet · All tutorials