Bash can run multiple processes concurrently, allowing you to speed up tasks like downloading files, processing logs, or batch operations. This is useful in automation, scripting, DevOps, and data pipelines.
&Start any command as a background job:
sleep 5 &
echo "Sleeping in background..."
Use jobs to list background tasks.
The wait command pauses until all background jobs finish.
echo "Starting jobs..."
sleep 3 & # Job 1
sleep 5 & # Job 2
wait # Waits for both to complete
echo "All done"
Wait for a specific PID:
wait 12345
Example: Download 3 files in parallel
for i in 1 2 3; do
curl -O http://example.com/file$i.txt &
done
wait
xargs -Pxargs can run multiple commands in parallel using -P:
cat urls.txt | xargs -n 1 -P 4 curl -O
| Option | Description |
|---|---|
-n 1 |
One argument per command |
-P 4 |
Run up to 4 processes in parallel |
parallel (optional)Powerful tool for multi-core execution:
cat files.txt | parallel gzip
Install it:
sudo apt install parallel
jobs # List jobs
fg %1 # Bring job 1 to foreground
bg %1 # Resume job in background
Store PID and kill manually:
my_script.sh &
pid=$!
sleep 5
kill $pid
| Tip | Why |
|---|---|
Use wait after & |
Prevent premature script exit |
Use xargs -P |
Efficient for batch workloads |
| Limit background jobs | Avoid resource overuse |
| Redirect output | Prevent mixed stdout/stderr |
📁 File:
advance/concurrency.md
Author: Kashif Alam