nohup rsync -avhPc /source/directory/ /destination/directory/ &
或者tmux/screen里直接执行
rsync -avhPc /source/directory/ /destination/directory/
好习惯用rsync 代替mv,可以一点一点拷贝。 -c 拷完检查完整性,-a归档模式,基本保留完整属性, -vh 打一大堆,数字人类可读以GB展示,-P partial files 按部分文件一点点拷贝.并打印进度条 99%之类的
cp (copy files and directories)
rm (remove files or directories)
dd (convert and copy a file)
tar (tape archive, for creating archives of files)
grep (search for text in files)
scp (可以用rsync替代)
I/O-bound or the tasks are not easily parallelizable
很多操作是受IO性能限制,或者本身不太容易并行。
rsync -avz -e ssh /path/to/local/file user@remotehost:/path/to/remote/file
## 如果实在想多线程
find /path/to/source -type f | parallel -j 10 rsync -avz {} user@remotehost:/path/to/destination/
其他多线程命令:
As for multi-threaded commands in Linux, here’s a list of commands and utilities that can utilize multiple threads:
pigz
: A multi-threaded implementation of gzip for compression.pbzip2
: A parallel implementation of bzip2 for data compression.lbzip2
: Another multi-threaded bzip2 utility.pxz
: Parallel LZMA compressor using liblzma.plzip
: A parallel, lossless data compressor based on the LZMA algorithm.zstd
: Offers multi-threaded compression and decompression.parallel
: Not a direct replacement for any specific command, but can be used to run any command in parallel.rsync
: Can utilize multiple cores when combined with tools likeparallel
.samtools
: Used in bioinformatics for processing SAM and BAM files, with multi-threading support for compression and decompression.ffmpeg
: A multimedia framework that can use multiple threads for video encoding and decoding.maketorrent-cli
: A command-line utility to create BitTorrent files in parallel.aria2c
: A lightweight multi-protocol & multi-source, cross platform download utility operated in command-line with multi-threading support.lftp
: While primarily single-threaded, it can open multiple connections to speed up FTP/SFTP transfers.
这些尤其是CPU限制的操作如压缩和编码。(对了,注意别把CPU打满把运行中的服务打爆了)
These utilities are specialized for certain tasks where multi-threading provides a benefit, particularly for CPU-bound operations like compression or encoding. When using any multi-threaded utility, it’s important to consider the resources available on the system to avoid overloading the CPU or other system resources.
SRE升级Path/成长之路: 1. https://wsgzao.github.io/post/sre/
2. https://www.bilibili.com/video/BV1Z5411D7Gb
滴滴崩溃听说是K8S版本升爆了(错误升到旧版导致依赖炸了,甚至无法重启)。猜测是重装。(纯属猜测,不保证准确性)
阿里云香港机房全面崩溃后,又全体服务崩溃。再加上之前上某海的高端操作。有些滑稽,经济差关系差难搞啊。
不过cf 11月2日也爆过,据说停电坑了、、
加上现在越来越不安稳了,云安全越来越难以保障。