June 20, 2021

Linux Commands

 

command what it does
awk ‘!/virus=””/’ > sometextfile pipes notification of nasties to a text file called sometextfile
cat /proc/meminfo shows your memory stuff (or just type “free”)
cat /proc/version shows what version of Linux you’re running</td
chmod -R 777 * changes ownership of all the files below where you are to read-write-executable…not normally a super idea, 755 is slightly better, or 644 much better
cp -R ./* /some/other/place/ copies recursively from where you are to /some/other/place/ folder you specify
cp -ur –reply=y /somefolder/* /home/whateverdir/ copies updates only to someplace you specify
chown -R user.group ./* changes ownership of user and group for all the files recursively from where you are, just pick your user and group
cut -b 42- photos.txt > photoscut.txt cuts off the first 42 characters in the lines in photos.txt and puts the cropped data in photoscut.txt
(pv -n /dev/sda | dd of=/dev/sdb bs=128M conv=notrunc,noerror) 2>&1 | dialog --gauge "Running dd command (cloning), please wait..." 10 70 0 copies the WHOLE drive of /dev/sda to /dev/sdb. Not just a partition, but a bit-for-bit thing. The rest of the command gives you a nifty progress bar, because dd takes a long time very often. Dd is really blunt instrument, if you don’t know what you’re doing you can easily lose ALL of your data, like if you pick the wrong drive or something. Use fdisk -l to tell what disks you have, and make triple sure you’re selecting the right one. You’ve been warned.
dd if=/dev/sdc of=/path/to/new.iso turns a disk into an .iso, in this case /dev/sdc, but make sure you’re using the right drive
date -s time.nrc.ca sets the date to internet time, or just tells you the date if you type “date” only
lsof -i shows who’s connected, and to what
lsblk shows what’s on your drive like: >: lsblk
NAME MAJ:MIN RM SIZE RO TYPE MOUNTPOINT
sda 8:0 0 50G 0 disk
+-sda1 8:1 0 48G 0 part /
+-sda2 8:2 0 1K 0 part
+-sda5 8:5 0 2G 0 part [SWAP]
sr0 11:0 1 2K 0 rom
date -s “11/20/2003 12:48:00” sets the time
dd if=/dev/zero of=whateverfile bs=1000 count=0 creates a 10G blank file called whateverfile
dig -x 17.254.3.183 reverse DNS on 17.254.3.183
du -hs * lists the directories below where you are and shows how much space they take
find / -type f -size +500000k -exec ls -lh {} \; finds large files and displays filesize of them
rdate -s tick.greyware.com sets local hardware time to the standard time on the remote time server on greyware.com
route add default gw 192.168.1.1 adds a default gateway when other things fail to do so, you’ll need one
rsync -auv -e ssh username@192.168.1.1:* /localwhereverdirectory/ syncs all the files over encrypted protocol in the home directory of 192.168.1.1 to the current directory on the machine you type this on, but only newer instances if their is a same file in the source and destination directories.
screen opens a new ‘virtual terminal’ so once you log out, it will keep whatever you’re doing in that terminal still open and running – very useful if you have a flaky Internet connection
screen -ls lists screen sessions that might still be running, should give you something like: 8337.pts-1.machinename , which you then can re-attach to
screen -r 8337.pts-1.machinename reattaches to and old screen session 8337.pts-1.machinename, but ONLY if it’s detached, if it’s still attached you have to use screen -X
screen -X 8337.pts-1.backup1 reattaches to a screen session that still says ‘attached’
sed -i -e 's/192.168.0.1/192.168.0.2/g' ./db* replaces all the 192.168.0.1 instances in all the files in the current directory starting with db with 192.168.0.2
tar -cvzpf archive.tgz ./whateverfolder zips (.tgz) an entire folder (change to suit folder name)
tar -xJvf file.pkg.tar.xz unzips a whatever.xz file
tar -pczvf archive.tar.gz ./somefolder does same thing, but with tar.gz
tar -xjvf whateverbz2zippedfile.bz2 unzips a whatever.bz2 file
umount -l /dev/sda1 forces umount to unmount a filesystem when it says it’s busy, but isn’t (make sure first)
watch -n .1 cat /proc/mdstat keeps track of a file in realtime, sort of like tail -f

migrating to a larger hard drive

If you want to make a backup image of a whole hard drive, then restore that image to your new larger drive, you can use dd. It is VERY VERY important to realize which is your source disk and which is your destination, or you’ll nuke all your data rapidly.

In this case, I hook up both source and destination drives via usb drive adapter, so both are plugged into my Linux laptop. My source destination is 500GB and the OS says it’s on /dev/sdd. It has both /dev/sdd1 and /dev/sdd2, but since I want a total copy, I just need to know it’s /dev/sdd, since dd makes a really low level exact copy of the whole drive.

My destination drive is another hard drive I store images on that’s /dev/sdb1. So I’m going to make a backup image first to this drive, then transfer that image to my destination drive eventually. I’m doing this in this example because I don’t have enough space on my OS hard drive to store the image, so I have to put it somewhere, and I want a backup image anyway in case I break something later and need to restore.

To make sure I have the right drives for source and destination I used fdisk like:

fdisk -l
 
Disk /dev/sdb: 2.7 TiB, 3000592977920 bytes, 732566645 sectors
Disk model: GoFlex Desk     
Units: sectors of 1 * 4096 = 4096 bytes
Sector size (logical/physical): 4096 bytes / 4096 bytes
I/O size (minimum/optimal): 4096 bytes / 4096 bytes
Disklabel type: gpt
Disk identifier: 3DF81F4A-3FB2-4EB6-AE14-51FA003E04D6
 
Device     Start       End   Sectors  Size Type
/dev/sdb1    256 732566527 732566272  2.7T Microsoft basic data
 
 
Disk /dev/sdd: 465.8 GiB, 500107862016 bytes, 976773168 sectors
Disk model: SSD 860 EVO 500G
Units: sectors of 1 * 512 = 512 bytes
Sector size (logical/physical): 512 bytes / 512 bytes
I/O size (minimum/optimal): 512 bytes / 512 bytes
Disklabel type: dos
Disk identifier: 0x4ef3b8de
 
Device     Boot    Start       End   Sectors  Size Id Type
/dev/sdd1           2048  62500863  62498816 29.8G 82 Linux swap / Solaris
/dev/sdd2  *    62500864 976771071 914270208  436G 83 Linux

Now I need to know where those drives are mounted, so I run:

mount
....
bunch of stuff
...
binfmt_misc on /proc/sys/fs/binfmt_misc type binfmt_misc (rw,relatime)
tracefs on /sys/kernel/debug/tracing type tracefs (rw,relatime)
/dev/sdb1 on /media/user/589415f9-5115-4972-a3a0-e1fec245caf8 type ext3 (rw,nosuid,nodev,relatime,uhelper=udisks2)
/dev/sdd2 on /media/user/53293b75-6bd2-4e42-b537-10fff5fb78b0 type ext4 (rw,nosuid,nodev,relatime,uhelper=udisks2)

So now I modify the following command to match my environment. YOURS IS GUARANTEED TO BE DIFFERENT, if you just copy paste without changing to reflect your environment, the best outcome is that it won’t work. The worst is you break something very important:

dd if=/dev/sdd conv=sync,noerror status=progress bs=64K | gzip -c  > /media/user/589415f9-5115-4972-a3a0-e1fec245caf8/backup_image.img.gz

This will just sit there forever until it finishes. to see how it’s doing, open another terminal and do:

>: ls -lah
total 71G
drwxr-xr-x  8 root      root 4.0K Apr  5 09:44 .
drwxr-x---+ 5 root      root 4.0K Mar 29 23:43 ..
-rw-r--r--  1 root      root 276M Apr  5 09:59 backup_image.img.gz

you can see the backup_image.img.gz getting bigger. I did this after around 5 minutes and 276MB of 500MB is how far it got, as an example. After while it started building a zip file, so then it changed to GB. You can also run iotop to see how much disk I/O you’re using, it will be very high if it’s working.

Here’s what it will look like when finished. This was a 500GB drive which took 4.5 HOURS!, though the compressed file was only 166GB, and since this drive only had 4MB free, there was a lot of compression. Because it takes so long, disable hibernate features on your laptop or it will go to sleep in the middle of the process:

dd if=/dev/sdd conv=sync,noerror status=progress bs=64K | gzip -c  > /media/user/589415f9-5115-4972-a3a0-e1fec245caf8/backup_image.img.gz
500105216000 bytes (500 GB, 466 GiB) copied, 16489 s, 30.3 MB/s
7631040+1 records in
7631041+0 records out
500107902976 bytes (500 GB, 466 GiB) copied, 16489.1 s, 30.3 MB/s

Once the command finishes, you have to disconnect your source disk/usb adapter and plug the new one in. Then you have to restore your saved image like:

gunzip -c /media/user/589415f9-5115-4972-a3a0-e1fec245caf8/backup_image.img.gz | dd of=/dev/sda status=progress