cmds

Table of Contents

1 EMACS

1.1 Useful keys

M-x load-file
C-x k : kill buffer 
C-x C-x:  start a a location l,  search for something to go to new location,  then press C-x C-x to return back to l
M-x occur:  search all line with matched regexp
C-x C-r : read only
C-x C-v:  reload file (not technically but it works well)
C-c {  in latex mode to create {}
C-j  insert paragraph break
C-c C-s:  insert section subsection paragraph and label
C-c C-e: insert environment (e.g. figure, equation, list etc)
C-c C-f C-e:  emphasize
C-c %   or  ;  :  comment out the paragraph or line
C-c C-r:  on a highlighted region to latex only that region
C-c C-c:  to run latex.  
C-c `:  keep on pressing this to review  errors after running latex
C-/:  undo 
C-x k: kill buffer
M-/: autocomplete
M-h : mark entire paragraph
C-x h: mark entire buffer
C-t: transpose 2 letters  , e.g.   letet C-t  => lette
M-t: transpose 2 words,  e.g.,  world hello  M-t  ->  hello word  ;   vu, nguyen M-t  => nguyen, vu  (note: cursor btw the 2 words,  Alt-b to move back a word)
C-x C-t: transpose 2 lines 
M-u : uppercase letters
M-c : capitalize letters
C-s C-w:  to the end of the word (handy)
Search and replace
C-s twice to redo the last  search
C-s C-w  search the word under the cursor

Windows resize
C-x + : make windows same height/width
C-x - : shrink window to fit content

1.2 TeX

M-x align-current: To align columns in LaTeX table, highlight the table region, then M-x align-current

1.3 Install Flyspell on Mac

  • Install cocoAspell (which provides aspell)
  • Install the appropriate dictionary (./configure; make; make install)

2 APT

#shows which package owns /path/to/file
apt-file search /path/to/file 
#allows reinstalling a package without having to remove it first (and risk removing its dependencies)
apt-get install --reinstall packagename 
apt-file list packagename
apt-show-versions | grep unstable
apt-get build-dep packagename

3 Development

3.1 JAVA

  • Install Sun/Oracle's JDK
# First add "contrib" to /etc/apt-sources.list
# Then
apt-get update & apt-get install java-package
make-jpkg /path/to/jdk-*-linux-x64.tar.gz
dpkg -i oracle-java-*_amd64.deb

4 Admin

4.1 Email

We use ssmtp to send emails. After installing, edit the /etc/ssmtp/ssmtp.conf file (this example uses Gmail's server):

root=username@gmail.com
mailhub=smtp.gmail.com:587
rewriteDomain=
hostname=fileserver.local
UseSTARTTLS=YES
AuthUser=username
AuthPass=password
FromLineOverride=YES

Now edit /etc/ssmtp/revaliases to add accounts that we want to be able to send mails. For example,

root:nguyenthanhvuh@gmail.com:smtp.gmail.com:587
auser:nguyenthanhvuh@gmail.com:smtp.gmail.com:587

We can test sending mails

$ echo "test" | mail -v -s "testing ssmtp setup" tnguyen@cs.umd.edu
$ echo "Do this" | mail -s "Todo" nguyenthanhvuh@gmail.com
$ echo "A TEST" > afile
$ mail -s "Todo" nguyenthanhvuh@gmail.com < afile

4.2 RAID

This is for RAID-5 setup, which requires a minimum of 3 drives. The usable space is (number of drives - 1) * size of smallest drive. Here we will use mdm to manage our RAID setup (apt-get install mdadm).

To create new drive partitions in the RAID array, we can use fdisk or parted (if the disk size > 2 terabytes). If use fdisk then choose the partition type to be raid (type fd). Now assume we want to use the 4 partitions sdc1, sdc1, sde1, and sdf1.

#build the RAID array
mdadm --create --verbose /dev/md0 --level=5 --raid-devices=4 /dev/sd[cdef]1

#it will take some time to build the array, we can watch its status with
watch cat /proc/mdstat

#once the array is built, we can check with
mdadm --detail /dev/md0

/dev/md0:
        Version : 1.2
  Creation Time : Fri Jun 24 15:04:32 2016
     Raid Level : raid5
     Array Size : 734954496 (700.91 GiB 752.59 GB)
  Used Dev Size : 244984832 (233.64 GiB 250.86 GB)
   Raid Devices : 4
  Total Devices : 4
    Persistence : Superblock is persistent

  Intent Bitmap : Internal

    Update Time : Sat Jun 25 00:34:10 2016
          State : clean 
 Active Devices : 4
Working Devices : 4
 Failed Devices : 0
  Spare Devices : 0

         Layout : left-symmetric
     Chunk Size : 512K

           Name : GiaoChi:0
           UUID : 1d431c63:a9fe7dca:ffb4dca5:58391d2e
         Events : 1097

    Number   Major   Minor   RaidDevice State
       0       8       33        0      active sync   /dev/sdc1
       1       8       49        1      active sync   /dev/sdd1
       2       8       65        2      active sync   /dev/sde1
       4       8       81        3      active sync   /dev/sdf1


#create file extension and mount
mkfs.ext4 /dev/md0
mkdir /MYNEWSTORAGE
#in /etc/fstab 
/dev/md0        /MYNEWSTORAGE            ext4        defaults        0        0
#finally mount the array so that we can being to access /MYNEWSTORAGE
mount -av


#edit configuration files so that mdm knows how to assemble the array when the system boots
echo "DEVICE partitions" > /etc/mdadm/mdadm.conf
echo "HOMEHOST fileserver" >> /etc/mdadm/mdadm.conf
echo "MAILADDR username@gmail.com" >> /etc/mdadm/mdadm.conf
mdadm --detail --scan --verbose >> /etc/mdadm/mdadm.conf

5 GPG

gpg --gen-key
gpg --export  -a "Name" > pub.key
gpg --export-secret-key -a "Name" > priv.key
gpg --import priv.key
gpg --list-keys
gpg --list-secret-keys
gpg --delete-key "Name"
gpg --delete-secret-key "Name"
gpg --edit-key "Name"  #here you can change passphrase of priv key with passwd

6 Miscs

6.1 Files

#move with overwrite (and will not ask for confirmation)
mv -f  

#do *not* overwrite (and will not ask for confirmation)
mv -n
  • sync dirs
#todir will become exactly like fromdir
rsync -val --delete --exclude "*.ext"  --exclude "dir" fromdir todir
  • syn website
# I use the this command to synchronize my webpages to the http server
rsync -azv --delete --exclude=.hg --exclude=.hgcheck -e ssh ~/www tnguyen@cse.unl.edu:/home/fac/tnguyen/public_html/
  • recursively change directory whose permision is 777 to 755
find . -type d -perm 777 -print -exec chmod 755 {} \;
  • recursively delete files with extensions *.ext
find -name \*.ext -delete
  • Dropbox find conflicts
find ~/Dropbox/ -path "*(*'s conflicted copy [0-9][0-9][0-9][0-9]-[0-9][0-9]-[0-9][0-9]*" -print
  • grep
grep "error" file.txt | grep -v "multiple output" | grep -v "notexist"
grep -A 10 -B 10 "hello world" file.txt:  output surrounding lines

6.2 Benchmarks

  • CPU
echo '2^2^25' | time -p bc > /dev/null
time -p echo "scale=5000; a(1)*4" | bc -l
sysbench --test=cpu --cpu-max-prime=20000 run #apt-get install sysbench

6.3 Keys

  • permanent switching capslock and control keys
#add/replace to /etc/default/keyboard
XKBOPTIONS="ctrl:nocaps" # Some people prefer "ctrl:swapcaps"
sudo dpkg-reconfigure -phigh console-setup
  • Emacs key in XFCE
$ xfconf-query -c xsettings -p /Gtk/KeyThemeName -s Emacs
OR
Menu -> Settings -> Settings Editor -> xsettings -> Gtk -> KeyThemeName,  enter Emacs in the field

6.4 Multimedia

  • Burn videos, e.g., mpg, to DVD (http://www.lamolabs.org/blog/author/slmingol/)
    1. convert video with ffmpeg
      ffmpeg -i input.m4v -target ntsc-dvd output.mpg
      
    2. do authoring
      dvdauthor --title -o dvd -f output.mpg
      dvdauthor -o dvd -T
      # NOTE: --title sets the title of the DVD, -T sets the table of contents. In both 
      #       above commands the -o switch is referencing a directory, NOT the actual dvd.
      
    3. roll the .mpg file into an ISO file
      mkisofs -dvd-video -o dvdimage.iso dvd
      # NOTE: mkisofs is making an actual DVD video ISO file using the directory, dvd.
      
    4. burn ISO to DVD
      growisofs -speed=1 -dvd-compat -Z /dev/dvd=dvdimage.iso
      NOTE: -speed=1 is for use with lower quality discs, increase as necessary
      NOTE: This approach can be used to convert basically any format (m4v, mp4, etc.) to a DVD. Simply change the input file accordingly.
      

6.5 SSH

  • password-less login
cat ~/.ssh/id_rsa.pub | ssh username@host 'cat >> ~/.ssh/authorized_keys'
  • scp
scp -P port_number username@host:/path/to/file /destination/dir
  • resume file transfer
rsync --partial --progress --rsh=ssh user@host:/path/to/file .
  • multi-hop ssh e.g., ssh to user1@host1:port1 and then ssh to user2@host2:port2 (as user1@host1).
# This method uses netcat (the nc command) and using ProxyCommand 

# add the following entry to file ~/.ssh/config
Host AName
Hostname host1
ProxyCommand ssh -q user1@host1 -p port1 nc host2 port2
StrictHostKeyChecking no
UserKnownHostsFile=/dev/null

$ ssh user2@AName
#first password asked is for user1@host1, second password asked is for user2@host2

6.6 Users

  • Add user to sudo
$ sudo adduser username sudo
  • Change username from olduser to newuser.
# make sure olduser is not logged in or using any process, e.g., restart and log in as another user.
$ sudo usermod -l newuser olduser
$ sudo groupmod -n newuser olduser
$ sudo usermod -d /home/newuser -m newuser
  • Disable users
passwd -l username  to lock , passwd -u username to unlock

6.7 Website

  • Password Protected Website
    # To protect a directory called 'SecDir' on your website, 
    # go to SecDir and create/edit .htaccess file as follows:
    AuthName "MySecDir"
    AuthType Basic
    AuthUserFile /path/to/.htpasswd
    require valid-user
    
    $ htpasswd -c /path/to/.htpasswd guestname #create a passwod
    $ chmod 644 /path/to/.htpasswd   #change permission 
    
    # to log in SecDir online, enter username: guestname and passwod: yourpass
    

6.8 Webserver

  • Run a local server
    $ cd /path/to/dir
    $ python -m SimpleHTTPServer #optional, can also enter the port e.g., 8080
    

7 MAC OS

  • Creating an ISO from a CD/DVD e.g., to back up CD
    Insert CD, open Disk Utils, Select the CD (not the drive), 
    Create File / New / Disk Image from ... / , Image Format:  CD/DVD Master  ...  rename cdr to iso
    
  • Burn ISO image to USB (adapted from Ubuntu website)
    1. First convert ISO to dmg file
      $ hdiutil convert -format UDRW -o ~/Downloads/ubuntu-10.04.3-desktop-i386.img ~/Downloads/ubuntu-10.04.3-desktop-i386.iso
      Reading Ubuntu 10.04.3 LTS i386          (Apple_ISO : 0)…
      ...............................................................................
      Elapsed Time: 35.932s
      Speed: 19.1Mbytes/sec
      Savings: 0.0%
      created: /Users/tnguyen/Downloads/ubuntu-10.04.3-desktop-i386.img.dmg
      
    2. Then find the USB device
      $ diskutil list
      /dev/disk0
         #:                       TYPE NAME                    SIZE       IDENTIFIER
         0:      GUID_partition_scheme                        *320.1 GB   disk0
         1:                        EFI                         209.7 MB   disk0s1
         2:                  Apple_HFS Mac                     319.7 GB   disk0s2
      /dev/disk1
         #:                       TYPE NAME                    SIZE       IDENTIFIER
         0:     FDisk_partition_scheme                        *2.0 GB     disk1
         1:             Windows_FAT_32                         2.0 GB     disk1s1
      
      $ diskutil unmountDisk /dev/disk1
      Unmount of all volumes on disk1 was successful
      
    3. Finally, copy data to USB disk
      $ sudo dd if=~/Downloads/ubuntu-10.04.3-desktop-i386.img.dmg of=/dev/disk1 bs=1m
      687+1 records in
      687+1 records out
      721127424 bytes transferred in 77.176835 secs (9343833 bytes/sec)
      
      $ diskutil eject /dev/disk1
      Disk /dev/disk1 ejected
      

8 Virtual Box

  • SSH from Host OS to Virtualbox (VB) Guest OS using port forwarding.
    1. The following assumes that the VB Guest is called myserver. Also make sure that the ssh server is installed on myserver.
    2. The Guest OS, by default, should have one interface already which is using NAT. First go to the Network settings, Advanced/Port Forwarding, and then add a new rule:
      Host port 2222, guest port 22, name ssh, other left blank
      

      or from command line

      HostOS$ VBoxManage modifyvm myserver --natpf1 "ssh,tcp,,2222,,22"
      
    3. To check the added rules:
      HostOS$ VBoxManage showvminfo myserver | grep 'Rule'
      
    4. Log in the Guest OS
      HostOS$ ssh -p 2222 username@localhost
      

      where username is your username in myserver.

    Others:

    • Setup password-less login so that we don't have to enter the password everytime.
      HostOS$ cat ~/.ssh/id_rsa.pub | ssh username@localhost -p 2222 'cat >> ~/.ssh/authorized_keys'
      
    • Copy files from host to guest using scp, or vice versa
      #copy from host to guest
      HostOS$ scp -P 2223 /path/to/file   fse16ae@localhost:/destination/dir
      #copy from guest to host
      HostOS$ scp -P 2222 user@localhost:/path/to/file /destination/dir
      

    Note that to add another Guest OS, use the same technique but choose different port number on the host OS, e.g., 2223.

    • Start Headless
      # List virtual machines
      HostOS$ VBoxManage list vms
      # Start VM in headless mode
      HostOS$ VBoxManage startvm Debian7 --type headless
      # Power off VM
      HostOS$ VBoxManage controlvm Debian7 poweroff
      

9 Configure and Compile Linux Kernel (Debian)

9.1 General Step

below describes a hybrid way of compiling kernel. This means the procedure uses some manual compilation steps but also takes advantages of distribution specific tools, e.g., Debian package management. There are more 'distro-specific' ways but these seem to be the easiest and simplest … at least for me.

Note: from experience I found most people don't have problems compiling kernel, but they get frustrated because not being able to boot to the new kernel or after booting into the kernel they lose their GUI and other stuff such as VirtualBox, etc. In general if the new kernel won't boot, you still can go to the old/current one. For graphic driver, just manually build it from src (see the simple steps below for NVIDIA cards). You can also build graphic drivers using other methods but for me from src is the easiest. Can't coment about AMD/ATI driver since I don't have one – but the steps probably are similar.

  1. Obtain the latest kernel source
    wget  http://www.kernel.org/pub/linux/kernel/v2.6/linux-2.6.38.5.tar.bz2  
    tar xf linux-3.19.tar.bz2  
    cd linux-3.19
    
  2. Build the kernel (see distribution specific below)
  3. Post kernel install
    • Build Graphic driver (NVIDIA)
      sudo sh  path/where/you/save/nvidia/driver/NVIDIA-Linux-x86_64-260.19.44.run
      
    • Others
      /etc/init.d/vboxdrv setup #for Virtualbox
      ln -sf /path/to/linux-3.19.tar.bz2   /usr/src/linux  #I don't know if this is really necessary
      

9.2 Debian/Ubuntu

  1. install the packages build-essential (and kernel-package for Debian < 8)
  2. Inside the unpacked kernel source linux-3.19
    cp /boot/config-2.6.32-5-amd64  .config   #copy the config comes with your distro as template ,  it's quite important to start from a working config
    make oldconfig    (it's most likely safe to hit Enter to accept  the default selections)
    make localmodconfig:  only use this if you know what it does
    
  3. Customize the kernel
    make menuconfig    
    #See below section for some configs for optimization
    #After done,  follow the prompt to save the config changes
    
  4. Build
    export CONCURRENCY_LEVEL=5
    # if you have multicores then this allows parallel compilation. The convention setting is N+1 where N is the number of available processors 
    # Note: do this at the same terminal screen where you plan to compile the kernel -- otherwise this VARIABLE will not be known.  Also many of the build steps are done sequentially so don't expect 100% parallelism.
    # Then exec the below command to create to 2 deb files : ~linux-image~  and ~linux-headers~
    $ sudo make-kpkg clean ; sudo fakeroot make-kpkg --initrd --append-to-version=-gcmmddyy-4 --revision=3.19 kernel_image kernel_headers
    

    Note: append-to-version=-something will make a custom name for your linux version (uname -r). I use the abbreviation of my machine (gc), the date (mmddyy), and a number (4) indicating this is the fourth time I compile this kernel on this date. revision=some_number sets the version of the kernel for apt (not terribly important). I set mine the building kernel version (3.19 in this case).

    For newer Debian (e.g., 8), use

    make deb-pkg LOCALVERSION=-giaochi112017 KDEB_PKGVERSION=$(make kernelversion)-anumber.  
    #See https://debian-handbook.info/browse/stable/sect.kernel-installation.html for details.
    
  5. Go do something while it compiles – the time it takes depends on the performance of your machine and features you build.
  6. Install
    cd ..   # go to the parent directory where the built deb files are stored
    sudo dpkg -i linux-headers-3.19-gcmmddyy-4_3.19_amd64.deb linux-image-3.19-gcmmddyy-4_3.19_amd64.deb
    
    # If this is the first time you compile the kernel then (Note this is required on my Debian Squeeze -- probably not necessary for other/newer systems - though doing it shouldn't cause problem)
    cp -v /usr/share/kernel-package/examples/etc/kernel/postinst.d/initramfs /etc/kernel/postinst.d/
    
  7. Reboot and choose the new kernel from the boot menu. If the kernel fails to boot (e.g. kernel panic), that means something is wrong – perhaps you've disable some required options. Reboot with your old kernel

You might lose your pretty GUI/X log in – if that's the case then install the appropriate Graphic driver. The way I do it is manually installing it from source (see steps for NVIDIA at the top of the page) Type uname -a to see something like Linux GiaoChi 3.19-gcmmddyy-4 #1 SMP PREEMPT … x86_64 GNU/Linux which indicates you're running the new kernel

Some configs (kernel 3.19) General Setup Kernel Compression Mode : Bzip2 Control Group support Group CPU Scheduler Automatic Process Group Scheduling Optimize for size: off Processor Type and Feature . Processor family: Core 2 / newer Xeon Preemption Model : Preemptible Kernel (Low-Latency Desktop) Timer frequency : 1000 HZ In general you can turn off the stuff you don't need – e.g., if you don't use ISDN then turn off its support. If you have XYZ branded sound card then you can turn off the supports for others. I generally don't modify these settings because most are built as modules (loaded only when needed).

Below are specific configs for my machines – probably will not work for you. Power Management Support: Suspend to RAM and standby: off Hibernation: off SFI support: off (not needed for my machine) Networking Support Amateur Radio: off Plan 9: off CAN bus subsystem support: off Device Driver Parallel port support: off

ATA/ATAPI/MFM/RLL support (DEPRECATED): off (was M) ISDN support: off Telephony support: off Input device support Joysticks/Gamepads: Off Tablets: off Touchscreens: off Misc devices: off Xen virtual kb and mouse: off HW I/O ports Gameport support: off Block devices Normal floppy disk: off Sound card support ALSA SPI sound devices: off USB sound devices: off PCMCIA sound devices: off Networking device support Ethernet (10000 Mbit): off Token Ring: off PCMCIA network device: off PPP support: off SLIP support: off Graphic Support /dev/agpgart: off

Author: tnguyen@GiaoChi

Created: 2017-12-09 Sat 16:22

Emacs 24.5.1 (Org mode 8.2.10)

Validate