Remote backup on a multi-user system

Having multiple users with directories set with non-readable files and directories is a problem when backing up data via ssh to or from a remote server. This is because the remote backup user cannot read all files and directories in the local system and the local root user cannot write to the sshfs mount or scp to the remote backup server – because I configure my servers to reject root login.

Therefore, it is necessary to use access control lists (ACL) to allow a remote backup user to have full read-only permission on SSH/NFS mounted filesystems that need to be backed up.

To accomplish this, after creating a local backup user, set read-only permissions on the desired directory:

sudo setfacl -m u:backup:r-X -R /data

-R applies the r-X attribute recursively to user “backup” in the local system.

To check the ACL, use:

getfacl /data

In the remote backup system, create a backup user who can ssh into the system to be backed up.

Use sshfs to mount /data/ in the remote system. The ACL will allow the backup user to read all files and directories.

One caveat is that it is necessary to issue the setfacl command every time before backing up. If a user sets chmod 700 or chmod 600 in a file or directory, that directory or file will become unreadable by the backup user. So it’s important to set the read permission to the backup user every time to make sure everything is readable during backup.

Advertisements

Uploading one directory recursively using sftp

I keep forgetting how to upload a directory recursively using the Linux sftp command line program.

The trick is to manually create the target directory in the sftp server then do put -r .

Example, to recursively upload the local directory “test” that contains subdirectories and files:

sftp> mkdir test
sftp> put -r test

Source

Manage syncthing remotely via ssh

I have two webservers that have no GUI and are managed remotely via ssh.

In order to synchronize data across the two servers in “real time”, I opted for syncthing. However, syncthing configuration is done on a web browser GUI which isn’t installed in my servers.

The way I found to have access to the GUI of each server was to create an ssh tunnel and tunnel my local web browser traffic into the remote servers:

ssh -L 12341:localhost:8384 webserver1
ssh -L 12342:localhost:8384 webserver2

This commands will allow you to tunnel traffic from localhost port 12341 into webserver1 port 8384 (and vice-versa) and from localhost port 12342 into webserver2 port 8384 (and vice-versa). In other words, when you point your browser to localhost:12341 in your local computer, it will connect to webserver1:8384 and localhost:12342 will connect o webserver2:8384.

Let’s encrypt Apache installation problem (Ubuntu 14.04)

Let’s encrypt is a cool initiative that provides free SSL certificates to enable https everywhere.

To make the certificate installation process as painless as possible, they provide automated tools for many OSes.

Download their tool, run it on your Apache server and you have https without those scary warnings that we get with unpaid SSL certificates.

Their tool mostly works. Running certbot-auto on Ubuntu 14.04 gave me the error below:

Error in checking parameter list: AH00526: Syntax error on line 115 of /etc/apache2/sites-enabled/000-default-le-ssl.conf:
SSLCertificateFile: file ‘/etc/apache2/insert_cert_file_path’ does not exist or is empty

The solution is to enable the Apache SSL module before running certbot-auto.

a2enmod ssl
./certbot-auto 

After doing this, everything worked as expected.

Credit: https://github.com/certbot/certbot/issues/1328

RStudio freezes when opening file

After I had to kill RStudio because the process it was running ate all the temporary space, RStudio would freeze when trying to open an R script.

In addition, when running a third party R function, I got this strange error saying it couldn’t change directory.

The fix was deleting this file:

rm ~/.config/QtProject.conf.lock

 

Credit: https://support.rstudio.com/hc/en-us/community/posts/207006938-RStudio-freezes-when-saving-or-loading

Installing Ubuntu on an Acer C720P chromebook

There is a lot of information available regarding installation of single-boot Ubuntu on an Acer C720P but here I’m posting what works for me in Jan 2017.

I am not going to explain how to prepare the Chromebook for  installation of Ubuntu, I’m just posting notes on certain configurations that work for me:

Basic hardware functionality

  1. Elantech touchpad: the newest kernels recognize the touchpad perfectly (>= 4.8). Previously, only the GalliumOS kernel made the touchpad usable for me.
    1. Download the amd64 kernel from here and install: http://kernel.ubuntu.com/~kernel-ppa/mainline/
    2. For example, for 4.9, you need to download   linux-headers-4.9.0-040900_4.9.0-040900.201612111631_all.deb
      linux-headers-4.9.0-040900-generic_4.9.0-040900.201612111631_amd64.deb
      linux-image-4.9.0-040900-generic_4.9.0-040900.201612111631_amd64.deb
    3. Install using dpkg -i linux-*.deb
  2. Touchscreen: works with any kernel.
  3. Microphone: does not work
  4. Speakers: work with any kernel

Enabling keyboard functions and keys

  1. Install programs to control brightness and map keys:
    1. apt-get install xbindkeys xdotool xbacklight
    2. create ~/.xbindkeysrc with these contents to map keys (customize to your needs)
      "xdotool keyup F1; xdotool key alt+Left"
      F1
      "xdotool keyup F2; xdotool key alt+Right"
      F2
      "xdotool keyup F3; xdotool key ctrl+r"
      F3
      "xdotool keyup F4; xdotool key F11"
      F4
      "xdotool keyup F5; gnome-screenshot -f $HOME/Pictures/Screenshot_$(date +%F_%T).png"
      F5
      "xdotool keyup F6; xbacklight -dec 3"
      F6
      "xdotool keyup F7; xbacklight -inc 3"
      F7
      "xdotool keyup F8; pactl set-sink-mute alsa_output.pci-0000_00_1b.0.analog-stereo toggle"
      F8
      #"xdotool keyup F9; pactl set-sink-volume alsa_output.pci-0000_00_1b.0.analog-stereo -10%"
      "xdotool keyup F9; amixer -D pulse sset Master 5%-"
      F9
      #"xdotool keyup F10; pactl set-sink-volume alsa_output.pci-0000_00_1b.0.analog-stereo +10%"
      "xdotool keyup F10; amixer -D pulse sset Master 5%+"
      F10
      
      "xdotool keyup control+shift+Down Arrow; xdotool key Page_Down; xdotool keydown shift"
      control+shift+Down Arrow
      
      "xdotool keyup control+shift+Up Arrow; xdotool key Page_Up"
      control+shift+Up Arrow
      
      "xdotool keyup control+shift+Left Arrow; xdotool key Home"
      control+shift+Left Arrow
      
      "xdotool keyup control+shift+Right Arrow; xdotool key End"
      control+shift+Right Arrow
      
      "xdotool keyup XF86PowerOff; xdotool key Delete"
      XF86PowerOff
    3. Notice that I disable the power button and map it to Del. To shutdown the computer, I use the shutdown button in Gnome panel.

Using a lightweight Desktop (Openbox):

  1. Even Gnome 2 uses too much memory for my likings. Openbox uses < 350 MB idle and Gnome ~ 600 MB idle. 250 MB in a 2 GB system is a lot for me. With Openbox I get a light Gnome-like desktop.
  1. Install Openbox:
    sudo apt-get install openbox
  2. Install gnome-session-fallback:
    1. sudo apt-get install gnome-session-fallback
    2. I use some Gnome applications.
    3. Also, Chromebooks don’t have a “Super” key, which is required in Openbox to add buttons to gnome-panel. In Gnome, you can add buttons by clicking the right mouse button and Alt only. So I login to Gnome, configure gnome-panel, logout and login to Openbox.
    4. Alternatively, plug in a keyboard and click Super+Alt and the right mouse key on the Gnome panel in Openbox to add items). But this means you always depend on an external keyboard to configure Gnome panel in Openbox.
  3. Install volti to control the sound volume. Gnome’s won’t work:
    sudo apt-get install volti
  4. Configure gnome-panel. The native Gnome indicator applet won’t work in Openbox, which means no network manager, no clock, etc.To configure Gnome panel in Openbox without plugging an external keyboard: login to Gnome (metacity) and configure the gnome-panel with the mouse, adding:
    1. A custom application launcher that will be the shutdown button. Select a shutdown icon to be displayed.
      1. Create this script and configure the shutdown button in Gnome panel to call this script:
        #!/bin/bash
        gmessage "Are you sure you want to shut down your computer?" -center -title "Take action" -font "Sans bold 10" -default "Cancel" -buttons "_Cancel":1,"_Log out":2,"_Reboot":3,"_Shut down":4,"_Suspend":5 >/dev/null
        case $? in
        1)
        echo "Exit";;
        2)
        killall openbox;;
        3)
        shutdown -r now;;
        4)
        shutdown -h now;;
        5)
        pm-suspend
        esac
      2. Remove the bottom panel bar for more screen real estate.
      3. Add a window selector to the top panel bar.
      4. Add desired application shortcuts.
      5. Add a clock.
    2. Logout from Gnome and login to Openbox.
    3. In a terminal, create .config/openbox/autostart.sh
    4. Install gxmessage to show the shutdown dialog when you press the shutdown button:
      apt-get install gxmessage
    5. Configure sudoers to allow any user to shutdown:
      1. sudo visudo
      2. Insert this at the end of the file:
        %users ALL=NOPASSWD: /usr/bin/shutdown
    6. I’m still working on how to make pm-suspend work without sudo.
  5. Configure start up applications: Edit .config/openbox/autostart.sh to start several applications upon login:
    gnome-panel & # openbox has no panels, you need to start one
    volti & # sound control
    nm-applet & # for network management
    xbindkeys # for key mapping to the weird Chromebook keyboard
    xscreensaver &
    synclient ClickFinger3=2 # to enable right click on the touchpad (tap the touchpad with 2 fingers simultaneously)
    synclient TapButton3=2 # to enable pasting with the touchpad (tap the touchpad with 3 fingers simultaneously)
    earlyoom # aggressively kill programs when there's no RAM available, see below
    
  6. Enable a screensaver:
    1. gnome-screensaver doesn’t work.
    2. Install xscreensaver instead and remove gnome-screensaver:
      sudo apt-get remove gnome-screensaver
      sudo apt-get install xscreensaver
    3. run xscreensaver-demo to configure the screensaver.
  7. Backup .config/ so anytime you have to reinstall you don’t need to reconfigure Gnome panel and start up applications.

Other configurations

  1. SSD settings: configure /etc/fstab with configurations that are more gentle on your SSD: Add discard,noatime,errors=remount-ro to /dev/sda1:
    /dev/sda1   /               ext4    discard,noatime,errors=remount-ro 0       1
  2. Swap: I used to disable swap completely and let programs crash if they used too much memory. However, there seems to be a bug in 16.04 and sometimes kswapd0 runs at 100% CPU for several minutes when there’s not swap. So now I:
    1. create a very small swap (10 MB)
    2. decrease swappiness (sudo sysctl vm.swappiness=10)
    3. use earlyoom (https://github.com/rfjakob/earlyoom) to aggressively kill programs. I add it to .config/openbox/autostart