Showing posts with label vnc. Show all posts
Showing posts with label vnc. Show all posts

Sunday, December 20, 2020

Buid docker image for M1 Mac with development tools and vnc

(1) With the release of docker preview for Mac M1, it is now possible to build some arm64 images.
Docker M1 preview
https://desktop.docker.com/mac/m1preview/Docker-AppleSilicon-Preview7.dmg

(2) Install the above preview and then use terminal the tools. The build script is based on Android SDK docker image for x64 and modify it for arm64. Moreover, swift 5.3.1, clang 10.0.0, kotlin 1.4.21, gradle 6.7.1, gcc 9.3.0, java 11, firefox, visual studio code and codeblocks are added for development purpose.

(3) Shell script run in Mac docker info # check if docker is installed properly in Mac M1
mkdir -p ${HOME}/android_sdk_vnc
cd ${HOME}/android_sdk_vnc
curl -O https://raw.githubusercontent.com/thyrlian/AndroidSDK/master/android-sdk/sshd-banner
curl -O https://raw.githubusercontent.com/thyrlian/AndroidSDK/master/android-sdk/supervisord.conf
curl -O https://raw.githubusercontent.com/thyrlian/AndroidSDK/master/android-sdk/vnc/supervisord_vncserver.conf
curl -O https://raw.githubusercontent.com/thyrlian/AndroidSDK/master/android-sdk/vnc/vncpass.sh
curl -O https://raw.githubusercontent.com/thyrlian/AndroidSDK/master/android-sdk/vnc/watchdog.sh

(4) Create the following Dockfile
Dockerfile    Select all
cat >Dockerfile <<'HEREEOF' # ====================================================================== # # Android SDK Docker Image # ====================================================================== # # Base image # ---------------------------------------------------------------------- # FROM arm64v8/ubuntu:20.04 # install essential tools RUN apt-get update && \ apt-get dist-upgrade -y && \ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends git wget wget2 zip unzip && \ apt-get install -y build-essential vim # install Android SDK RUN DEBIAN_FRONTEND=noninteractive apt-get install -y android-sdk && \ rm -f /usr/lib/android-sdk/build-tools/27.0.1 # install Android SDK cmdline tools ARG ANDROID_SDK_VERSION=6858069 ENV ANDROID_SDK_ROOT /usr/lib/android-sdk RUN wget2 -q https://dl.google.com/android/repository/commandlinetools-linux-${ANDROID_SDK_VERSION}_latest.zip && \ unzip commandlinetools*linux*${ANDROID_SDK_VERSION}*.zip && \ mv cmdline-tools tools && \ mkdir -p ${ANDROID_SDK_ROOT}/cmdline-tools && \ mv tools ${ANDROID_SDK_ROOT}/cmdline-tools/ && \ (cd /usr/lib/android-sdk/cmdline-tools; ln -s tools latest) && \ rm -f commandlinetools*linux*.zip # download and install Gradle ARG GRADLE_VERSION=6.7.1 ARG GRADLE_DIST=bin RUN wget2 -q https://services.gradle.org/distributions/gradle-${GRADLE_VERSION}-${GRADLE_DIST}.zip && \ rm -fr /usr/share/gradle && \ unzip gradle*.zip -d /usr/share && \ rm gradle*.zip && \ mv /usr/share/gradle-${GRADLE_VERSION} /usr/share/gradle # download and install Kotlin compiler ARG KOTLIN_VERSION=1.4.10 RUN wget2 -q https://github.com/JetBrains/kotlin/releases/download/v${KOTLIN_VERSION}/kotlin-compiler-${KOTLIN_VERSION}.zip && \ unzip *kotlin*.zip && \ mv kotlinc /usr/share && \ rm *kotlin*.zip # install swift5 compiler RUN wget -qO- https://packagecloud.io/install/repositories/swift-arm/release/script.deb.sh | bash RUN apt install -y swiftlang # setup adb server EXPOSE 5037 # set the environment variables ARG JDK_VERSION=1.11.0 ENV JAVA_HOME /usr/lib/jvm/java-${JDK_VERSION}-openjdk-arm64 ENV GRADLE_HOME /usr/share/gradle ENV KOTLIN_HOME /usr/share/kotlinc ENV ANDROID_SDK_ROOT /usr/lib/android-sdk ENV ANDROID_HOME /usr/lib/android-sdk ENV ANDROID_TOOLS /usr/lib/android-sdk/tools ENV ANDROID_PLATFORMS /usr/lib/android-sdk/platforms ENV ANDROID_PLATFORM_TOOLS /usr/lib/android-sdk/platform-tools ENV ANDROID_BUILD_TOOLS /usr/lib/android-sdk/build-tools ENV ANDROID_CMDLINE_TOOLS /usr/lib/android-sdk/cmdline-tools/tools ENV PATH ${PATH}:${GRADLE_HOME}/bin:${KOTLIN_HOME}/bin:${ANDROID_BUILD_TOOLS}/debian:${ANDROID_TOOLS}/bin:${ANDROID_PLATFORM_TOOLS}:${ANDROID_CMDLINE_TOOLS}/bin:${ANDROID_SDK_ROOT}/emulator # install system image and accept licence, installation of emulator will fail for arm64 system RUN yes Y | ${ANDROID_CMDLINE_TOOLS}/bin/sdkmanager --install "platform-tools" "system-images;android-24;google_apis;arm64-v8a" "platforms;android-24" "build-tools;30.0.3" && \ wget https://github.com/qhuyduong/arm_adb/releases/download/v1.0.39-aarch64/adb && \ chmod +x adb && mv adb /usr/lib/android-sdk/platform-tools/ # install and configure SSH server EXPOSE 22 ADD sshd-banner /etc/ssh/ # ADD authorized_keys /tmp/ RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends openssh-server supervisor locales && \ mkdir -p /var/run/sshd /var/log/supervisord && \ locale-gen en en_US en_US.UTF-8 && \ apt-get remove -y locales && apt-get autoremove -y && \ FILE_SSHD_CONFIG="/etc/ssh/sshd_config" && \ echo "\nBanner /etc/ssh/sshd-banner" >> $FILE_SSHD_CONFIG && \ echo "\nPermitUserEnvironment=yes" >> $FILE_SSHD_CONFIG && \ ssh-keygen -q -N "" -f /root/.ssh/id_rsa && \ FILE_SSH_ENV="/root/.ssh/environment" && \ touch $FILE_SSH_ENV && chmod 600 $FILE_SSH_ENV && \ printenv | grep "JAVA_HOME\|GRADLE_HOME\|KOTLIN_HOME\|ANDROID_SDK_ROOT\|LD_LIBRARY_PATH\|PATH" >> $FILE_SSH_ENV && \ echo "\nauth required pam_env.so envfile=$FILE_SSH_ENV" >> /etc/pam.d/sshd && \ FILE_AUTH_KEYS="/root/.ssh/authorized_keys" && \ touch $FILE_AUTH_KEYS && chmod 600 $FILE_AUTH_KEYS && \ for file in /tmp/*.pub; \ do if [ -f "$file" ]; then echo "\n" >> $FILE_AUTH_KEYS && cat $file >> $FILE_AUTH_KEYS && echo "\n" >> $FILE_AUTH_KEYS; fi; \ done && \ (rm /tmp/*.pub 2> /dev/null || true) # install and configure VNC server ENV USER root ENV DISPLAY :1 EXPOSE 5901 ADD vncpass.sh /tmp/ ADD watchdog.sh /usr/local/bin/ ADD supervisord_vncserver.conf /etc/supervisor/conf.d/ RUN apt-get update && \ DEBIAN_FRONTEND=noninteractive apt-get install -y --no-install-recommends xfce4 xfce4-goodies xfonts-base dbus-x11 tightvncserver expect && \ apt-get install -y --no-install-recommends firefox codeblocks && \ chmod +x /tmp/vncpass.sh; sync && \ /tmp/vncpass.sh && \ rm /tmp/vncpass.sh && \ apt-get remove -y expect && apt-get autoremove -y && \ FILE_SSH_ENV="/root/.ssh/environment" && \ echo "DISPLAY=:1" >> $FILE_SSH_ENV # install and patch Visual Studio Code RUN apt update && apt install -y libsecret-1-0 libxss1 libgbm1 && apt-get clean RUN wget -O code_arm64.deb https://aka.ms/linux-arm64-deb && dpkg -i code_arm64.deb && rm code_arm64.deb RUN sed 's/BIG-REQUESTS/_IG-REQUESTS/' /usr/lib/aarch64-linux-gnu/libxcb.so.1.1.0 | tee /usr/share/code/libxcb.so.1 > /dev/null ADD supervisord.conf /etc/supervisor/conf.d/ CMD ["/usr/bin/supervisord"] HEREEOF


(5) build the Dockfile
docker build -f Dockerfile -t arm64v8/android-sdk-vnc .


(6) Copy Mac host id_rsa.pub to authorized_keys and start the container
cp ~/.ssh/authorized_keys .
cat ~/.ssh/id_rsa.pub >> authorized_keys


# To start the container as daemon
docker run -d --name android-sdk-vnc -p 5901:5901 -p 2222:22 -p 5037:5037 \
-v $(pwd)/authorized_keys:/root/.ssh/authorized_keys arm64v8/android-sdk-vnc

# To connect the container via ssh
ssh root@<IP address of your Mac M1> -p 2222

# To stop the container
docker container stop android-sdk-vnc


(7) To connect to the vnc of the docker image after starting the container
Use the Mac Finder "Go" -> "Connect To Server" and enter "vnc://0.0.0.0:5901"
For other machines in the network enter "vnc://<IP address of your Mac M1>:5901"

The passwords are in the vncpass.sh when building which are
password="android"
password_view_only="docker"

(8) There is a common problem if you are connecting remotely using Mac Terminal, so the solution is to not forward your locale. Edit /etc/ssh/ssh_config and comment out SendEnv LANG LC_* line.

(9) Testing swift
# attach to the running container
docker exec -it android-sdk-vnc bash
# test
mkdir -p ${HOME}/Projects
cd ${HOME}/Projects
git clone https://github.com/apple/example-package-dealer.git
cd example-package-dealer
swift run Dealer




If you need port forwarding on macOS, see this article https://www.papercut.com/kb/Main/MacPortForwarding

Saturday, December 19, 2020

How to setup debian vm with VNC on New Mac Mini M1

(1) Download the ubuntu kernel and images
cd ${HOME}
mkdir debianvm
curl -o - https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-vmlinuz-generic | gunzip > debianvm/vmlinuz
curl -o debianvm/initrd https://cloud-images.ubuntu.com/releases/focal/release/unpacked/ubuntu-20.04-server-cloudimg-arm64-initrd-generic
curl -o - https://cloud-images.ubuntu.com/releases/focal/release/ubuntu-20.04-server-cloudimg-arm64.tar.gz | tar zxvC debianvm

(2) download the raspbian arm64 image
curl -OL https://downloads.raspberrypi.org/raspios_arm64/images/raspios_arm64-2020-08-24/2020-08-20-raspios-buster-arm64.zip
unzip 2020-08-20-raspios-buster-arm64.zip -d debianvm

(3) Copy and expand the disk to say 38G (Don't use this method to expand disk on external HD)
cp debianvm/focal-server-cloudimg-arm64.img debianvm/raspberry-arm64.img
dd if=/dev/zero of=debianvm/raspberry-arm64.img seek=40000000 obs=1024 count=0

(4) Compile and use the latest vftool which is enhanced to mount multiple disks.
cd ${HOME}
git clone https://github.com/evansm7/vftool
cd vftool && xcodebuild

(5) start the vm say
screen -S vm
cd ${HOME}/debianvm
${HOME}/vftool/build/Release/vftool -k vmlinuz -i initrd -d focal-server-cloudimg-arm64.img -d raspberry-arm64.img -d 2020-08-20-raspios-buster-arm64.img -m 2048 -a "console=hvc0"
Detach the screen using Ctrl-A D

(6) start the tty say
screen -S tty /dev/ttys001

(7) prepare the ubuntu cloud image
mkdir /mnt
mount /dev/vda /mnt
chroot /mnt

touch /etc/cloud/cloud-init.disabled

echo 'root:root' | chpasswd

ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

# add fixed IP
cat <<EOF > /etc/netplan/01-dhcp.yaml 
network:
    renderer: networkd
    ethernets:
        enp0s1:
            dhcp4: true
            addresses: [192.168.64.33/24]
    version: 2
EOF

# add IP when login prompt
cat <<EOF >> /etc/issue
enp0s1: \4

EOF

exit
umount /dev/vda



(8) copy the rootfs to raspberry-arm64.img
#Detach the screen using Ctrl-A D
screen -r vm # resume vm screen # stop the vm by ctrl-C and restart the ubuntu vm
cd ${HOME}/debianvm
${HOME}/vftool/build/Release/vftool -k vmlinuz -i initrd -d focal-server-cloudimg-arm64.img -d raspberry-arm64.img -d 2020-08-20-raspios-buster-arm64.img -m 2048 -a "console=hvc0 root=/dev/vda"

#Detach the screen using Ctrl-A D
#Notice the tty number when starting the vm and start the tty say
screen -S tty /dev/ttys001

# login with id root and password root
# and copy the rootfs to the expanded disk raspberry-arm64.img
dd if=/dev/vdc2 of=/dev/vdb bs=64K conv=noerror,sync
exit

(9) prepare the raspberry-arm64 image
# stop the vm and restart the vm
# Control-A D to detach the tty screen # resume vm screen screen -r vm # Control-C to stop the vm cd ${HOME}/debianvm
${HOME}/vftool/build/Release/vftool -k vmlinuz -i initrd -d raspberry-arm64.img -m 2048 -a "console=hvc0"
mkdir /mnt
mount /dev/vda /mnt
chroot /mnt

echo 'root:root' | chpasswd

#!!! delete or comment out the lines in /etc/fstab, or there will be "A start job is running on /dev/disk/.." blocking!!
cat <<EOF > /etc/fstab
proc            /proc           proc    defaults          0       0
#PARTUUID=ad09722e-01  /boot           vfat    defaults          0       2
#PARTUUID=ad09722e-02  /               ext4    defaults,noatime  0       1
EOF

ssh-keygen -f /etc/ssh/ssh_host_rsa_key -N '' -t rsa
ssh-keygen -f /etc/ssh/ssh_host_dsa_key -N '' -t dsa
ssh-keygen -f /etc/ssh/ssh_host_ecdsa_key -N '' -t ecdsa
ssh-keygen -f /etc/ssh/ssh_host_ed25519_key -N '' -t ed25519

mkdir -p /etc/network/interfaces.d/
cat <<EOF > /etc/network/interfaces.d/enp0s1 
allow-hotplug enp0s1
iface enp0s1 inet dhcp
auto lo
iface lo inet loopback
EOF

cat <<EOF >> /etc/dhcpcd.conf
# add fixed IP
interface enp0s1
static ip_address=192.168.64.10/24
EOF

# put mac host ${HOME}/.ssh/id.rsa.pub key here
mkdir -p /root/.ssh
cat <<EOF > /root/.ssh/authorized_keys
ssh-rsa <mac id.rsa.pub key here>
EOF

# add IP when login prompt
cat <<EOF >> /etc/issue
enp0s1: \4

EOF

exit
umount /dev/vda

(10) login raspberry OS arm64 with id root and password root
# stop the vm and restart the vm with
cd ${HOME}/debianvm
${HOME}/vftool/build/Release/vftool -k vmlinuz -i initrd -d raspberry-arm64.img -m 2048 -a "console=hvc0 root=/dev/vda rw"
# set date say
date -s "29 DEC 2020 14:54:00"
resize2fs /dev/vda
dpkg-reconfigure tzdata # set timezone
apt update
systemctl disable rpi-eeprom-update.service
systemctl disable rng-tools.service

(11) install utilities in raspberry os that can handle qcow2 disk image conversion and resize
apt install qemu-utils libguestfs-tools
hostname -I # get ip address

(13) Install Realvncserver and connect from Mac
passwd pi # set user pi password
# login as user pi
login pi
# Use raspi-config to enable SSH and VNC
sudo apt install realvnc-vnc-server
sudo raspi-config
# Choose 5 Interfacing Options -> P2 SSH
# Choose 5 Interfacing Options -> P3 VNC

# enter vnc key, can put this in /etc/rc.local without sudo
sudo vnclicense -add VKUPN-MTHHC-UDHGS-UWD76-6N36A

# setup vncpasswd for user mode, only user mode can run browser
vncpasswd -user

# edit config file to have these two lines for vnc password auth
echo -e "Authentication=VncAuth" | tee -a ~/.vnc/config.d/vncserver-x11
echo -e "Encryption=PreferOff" | tee -a ~/.vnc/config.d/vncserver-x11
cp ~/.vnc/config.d/vncserver-x11 ~/.vnc/config.d/Xvnc
# start service
vncserver

#put these in rc.local to automate after reboot, Raspberry Pi hardware has bundled vnclicense and no need to add it
/usr/bin/vnclicense -add VKUPN-MTHHC-UDHGS-UWD76-6N36A
sudo -H -u pi /usr/bin/vncserver

#check which port is listening
netstat -tlpn | grep -i listen

This would configure VNC in Virtual Mode
# Screen Sharing Connect from Mac
# In Finder, Go -> Connect to Server, enter IP address and port number of the vm e.g.
# Enter the vncpassword to connect, and connection needs port number
vnc://192.168.64.10:5901

For configuring VNC in Service Mode, please refer to https://www.raspberrypi.org/documentation/remote-access/vnc/
For differences in these modes, please refer to https://help.realvnc.com/hc/en-us/articles/360002253238-Understanding-VNC-Server-Modes
#Start or stop the service with:
# systemctl (start|stop) vncserver-x11-serviced.service
#Mark or unmark the service to be started at boot time with:
# systemctl (enable|disable) vncserver-x11-serviced.service
#sudo systemctl start vncserver-x11-serviced.service
For Service Mode the connection from Mac is (no port number), vnc://192.168.64.10

If you need port forwarding on macOS, so that you can vnc connect from other machines on the LAN to this vm, see this article https://www.papercut.com/kb/Main/MacPortForwarding That is run
echo "
rdr pass on lo0 inet proto tcp from any to self port 5910 -> 192.168.64.10 port 5901
rdr pass on en0 inet proto tcp from any to any port 5910 -> 192.168.64.10 port 5901
rdr pass on wlan0 inet proto tcp from any to any port 5910 -> 192.168.64.10 port 5901
" | sudo pfctl -ef -
Then, test connect from other machines to :5901 on the LAN.

# To install Visual Studio Code
wget -O code_arm64.deb https://aka.ms/linux-arm64-deb && sudo dpkg -i code_arm64.deb && rm code_arm64.deb
sed 's/BIG-REQUESTS/_IG-REQUESTS/' /usr/lib/aarch64-linux-gnu/libxcb.so.1.1.0 | sudo tee /usr/share/code/libxcb.so.1 > /dev/null

Demo - VNC to rapberrypi desktop vm screen capture

P.S. Only vm can run realvncserver, docker environment can only run lightweight vncserver e.g. tightvncserver

(14) For qcow2 image, the conversion to raw image is
wget https://cloud.debian.org/images/cloud/buster/20201214-484/debian-10-generic-arm64-20201214-484.qcow2
qemu-img dd -f qcow2 -O raw bs=64k if=debian-10-generic-arm64-20201214-484.qcow2 of=debian-10.img
# scp debian-10.img to the Mac Mini host and prepare a new expanded debian-arm64.img similar to the raspberry-arm64.img as above
# After restarting vm and mounting images, the dd command to extract rootfs is
# Assume debian-arm64.img is vdb and debian-10.img is vdc
dd if=/dev/vdc1 of=/dev/vdb bs=64K conv=noerror,sync
# For debian image, add this to /etc/hosts 127.0.1.1 debian