My Cubox server offers a lot of services and one I specially appreciate when I'm not at home is that it contains all my music I can access through HTTP(S) interface. This is really fine for Linux clients where mplayer is installed, but it's not the case for Windows : VLC refuse to play my music (which requires login/password) and Windows player doesn't support m3u playlists, so I have to play each track individually.

I started to look for streamer software and the biggest open source one is Icecast which implement SHOUTcast standard. Like for all my online services, software must be lightweight (I don't have a lot of RAM) ! Plus I don't have any sound card plugged and don't want to spare cpu bandwidth with decoding/encoding files. This is a reason why basic HTTP(S) transfert is good : files are trasfered as is. Even if it's not clearly indicated by the documentation, icecast coupled with ezstream has all the qualities I need ! I was really suprised to find how it was easy to setup !

Here is a tutorial for basic setup (with current Debian stable distribution) :

If ezstream is available in your repository

sudo apt-get install icecast2 ezstream

If not (for an ARM target)

sudo apt-get install icecast2 libshout3 libtagc0 libxml2
wget http://ftp.de.debian.org/debian/pool/main/e/ezstream/ezstream_1.0.1-1_armhf.deb
sudo dpkg -i ezstream_1.0.1-1_armhf.deb

Now, we have to configure icecast. You must edit /etc/icecast2/icecast.xml. Update (at least) :

• Admin name/address
• Server address
• Optional : server port

Some of these values has already been configured by installer. There is a lot of avaible options not needed for basic setting. After that, restart icecast :

sudo service icecast restart

Now, add a NAT rule to redirect external port (8000 by default) to your server. Then, copy an example of ezstream configuration :

cp /usr/share/doc/ezstream/examples/ezstream-minimal.xml .

and edit it (be careful, here end tag are crafted by my editor) :

<ezstream>

  <servers>
    <server>
      <hostname>127.0.0.1</hostname>
      <port>8000</port>
      <password>XXX</password>
    </server>
  </servers>

  <streams>
    <stream>
      <mountpoint>/stream.mp3</mountpoint>
      <format>MP3</format>
    </stream>
  </streams>

  <intakes>
    <intake>
      <filename>/media/MyTrack.mp3</filename>
      <stream_once>1</stream_once>
    </intake>
  </intakes>

</ezstream>

I choose to play track only once. If not set, it will be played indefinitely. Now, we can run ezstream :

ezstream -c ezstream-minimal.xml

On Windows, we can use VLC to read this stream by opening :

http://icecast.soutade.fr:8000/stream.mp3

This is a basic setup, but we can do a lot of more complex things by autogenerating config file and auto start ezsteram from a web frontend for example.

This article you're reading is hosted on my own server. This last one runs a lot of services : web, mail, database, XMPP... and to manage it I need an SSH connection which is the more secure way to connect to a remote server. But, how I can trust this connection in an hostile environment ?

Connection protocols and key exchange has greatly evolved the last 20 years, but there are still based on a root asymmetric key pair (RSA, DSA, ECDSA...). When you connect to a server for the first time, you get a message like this :

The authenticity of host 'mint.phcomp.co.uk (78.32.209.33)' can't be established.
RSA key fingerprint is 6a:de:e0:af:56:f8:0c:04:11:5b:ef:4d:49:ad:09:23.
Are you sure you want to continue connecting (yes/no)? no

This is a human readable fingerprint of the root key used to establish a connection. Personally, I don't know my server fingerprint by heart. There is some solutions to check it :

• Manually by printing it on a paper/on your phone/on a USB key
• Register it with a DNS record, but DNS server/response can be easily spoofed
• Using a public key based connection (you need to keep it on a USB key)

The better remains having the secret (key or fingerprint) somewhere you could access it. I propose in this article an other solution you can always run in an hostile environment without any previous setup.

The idea is to create a restricted user that can only run a verification script that will check fingerprint once connection is established which avoid Man In The Middle attacks !

Setup

First, we'll have to create this user named check-user :

su
useradd --create-home --no-user-group --shell /bin/rbash check-user
cd /home/check-user

You can set a password or not. I don't do it, so I cannot open a connection from external nor internal as my server always checks for password (we can still use su/sudo command). I also set a restricted shell (rbash).

Then, we have to create a key pair

su check-user
ssh-keygen
cp .ssh/id_rsa.pub .ssh/authorized_keys

You can set or not a password for this key. Then, edit .ssh/authorized_keys and add :

command="rbash check_ssh_server.sh" ssh-rsa AAAA...

Now, downloads check_ssh_server.sh in /home/check-user and set execution permissions.

Then, go to your webserver directory were you can put some downloadable files (something like /var/www) and copy SSH the private key.

cd /var/www
cp /home/check-user/.ssh/id_rsa ssh_check
chmod a+r ssh_check

Now, you can edit and run check_ssh_client.sh from any network !

How does it works ?

The client starts by establishing an SSH connection and close it immediately in order to retrieve remote fingerprint. Then, it downloads check-user SSH private key and use it to connect to the server and send the fingerprint. The only command that can be run with this key is rbash check_ssh_server.sh which get the fingerprint and compare with the ones installed on the server side. A message is then displayed which indicates if the connection is secure or not.

Scripts

check_ssh_server.sh

#!/bin/bash

target_key=`echo $SSH_ORIGINAL_COMMAND| tr -d "\r\n"`

if [ -z "${target_key}" ] ; then
    echo "Empty key provided, abort"
    exit 0
fi

for keyfile in /etc/ssh/ssh_host_*_key.pub ; do
    a=`ssh-keygen -l -f ${keyfile}|grep "${target_key}"` # To avoid print
    if [ $? -eq 0 ] ; then
        echo "Target key found, your connection is secure !"
        exit 0
    fi
done

echo "!!! WARNING !!! Key not found, the connection may not be secure"

exit 1

check_ssh_client.sh

#!/bin/bash

KEY_TRACE="Server host key:"
SSH_CHECK_KEY="https://soutade.fr/files/ssh_check/ssh_check"
REMOTE_USER="check-user"

if [ -z "$1" ] ; then
    echo "usage : $@ <ssh server>"
    exit 0
fi

echo "Retrieve remote key for $1"
tmp_file=`mktemp`
ssh -v -o "NumberOfPasswordPrompts=0" $@ >${tmp_file} 2>&1
key=`cat ${tmp_file}|grep "${KEY_TRACE}"`
key=`echo ${key}|cut -d" " -f6`
rm -f ${tmp_file}

echo "Retrieve SSH private key from ${SSH_CHECK_KEY}"
wget -O ssh_check ${SSH_CHECK_KEY}
chmod 0400 ssh_check

echo "Check for key ${key}"
ssh -l ${REMOTE_USER} -i ssh_check $@ "${key}"
echo "Cleaning"
rm -f ssh_check

For a project, I need to run java on an ARMv7 core. I could have used my Cubox server, but I don't have enough memory to run web services & java in parallel. Fortunately, I could find a SolidRun ClearFog Base board. This board is mainly network focused and only have an old dual Cortex A9 with 1GB of memory which is quick reached by today web applications, but it should be okay with a single one (I hope).

SolidRun provides documentation & ready to use OS (Debian FreeBSD, Yocto, OpenWRT), so board start should take less than one half hour. Nevertheless, documentation is split in multiple files, and there is no step by step guide. Moreover, Debian images seems crafted (we can't read main ext4 partition). After 2 hours of fails I started to turn crazy ! Here is my step by step tutorial.

The goal is to install Debian on an 8GB SDCard

1) Get an 8GB SDCard and delete all partitions thanks to gparted (or fdisk).

2) Within gparted create an ext4 partition that starts at 4MB with 8GB length

3) Download target Debian images from SolidRun resources. You should take .tar.xz file which is a compressed tar file of root filesystem.

4) Go to SDCard main partition and extract data from Debian image (using sudo)

cd /media/soutade/UUID/
sudo tar -Jxvf ~/clearfog/sr-a38x-debian-buster-20200218.tar.xz
sync

5) Update fstab with your new SDCard UUID

sudo emacs etc/fstab

6) Download U-Boot for base-sdhc variant

7) Burn it to SDCard at address 512 (first sector). We assume SDCard is /dev/sdb (check it before applying this command)

sudo dd if=u-boot-clearfog-base-sdhc.kwb of=/dev/sdb1 seek=1 bs=512
sync

8) Configure ClearFog switches to start using SDcard (Off Off On On On)

9) Insert SDCard into the board, connect serial line USB to your computer

10) Power up the board and start serial line screen

sudo screen -L /dev/ttyUSB0 115200

11) Stop U-Boot process by hit a key

12) Setup kernel bootargs. Here we setup root filesystem read/write at startup, but it should be re mounted rw by fstab. Only root argument is mandatory

setenv bootargs root=/dev/mmcblk0p1 rw
env save
boot

The board should now start with a fresh Debian ! I suggest to connect within SSH which is more comfortable to use than serial line console. Login/password is debian:debian.

Have fun !

In the beginning of October, Debian pushed a security update for libssl. After installing it, all new SSH connections fails with message (even with correct password, or root login) :

fatal: privsep_preauth: preauth child terminated by signal 31

After searching on Internet, I found that nor SSH, nor libssl were in cause. It was due to an old kernel. I was running Linux 3.14 kernel because http://repo.r00t.website is not maintained.

Fortunately, Solid Run still maintains Linux kernel source tree on Github. Next instructions are based on this page.

First, mount Cubox-i filesystem from SDcard (assume it's in /mnt/cubox).

At startup, uBoot is configured to load zImage and dtb/$dtb_file. zImage is a symbolic link allowing us to have multiple kernel in /boot, let's do the same for dtb directory :

cd /mnt/cubox
cd boot
sudo mv dtb/ 3.4.14
sudo mkdir dtbs
sudo mv 3.4.14 dtbs
sudo ln -s dtbs/3.4.14/ dtb

Next, kernel compilation. The linked page suggest to do a git clone which is very big (~3GB), I suggest to download a snapshot from Github. Now, we'll follows Solid Run instructions :

sudo apt install crossbuild-essential-armhf
cd linux_sources
export CROSS_COMPILE=arm-linux-gnueabihf-
export ARCH=arm
make imx_v7_cbi_hb_defconfig
make -j4 zImage dtbs modules

Then, install compiled files :

export INSTALL_PATH=$PWD/linux_install
export INSTALL_MOD_PATH=$PWD/modules_install
mkdir linux_install
make install modules_install dtbs_install
cp arch/arm/boot/zImage linux_install/vmlinuz-4.9.124
sudo cp -r linux_install/* /mnt/cubox/boot/
sudo cp -r modules_install/lib/modules/4.9.124/ /mnt/cubox/lib/modules/

Linux creates an image compressed with lzop which not seems to be supported by my version of uBoot, so we need to manually copy created zImage.

Modules installation can be done in one line :

sudo make modules_install INSTALL_MOD_PATH=/mnt/cubox/

Optionally, you can export headers :

sudo make headers_install INSTALL_HDR_PATH=/mnt/cubox/usr/local/include

Switch kernel

cd /mnt/cubox
sudo rm dtb
sudo ln -s dtbs/4.9.124/ dtb
sudo rm zImage
sudo ln -s vmlinuz-4.9.124 zImage
sync

Unmount and unplug SDcard. Power up. It should now run new Linux kernel !

Solid Run also have a repository for a Debian package for kernel, but for now I didn't saw any binary repository available on Internet.

Warning, Github kernel make my server crash a lot of time due to an error in ext4/fs driver. I compiled a vanilla kernel, from linux-4.19.y branch (same as Debian stable one). Use the same instructions for compilation (just add dtbs_install to make install command). My .config is available here. I didn't test HDMI, Bluetooth nor IR (red LED is off). Last thing : root partition is now on /dev/mmcblk1p1, don't forget to update kernel command line !

Enabling serial console

The serial console seems to not work anymore. To enable it, first edit /etc/inittab and add at then end :

1:2345:respawn:/sbin/getty -L ttymxc0 115200 vt100

Then, we need to enable getty with SystemD to have login prompt at startup :

sudo systemctl enable "getty@ttymxc0"

Finally, we need to update kernel command line. Edit /boot/boot.cmd and put :

consoleconsole=ttymxc0,115200n8

Build boot.scr from boot.cmd, documentation here:

mkimage -C none -A arm -T script -d boot.cmd boot.scr
sync

Another day, another script. This one helps to compute the maximum stack usage of a C program. In facts, it combines the output of cflow and GCC GNAT to find the heaviest path used (which is not necessary the deepest). The first one compute target software call graph while option -fstack-usage of GCC creates .su files containing stack usage of all functions.

Targets software are simple embedded software. This script is a simple base not intended to run on all cases, handle dynamic stack nor recursive functions (if you wish to add it...).

A file version is available here.

#!/usr/bin/env python

import os
import re
import argparse

class SUInfo:
    def __init__(self, filename, line, func_name, stack_size):
        self.filename = filename
        self.line = line
        self.func_name = func_name
        self.stack_size = stack_size

    def __str__(self):
        s = '%s() <%s:%s> %d' % (self.func_name, self.filename, self.line, self.stack_size)
        return s

class FlowElement:
    def __init__(self, root, depth, stack_size, suinfo):
        self.root = root
        self.depth = depth
        self.stack_size = stack_size
        self.suinfo = suinfo
        self.childs = []

    def append(self, suinfo):
        self.childs.append(suinfo)

    def __str__(self):
        spaces = '    ' * self.depth
        su = self.suinfo
        res = '%s-> %s() %d <%s:%d>' % (spaces, su.func_name, su.stack_size,
                                        su.filename, su.line)
        return res

def display_max_path(element):
    print('Max stack size %d' % (element.stack_size))
    print('Max path :')
    res = ''
    while element:
        res = str(element) + '\n' + res
        element = element.root
    print(res)

cflow_re = re.compile(r'([ ]*).*\(\) \<.* at (.*)\>[:]?')

def parse_cflow_file(path, su_dict):
    root = None
    cur_root = None
    current = None
    cur_depth = 0
    max_stack_size = 0
    max_path = None
    with open(path) as f:
        while True:
            line = f.readline()
            if not line: break
            match = cflow_re.match(line)
            if not match: continue

            spaces = match.group(1)
            # Convert tab into 4 spaces
            spaces = spaces.replace('\t', '    ')
            depth = len(spaces)/4
            filename = match.group(2)
            (filename, line) = filename.split(':')
            filename = '%s:%s' % (os.path.basename(filename), line)

            suinfo = su_dict.get(filename, None)
            # Some functions may have been inlined
            if not suinfo:
                # print('WARNING: Key %s not found in su dict"' % (filename))
                continue

            if not root:
                root = FlowElement(None, 0, suinfo.stack_size, suinfo)
                cur_root = root
                current = root
                max_path = root
                max_stack_size = suinfo.stack_size
            else:
                # Go back
                if depth < cur_depth:
                    while cur_root.depth > (depth-1):
                        cur_root = cur_root.root
                # Go depth
                elif depth > cur_depth:
                    cur_root = current
                cur_depth = depth
                stack_size = cur_root.stack_size + suinfo.stack_size
                element = FlowElement(cur_root, cur_depth,
                                      stack_size,
                                      suinfo)
                current = element
                if stack_size > max_stack_size:
                    max_stack_size = stack_size
                    max_path = current
                cur_root.append(element)
    display_max_path(max_path)

su_re = re.compile(r'(.*)\t([0-9]+)\t(.*)')

def parse_su_files(path, su_dict):
    for root, dirs, files in os.walk(path):
        for sufile in files:
            if sufile[-2:] != 'su': continue
            with open(os.path.join(path, sufile)) as f:
                while True:
                    line = f.readline()
                    if not line: break
                    match = su_re.match(line)
                    if not match:
                        # print('WARNING no match for "%s"' % (line))
                        continue
                    infos = match.group(1)
                    (filename, line, size, function) = infos.split(':')
                    stack_size = int(match.group(2))
                    key = '%s:%s' % (filename, line)
                    su_info = SUInfo(filename, int(line), function, stack_size)
                    su_dict[key] = su_info


if __name__ == '__main__':
    optparser = argparse.ArgumentParser(description='Max static stack size computer')
    optparser.add_argument('-f', '--cflow-file', dest='cflow_file',
                           help='cflow generated file')
    optparser.add_argument('-d', '--su-dir', dest='su_dir',
                           default='.',
                           help='Directory where GNAT .su files are generated')
    options = optparser.parse_args()

    su_dict = {}

    parse_su_files(options.su_dir, su_dict)
    parse_cflow_file(options.cflow_file, su_dict)

Usage & example

Let's take this simple software as example.

First, compile your software using -fstack-usage options in CFLAGS. It will creates an .su file for each object file. Then, launch cflow on your software. Finally, call my script.

mkdir test
cd test
gcc -fstack-usage gget.c -lpthread -lcurl
cflow gget.c > cflow.res
./cflow.py -f cflow.res

Result:

Max stack size 608
Max path :
-> main() 352 <gget.c:493>
    -> do_transfert() 160 <gget.c:228>
        -> progress_cb() 96 <gget.c:214>