leethax.org

Compiling PHP SQLite extension from source with encryption features

2020-02-06T14:13:00+00:00

I am writing this post because, as often this is not documented anywhere, but if you purchased a SQLite Encryption Extension (SEE) License and want to use it within PHP, you will have to recompile the PHP SQLite3 extension from source.

What you will need

PHP source code for your current version of PHP, found on https://www.php.net/downloads.php
SEE source code found on the version above
php-dev tooling for your distro, (apt install php-dev for Debian/Ubuntu)

Prepare the source dirs

I am assuming php 7.2.27 here but correct with the version that you’re using.

mkdir ~/src
tar zxf see-sources.tar.gz -C src
tar zxf php-7.2.27.tar.gz -C src

Prepare SEE source

First of all the SQLite amalgamation with SEE needs to be prepared, go to your SEE source repository and issue the following:

cd ~/src/see-sources/
cat see-prefix.txt sqlite3.c see.c >sqlite3-see.c
cp sqlite3-see.c ~/src/php-7.2.27/src/ext/sqlite3/libsqlite/sqlite3.c
cp sqlite3.h ~/src/php-7.2.27/ext/sqlite3/libsqlite/

You have now successfully replaced the bundled sqlite3 library in the PHP source by the one with encryption features. All what’s left to do is compile it, and replace it in the running PHP. But before that…

Compile the SQLite3 extension

Compiling the PHP extension is easy, but what the documentation does not say is that a flag needs to be enabled in order for encryption to be loaded. Also, the m4 file has the wrong name for some reason and must be renamed first:

mv config0.m4 config.m4

Edit config.m4 with your favorite editor, and add this to the other_flags variable (line 81 in php 7.2, could be another location depending on version):

     other_flags="-DSQLITE_HAS_CODEC=1 -DSQLITE_ENABLE_FTS3=1 -DSQLITE_ENABLE_FTS4=1 -DSQLITE_ENABLE_FTS5=1 -DSQLITE_CORE=1 -DSQLITE_ENABLE_COLUMN_METADATA=1"

Now it is ready to compile so just issue the regular commands:

phpize
./configure
make
make install

You should get the following output if all goes well:

Configuring for:
PHP Api Version:         20170718
Zend Module Api No:      20170718
Zend Extension Api No:   320170718

Installing shared extensions:     /usr/lib/php/20170718/
Installing header files:          /usr/include/php/20170718/

Voila! Your extension should be installed in the correct location. Don’t forget to reload PHP-FPM or Apache if you’re using PHP on the server side.

Golang testing with sqlmock and gorm

2019-08-28T15:46:00+00:00

While writing tests for a Go project I needed a way to test my database queries. For this the library sqlmock exists. This works great and also does what I wanted. However a problem arose that it seemed to share the connection across mocks even though they were re-init’d at the start of the test. This was due to me not implicitly calling a DSN for the sqlmock.

Before I had the following:

does not work as intended

// we only need the mock interface
_, mock, _ := sqlmock.New()

// open the database for gorm (*gorm.DB)
mockedGorm, _ := gorm.Open("sqlmock", "sqlmock_db_0")

// init your own database interface
dbInterface := NewDatabase(conf.Database)

// set the *gorm.DB on your interface
dbInterface.Conn = mockedGorm

// close the DB when the function is over
defer mockedGorm.Close()

This did work, however in running the second test, it complained the connection was closed. Weird because I called sqlmock.New() in each test. Turns out because you provide a DSN to Gorm you need to use the sqlmock.NewWithDSN("some_dsn_string") method.

does work

// we only need the mock interface
// explicitely set the DSN
_, mock, _ := sqlmock.NewWithDSN("sqlmock_db_0")

// open the database for gorm (*gorm.DB)
// ensure the DSN matches.
mockedGorm, _ := gorm.Open("sqlmock", "sqlmock_db_0")

// init your own database interface
dbInterface := NewDatabase(conf.Database)

// set the *gorm.DB on your interface
dbInterface.Conn = mockedGorm

// close the DB when the function is over
defer mockedGorm.Close()

What is important is that you do not share the DSN across your test functions as it’ll keep sharing the stubbed DB conn.

For easier tests and preventing to initiate this each time we can do the following:

var (
	dsn_count int64
)

func provideMock() (sqlmock.Sqlmock, *gorm.DB, *DispatchDB) {
	dsn := fmt.Sprintf("sqlmock_db_%d", dsnCount)
	_, mock, err := sqlmock.NewWithDSN(dsn)
	if err != nil {
		panic(err)
	}
	mockedGorm, err := gorm.Open("sqlmock", dsn)
	if err != nil {
		panic(err)
	}
	dbInterface := NewDatabase(conf.Database)
	dbInterface.Conn = mockedGorm
	dsnCount++

	return mock, mockedGorm, dbInterface
}

func TestMain(m *testing.M) {
	dsnCount = 0
	code := m.Run()
	os.Exit(code)
}

func Test_someFunc(t *testing.T) {
	mock, mockedGorm, dbInterface := provideMock()
	defer mockedGorm.Close()

	// your tests using the mock
}

MySQL SHOW TABLES and information_schema privileges

2019-08-23T06:45:00+00:00

This subject came up when writing monitoring scripts: How do you give privileges to a MySQL or MariaDB user, so it will be able to access the table metadata without accessing the data?

As a refresher, MySQL schema and table metadata is accessible in the ISO-standard set of views contained in the information_schema database. It can also be found at a more limited level with the SHOW TABLES and SHOW TABLE STATUS commands.

Unfortunately, turns out there are no metadata privileges for such needs. The rule is the following:

Each MySQL user has the right to access these tables, but can see only the rows in the tables that correspond to objects for which the user has the proper access privileges. In some cases (for example, the ROUTINE_DEFINITION column in the INFORMATION_SCHEMA ROUTINES table), users who have insufficient privileges see NULL. These restrictions do not apply for InnoDB tables; you can see them with only the PROCESS privilege. The same privileges apply to selecting information from INFORMATION_SCHEMA and viewing the same information through SHOW statements. In either case, you must have some privilege on an object to see information about it. © 2019, Oracle Corporation and/or its affiliates

That means the user must be granted some kind of privilege on the destination object, or it won’t be able to read any metadata. The issue here, is that we don’t want to give our user any kind of potentially sensitive or destructive privilege (i.e. nothing in the SELECT/INSERT/UPDATE/DELETE range).

A quick call to SHOW PRIVILEGES gives us the following overview:

+-------------------------+---------------------------------------+-------------------------------------------------------+
| Privilege               | Context                               | Comment                                               |
+-------------------------+---------------------------------------+-------------------------------------------------------+
| Alter                   | Tables                                | To alter the table                                    |
| Alter routine           | Functions,Procedures                  | To alter or drop stored functions/procedures          |
| Create                  | Databases,Tables,Indexes              | To create new databases and tables                    |
| Create routine          | Databases                             | To use CREATE FUNCTION/PROCEDURE                      |
| Create temporary tables | Databases                             | To use CREATE TEMPORARY TABLE                         |
| Create view             | Tables                                | To create new views                                   |
| Create user             | Server Admin                          | To create new users                                   |
| Delete                  | Tables                                | To delete existing rows                               |
| Drop                    | Databases,Tables                      | To drop databases, tables, and views                  |
| Event                   | Server Admin                          | To create, alter, drop and execute events             |
| Execute                 | Functions,Procedures                  | To execute stored routines                            |
| File                    | File access on server                 | To read and write files on the server                 |
| Grant option            | Databases,Tables,Functions,Procedures | To give to other users those privileges you possess   |
| Index                   | Tables                                | To create or drop indexes                             |
| Insert                  | Tables                                | To insert data into tables                            |
| Lock tables             | Databases                             | To use LOCK TABLES (together with SELECT privilege)   |
| Process                 | Server Admin                          | To view the plain text of currently executing queries |
| Proxy                   | Server Admin                          | To make proxy user possible                           |
| References              | Databases,Tables                      | To have references on tables                          |
| Reload                  | Server Admin                          | To reload or refresh tables, logs and privileges      |
| Replication client      | Server Admin                          | To ask where the slave or master servers are          |
| Replication slave       | Server Admin                          | To read binary log events from the master             |
| Select                  | Tables                                | To retrieve rows from table                           |
| Show databases          | Server Admin                          | To see all databases with SHOW DATABASES              |
| Show view               | Tables                                | To see views with SHOW CREATE VIEW                    |
| Shutdown                | Server Admin                          | To shut down the server                               |
| Super                   | Server Admin                          | To use KILL thread, SET GLOBAL, CHANGE MASTER, etc.   |
| Trigger                 | Tables                                | To use triggers                                       |
| Create tablespace       | Server Admin                          | To create/alter/drop tablespaces                      |
| Update                  | Tables                                | To update existing rows                               |
| Usage                   | Server Admin                          | No privileges - allow connect only                    |
+-------------------------+---------------------------------------+-------------------------------------------------------+

We just need to pick up a privilege in the Tables context for our monitoring user to be able to read metadata. All things considered, the References seems to be the less destructive - it only allows a user to create foreign keys on a table. Create could also be used if you don’t fear being spammed with useless tables.

To implement that we just need to grant the chosen privilege to our monitoring user. It will now be able to access object metadata in the information_schema views.

grant references on backenddb.* TO 'monitoring_user'@'%';

Rundeck accepting incoming webhooks from e.g. Github

2019-05-18T22:30:00+00:00

We wanted to start using Rundeck more to replace an aging Jenkins system. However one thing was really missing from Rundeck, which was the ability to accept an incoming Webhook from Github. This boggled my mind for a bit, why isn’t this simply accepted. If we have a parameter called payload we’re good to go as a POST right. All I needed was something that could convert between the POST that Github sends as a Webhook and what Rundeck wants through their API.

I decided to solve this using nginx-njs which is a Javascript implementation inside nginx instead of using for example Lua.

What this eventually allows you to do is configure Github to use an URL like the following:

https://user:pass@rundeck.yourcool.domain/build/5fdf3803-d19d-4436-91d3-2d700904dbfb

# or using a name you mapped

https://user:pass@rundeck.yourcool.domain/buildByName/mycooljob

I hope this was helpful for someone trying to solve the same problem.

Setup

You’ll need nginx with njs support, their upstream repo’s for Debian have this out-of-the box. See here for how to get the repository setup.

apt-get install nginx nginx-module-njs

nginx

The following changes need to be made to /etc/nginx/nginx.conf

load_module modules/ngx_http_js_module.so;
load_module modules/ngx_stream_js_module.so;

http {
    js_include /etc/nginx/webhook.js;
    js_set $webhook payload_convert;
    js_set $jobid getJobID;
    js_set $jobUUIDFromName getBuildByName;
}

And we need the vhost config, e.g. /etc/nginx/conf.d/rundeck.conf

server {
    listen 80 default_server; 
    listen [::]:80 default_server;

    # Redirect all HTTP requests to HTTPS with a 301 Moved Permanently response.
    return 301 https://$host$request_uri;
}


server {
    listen       443 ssl;
    server_name  rundeck.yourcool.domain;

    #ssl_certificate yourcertshere;
    #ssl_certificate_key yourkeyhere;


    location /buildByName {
    	rewrite ^(/buildByName/.*) /build/$jobUUIDFromName last;
    }

    location /build {
    	auth_basic "Rundeck Build Server";
	# guide how to create this is here:
	# https://docs.nginx.com/nginx/admin-guide/security-controls/configuring-http-basic-authentication/
	auth_basic_user_file /etc/nginx/htpasswd;

        mirror /_NULL;                    # Create a copy of the request to capture request body
        client_body_in_single_buffer on;  # Minimize memory copy operations on request body
        client_body_buffer_size      256k; # Largest body to keep in memory (before writing to file)
        client_max_body_size         256k;

        proxy_pass_request_headers off;
	proxy_pass_request_body off;
	proxy_set_body $webhook;

	# best create a fixed token for Rundeck to allow from here, can be done as described in:
	# https://docs.rundeck.com/docs/api/#token-authentication using the framework.properties
	proxy_set_header 'X-Rundeck-Auth-Token' 'yourtoken';

	proxy_set_header 'Accept' 'application/json';
	proxy_set_header 'Content-Type' 'application/json';

        proxy_pass http://localhost:4440/api/31/job/$jobid/run;
    }

    location = /_NULL {
        internal;
        return 204;
    }

    # actually proxy Rundeck
    location / {
        proxy_pass http://localhost:4440;
    }
}

And finally the /etc/nginx/webhook.js

// grab the payload from the incoming POST
function payload_convert(req) {
	// place it in a try/catch in case the JSON can't create the object
	try {
		if ( req.variables.request_body.length > 0 ) {
			var buf = {};
			buf['options'] = {};
			buf['options']['payload'] = req.variables.request_body;

			return JSON.stringify(buf);
		} else {
			req.return(500, "Body cannot be empty"); 
		}
	} catch (e) {
		// log the exception
		req.log(e);
		req.return(500, "Something went horribly wrong"); 
	}
}

// get the job id
function getJobID(req) {
	return req.uri.replace('/build/', '');
}

function getBuildByName(req) {
	// list of jobs and their uuids
	var jobs = {
		'mycooljob': '5fdf3803-d19d-4436-91d3-2d700904dbfb'
	};

	var jobName = req.uri.replace('/buildByName/', '');	
	
	return jobs[jobName];
}

Rundeck

Ensure you have a Token set you can always use by nginx, e.g. create /etc/rundeck/tokens.properties

Note: the user has to exist!

nginx: my-super-duper-token

And inside /etc/rundeck/framework.properties we add:

rundeck.tokens.file=/etc/rundeck/tokens.properties

Rundeck Job

Ensure your job accept an option called payload as a text field.

Debian Stretch Reprepro ‘The following signatures were invalid’

2018-03-05T19:11:00+00:00

I’ve been using reprepro for a while now to run a Debian APT repo. Which worked fine but on Stretch it has been throwing errors left and right.

If you add an existing distro upstream you’ll get something like the following:

W: GPG error: https://somehost.tld distro InRelease: The following signatures were invalid: HASH
W: The repository 'https://somehost.tld distro InRelease' is not signed.
N: Data from such a repository can't be authenticated and is therefore potentially dangerous to use.
N: See apt-secure(8) manpage for repository creation and user configuration details.

This is due to Stretch no longer allowing SHA1 signing because it’s insecure or whatever.

However, nothing seems to properly show what to do to fix this, so here goes:

GnuPG configuration

Important things to do, first off we got to tell GnuPG to no longer prefer SHA1 at all.

Open up ~/.gnupg/gpg.conf

# Prioritize stronger algorithms for new keys.
default-preference-list SHA512 SHA384 SHA256 BZIP2 ZLIB ZIP Uncompressed
# Use a stronger digest than the default SHA1 for certifications.
cert-digest-algo SHA512

# these are critical, without it it'll keep using SHA1 for whatever reason
personal-digest-preferences SHA512
personal-cipher-preferences SHA512

# Optional, but recommended
default-key YOUR-NEW-KEY-HASH

If your key was 2048 bits at least, you’re now good to go. In my case it was simply a matter of removing the new Stretch distro dir (since I had no packages in it anyway) and having reprepro generate it on includedeb. After that moment it was working for me.

Journey goes on…

Secondly, if your original key wasn’t atleast 2048bit yet you’ll have to either create a new key or update the existing one.

New Key

To create a new one, quite easy just do gpg --gen-key and follow the steps. With the new config in place you’ll be good to go. Afterwards just do gpg --list-keys and take the new sub-key hash and configure reprepro to use it (/reprepro/dir/conf/distributions) and mark that for the following:

SignWith: YOUR-NEW-KEY-HASH

You can also add it to ~/.gnupg/gpg.conf

default-key YOUR-NEW-KEY-HASH

Update the Key

Now if you want to use an existing key so you don’t have to replace it everywhere (it’ll keep working for pre-Stretch, so this is good) we have to do a bit more steps.

To remove the SHA1 digest from the key these steps are needed, we’ll need to use gpg --edit-key KEYHASH. Now this command is very arcane and now very clear so here are some notes of what I found.

When I do gpg --edit-key ABCDEFG I get an interactive prompt:

gpg> showpref
[ultimate] (1). terwey <foo@bar.com>
     Cipher: AES256, AES192, AES, CAST5, 3DES
     Digest: SHA512, SHA384, SHA256, SHA224, SHA1
     Compression: BZIP2, ZLIB, ZIP, Uncompressed
     Features: MDC, Keyserver no-modify

But removing something isn’t so easy, it has to be in the format this system understands

gpg> pref
[ultimate] (1). terwey <foo@bar.com>
     S9 S8 S7 S3 H10 H9 H8 H11 Z3 Z2 Z1 Z0 [mdc] [no-ks-modify]

Ehr, yeah ok, great what does this mean? It’s the same info as before just the internal values, he’s a little table. Everything prefixed with an S is a Cipher, H is the Digest and Z is the compression.

This table is based on me playing a bit with the options it can set and what it returned, it’s not gospel.

GNUPG Code	Meaning
S13	CAMELLIA256
S12	CAMELLIA192
S11	CAMELLIA128
S10	TWOFISH
S9	AES256
S8	AES192
S7	AES
S4	BLOWFISH
S3	CAST5
H11	SHA224
H10	SHA512
H9	SHA384
H8	SHA256
H3	RIPEMD160
H2	SHA1
H1	MD5
Z3	BZIP2
Z2	ZLIB
Z1	ZIP
Z0	Uncompressed

So it seems what I want is the following:

gpg> setpref S9 S8 H10 H8 Z3 Z2 Z1 Z0
Set preference list to:
     Cipher: AES256, AES192, 3DES
     Digest: SHA512, SHA256, SHA1
     Compression: BZIP2, ZLIB, ZIP, Uncompressed
     Features: MDC, Keyserver no-modify

That’s weird, SHA1 is still there… digging a little deeper in RFC 4880 - OpenPGP Message Format

Since SHA1 is the MUST-implement hash algorithm, if it is not explicitly in the list, it is tacitly at the end. However, it is good form to place it there explicitly.

So yeah, guess for my key it doesn’t really matter. However it could be in your key it’s at the front and then it’s the preferred digest to use. In that case, save the key with the new preference and resign the repo.

systemd timers - how to restart a service every X hours

2017-11-17T15:09:00+00:00

Sometimes you have to face nasty service crashes, e.g. due to memory leaks (yes, logstash, I’m talking about you). Using the systemd timers feature is a handy and idiomatic way to restart systemd services (ofc we could use cron, but that’s not what this is about anyway). However, everyone knows by now that systemd can be mystical sometimes and its arcanes are somehow complex to understand.

That being said, I want to restart my logstash service 4 times after it’s been started. Therefore I need two files:

A service unit file that restarts my logstash service
A timer file that triggers my restart service

The restart service unit file looks straightforward:

# /etc/systemd/system/logstash-restart.service
[Unit]
Description=restart logstash

[Service]
Type=oneshot
ExecStart=/bin/systemctl restart logstash

However, I struggled a bit on how to restart the service every 4 hours after activation. I did not want to use the OnCalendar option because that would be akin to using cron and wouldn’t leverage the granularity of systemd timer functions. As it turns out, timers can be scheduled to be activated at a given time interval after they’ve been started with the OnActiveSec directive but that would only kick in once. The timer needs to be told to activate again after its initial activation with the OnUnitActiveSec directive. Eventually, the time when the timer was activated can be stored on disk and that can be done with the Persistent option. This option is false by default, which is OK because it doesn’t need to be enabled in my case - I want the timer to activate only after the service has been running for a given number of hours. This gives us the following timer file:

# /etc/systemd/system/logstash-restart.timer
[Timer]
OnActiveSec=4h
OnUnitActiveSec=4h

[Install]
WantedBy=timer.target

With both of those files on disk, there remains the need to enable and start the timer (that creates a symbolic link at the appropriate target location so it will be started at next boot, and also starts the countdown from now on):

# systemctl enable logstash-restart.timer
# systemctl start logstash-restart.timer

Checking if the timer shows up:

# systemctl list-timers --all
NEXT                         LEFT          LAST                         PASSED       UNIT                         ACTIVATES
Fri 2017-11-17 20:32:19 CET  3h 59min left Thu 2017-11-16 19:50:23 CET  20h ago      logstash-restart.timer       logstash-restart.service

systemd tracks execution times closely so it’s easy to tell whenever the timer has fired and when it will activate next.

True Headless Raspberry Zero W for Spotifyd

2017-11-08T12:00:00+00:00

I finally decided to buy a Raspberry Pi Zero W, since well it was all the rage and for a specific project I couldn’t use my Unwired One. So this drove me nuts, I actually had to attach a monitor for it to boot.

The problem is that from Mac/Windows you cannot see the other partitions besides /boot. However luckily there is a trick that it can copy files from there to the appropriate directories. Turns out other people are lazy too ;)

Here are the steps I took to make a Raspberry Pi boot without having to attach a monitor and have it auto-join your WiFi on first boot!

This assumes you flashed the card with Raspbian, whichever version I guess (I used Lite).

Prepping

Mount the device’s card on a machine, I used OSX but on Linux it should be the same.

Open up the mounted SD card, this should be the /boot volume.
Create an empty file called ssh. This will enable SSH on bootup.
Then create a file called wpa_supplicant.conf add the following to it and adjust as needed.

ctrl_interface=DIR=/var/run/wpa_supplicant GROUP=netdev
update_config=1

network={
    ssid="YOUR_SSID"
    psk="YOUR_SSID_PASSWORD"
    key_mgmt=WPA-PSK
}

On boot this file will be copied to /etc/wpa_supplicant/wpa_supplicant.conf

Now there should be 2 extra files on your SD card, ssh and wpa_supplicant.conf.

After booting these files will vanish from /boot and will be copied to the appropriate locations.

Now you unmount the SD-Card and stick it in your Raspberry Pi Zero W and have it boot and it will automatically join the network. Check your Router for the IP address it joined with. You’ll now be able to login using user pi and password raspberry.

Spotifyd

So I wanted to have a Spotify Connect device, since I got a nice stereo and it doesn’t natively have networking capabilities.

Turns out there’s an awesome Github project: Spotifyd/spotifyd

Setup

Time to setup the Spotifyd, you can grab the latest on their releases page.

wget https://github.com/Spotifyd/spotifyd/releases/download/untagged-9611d797f6d3344e6a8e/spotifyd-2017-07-22-armv7.zip
unzip spotifyd-2017-07-22-armv7.zip
chmod +x spotifyd
mv spotifyd /usr/bin/

Then we create a file for Spotifyd to work from, I took the example from their README, excellent stuff. This was the bare minimum I needed since I already set the default device previous. I have to software control over the volume, but it does work from Spotify’s UI.

The following should go in ~/.config/spotifyd/spotifyd.conf

[global]
backend = alsa
device_name = ZeroW
bitrate = 320
device = plughw:CARD=Audio # add this (from `aplay -L`) when you skipped the systemwide audio device

If you skipped the systemwide audio setup, you’ll have to check with aplay -L which device you need. I recommend you to use the plughw one to prevent resampling issues.

For more advanced things, like caching of files, go checkout their Github project, it’s well documented.

systemd

Spotifyd also has a systemd service file available for automatic starting of the daemon on boot.

I want the Spotifyd daemon to run on system start without logging in.

wget https://raw.githubusercontent.com/Spotifyd/spotifyd/master/contrib/spotifyd.service
sudo mv spotifyd.service /etc/systemd/system/
sudo systemctl daemon-reload
sudo systemctl enable spotifyd.service
sudo mv .config/spotifyd/spotifyd.conf /etc/

Now make the following adjustment to /etc/systemd/system/spotifyd.service.

## existing
ExecStart=/usr/bin/spotifyd --no-daemon
## change to
ExecStart=/usr/bin/spotifyd --no-daemon --config /etc/spotifyd.conf

That’s it for having it auto start :)

NAD DAC

So the first test with Spotifyd was done using a FiiO E10 DAC, now I thought, great time to wrap this up and attach my NAD DAC 1. Yeah so here the problems began. My initial testing I used the FiiO using type hw instead of type plughw. This was fine, since the FiiO could do both 44.1kHz and 48kHz. The NAD DAC 1 only takes 48000Hz, and Spotify feeds it 44100Hz which ALSA then has to do something with.

This can be fixed though, turns out for this card I did have to define an actual device in the spotifyd.conf, so I went back to aplay -L|grep -A1 plughw and got this as output:

plughw:CARD=Headset,DEV=0
    USB Headset, USB Audio

For resampling to work, we need to use the plughw:CARD=Headset one and not the hw one. I couldn’t get it to work purely with the asound.conf so I gave up and just added the following to ~/.config/spotifyd/spotify.conf

device = plughw:CARD=Headset

Optional Systemwide audio

On the Pi we have to do a few things, first find out your Audio device name (I use an external DAC, saves a lot of hassle), you can do so with aplay -l however this command will output the valid names for the config file. I couldn’t get this working with my NAD DAC. It did work with my FiiO DAC.

$ aplay -l | awk -F \: '/,/{print $2}' | awk '{print $1}' | uniq
ALSA
Audio # this is the one I need

The following can then be placed in the file /etc/asound.conf

pcm.!default {
        type hw
        card Audio
}

ctl.!default {
        type hw
        card Audio
}

Prometheus Slack Alerts with links to Grafana

2017-11-06T18:45:00+00:00

Prometheus is an excellent timeseries-based monitoring platform. As well, it’s easily pluggable with the equally excellent dashboard software, Grafana.

One of my colleagues thought it would be really cool if we got links to the Grafana dashboard for the given server when an alert is received in Slack. That looked possible, because the Grafana dashboard links use the node name and port as URL parameters, see below:

http://grafana:3000/dashboard/db/node-exporter-full?refresh=1m&orgId=1&var-node=web-01&var-port=9100

var-node and var-port parameters just need to be filled with this info. However, this is received from Prometheus with a single label, such as instance=web-01:9100.

We can use the power of Prometheus templating functions to generate a URL using the reReplaceAll regular expression function.

For example, the following template will capture the left part of the instance label (the host name), and output it:

{{ reReplaceAll "(.*):(.*)" "${1}" .Labels.instance }}

Here goes our final template to generate a slack notification including the URL, to be added to the alertmanager configuration:

receivers:
- name: ops-slack
  slack_configs:
  - username: prometheus
    send_resolved: true
    channel: alerts
    title: '[{{ .Status | toUpper }}{{ if eq .Status "firing" }}:{{ .Alerts.Firing | len }}{{ end }}] Monitoring Event Notification'
    text: >-
        {{ range .Alerts }}
          *Alert:* {{ .Annotations.summary }} - `{{ .Labels.severity }}`
          *Description:* {{ .Annotations.description }}
          *Graph:* <http://grafana.com:3000/dashboard/db/node-exporter-full?refresh=1m&orgId=1&var-node={{ reReplaceAll "(.*):(.*)" "${1}" .Labels.instance }}&var-port={{ reReplaceAll "(.*):(.*)" "${2}" .Labels.instance }} |:chart_with_upwards_trend:>
          {{ end }}
        {{ end }}

Unwired One / Blackswift stuff

2017-11-05T12:00:00+00:00

Quite a while ago I backed the Kickstarter campaign for the Black Swift and did receive it. However I didn’t really do anything with it, and by the time I received it it was already renamed to Unwired One. I played with it once in January 2017 and back then had huge problems getting the WiFi to work. Turns out factory default the internal IP it uses for LAN is 192.168.1.1 which is the same as most routers your home network is configured with. Fast forward to November 2017 when I had an idea for a project (which I’m describing here) and I again totally forgot about this bug. So I’ll be documenting it here for prosperity as the original Unwired One forums are offline :(

Preparation

Fixing the WiFi

First join the built-in WiFi Unwired One this is an unprotected access point. Then grab a Terminal and SSH in, username root password admin.

We have to change the lan interface to not be on 192.168.1.1 but something else.

$ ssh root@192.168.0.254
$ vi /etc/config/network # or nano, whichever you prefer

Make the following changes:

Original:

config interface 'lan'
        option ifname 'eth1'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.1.1' # change this line!
        option netmask '255.255.255.0'

New:

config interface 'lan'
        option ifname 'eth1'
        option force_link '1'
        option type 'bridge'
        option proto 'static'
        option ipaddr '192.168.5.1'
        option netmask '255.255.255.0'

Save the file, and do /etc/init.d/network restart or simply a reboot.

Upgrading the Firmware

When the device comes back online, let’s do an upgrade of the firmware, as mine was still running the BARRIER BREAKER (Barrier Breaker, r47068) 2015-10-15 13:09:30 release. The latest one you can find here: openwrt-ar71xx-generic-unwone-squashfs-sysupgrade.bin. The manual says you can do a wget directly on the device, this however didn’t work for me so I just copied to the file via SCP to the device, be sure to do this to the /tmp dir as that’s the memory mounted device tmpfs drive.

$ scp openwrt-ar71xx-generic-unwone-squashfs-sysupgrade.bin 192.168.0.254:/tmp
$ ssh 192.168.0.254
$ sysupgrade -v /tmp/openwrt-ar71xx-generic-unwone-squashfs-sysupgrade.bin # add the -n flag to reset to factory defaults

After this it will reboot and come back online with CHAOS CALMER (Chaos Calmer, r49404) 2016-08-16 00:18:25.

Joining the WiFi

Now we can simply follow the Instructions from the Manual but to rehash I’ll give the gist here.

Go to 192.168.0.254 with a browser
In the menu bar hover Network and then click Wifi
Hit the Scan button
Find your network, click Join Network
Fillout your password, and assign the lan firewall zone.
Hit save, wait for the next page to load and click Save and Apply
Wait for it to come back up joining your WiFi, at this point you’ll be kicked off the Unwired One network.

Now login to your router to see which new DHCP lease was given off to the device, or use a Bonjour Browser to find it.

SSH Refused

If you’re locked out of your Unwired One by SSH but can access the WebUI, the UI solution might work too. It’s written below.

I had the case I got locked out of the SSH but not the WebUI, on inspecting the logs I saw the following:

authpriv.warn dropbear[3222]: Failed loading /etc/dropbear/dropbear_ecdsa_host_key
authpriv.warn dropbear[3222]: Failed listening on '22': Error listening: Address already in use
authpriv.info dropbear[3222]: Early exit: No listening ports available.

Which led me to believe the Dropbear configuration was broken, so I went over to the System -> Administration interface and added a second SSH port (in my case I used 222) on an unspecified network.

On further inspection I noticed that sshd was running:

root@UnwiredOne:~# netstat -anp
Active Internet connections (servers and established)
Proto Recv-Q Send-Q Local Address           Foreign Address         State       PID/Program name    
tcp        0      0 0.0.0.0:2222            0.0.0.0:*               LISTEN      3934/dropbear
tcp        0      0 0.0.0.0:22              0.0.0.0:*               LISTEN      2532/sshd

After doing a /etc/init.d/sshd stop and an /etc/init.d/dropbear restart I was suddenly able to login again to my Unwired One.

My current /etc/config/dropbear now looks like this.

config dropbear
        option PasswordAuth 'on'
        option Port '22'

config dropbear
        option PasswordAuth 'on'
        option Port '2222'

Disabling SSHD from the UI

System -> Startup
Click Enable for sshd
Click Stop for sshd
Click Restart for Dropbear

Headless Teamviewer with custom resolution

2017-03-08T22:45:00+00:00

I wrote this to use Teamviewer on a remote machine without a graphics-card but do want to use high resolutions. I wanted to use full-screen on my 21:9 shinyness which does 3440x1440. However I couldn’t get past 1324x768 with normal configs. Hard pressed to get it working I decided to sacrifice an evening to it. It now works, and it’s gorgeous :)

On Debian Jessie this file was found at /usr/share/X11/xorg.conf.d/xorg.conf

Other packages needed were: xserver-xorg-video-dummy and Teamviewer.

Sample of Teamviewer showing the new available resolutions:

# Xorg.conf config for dummy video driver
# For usage with for example TeamViewer on a machine without a monitor attached
# and you wanted more then just 1024x768 ;)
#
# Use at own risk, loosly based on info scattered around but these links really helped
# http://arachnoid.com/modelines/ for the modelines (lot of trial and error to figure out which worked over Teamviewer and Xorg)
# https://www.xpra.org/xorg.conf sample config from xpra who seems to use the dummy driver a lot (thanks guys!)

Section "Device"
	Identifier "dummy_videocard"
	Option "NoDDC" "true"
	Option "IgnoreEDID" "true"

	# Debian: apt-get install xserver-xorg-video-dummy
	Driver "dummy"

	# You could lower this to suit your needs, however you will need
	# a quite high amount to run such crazy resolutions
	VideoRam 524288
EndSection

Section "Monitor"
	Identifier "dummy_monitor"

	# 16:9 4k
	Modeline "3840x2160_20.00" 218.15 3840 4016 4416 4992 2160 2161 2164 2185

	# 21:9 "4k"
	Modeline "3440x1440_20.00" 124.95 3440 3520 3864 4288 1440 1441 1444 1457

	# Usual 27" suspect before 4k era
	Modeline "2560x1440" 42.12 2560 2592 2752 2784 1440 1475 1478 1513

	# Oddball 1920 resolution
	Modeline "1920x1440" 69.47 1920 1960 2152 2384 1440 1441 1444 1457

	# 16:10 "Full-HD"
	Modeline "1920x1200" 26.28 1920 1952 2048 2080 1200 1229 1231 1261

	# 16:9 Full HD
	Modeline "1920x1080" 23.53 1920 1952 2040 2072 1080 1106 1108 1135

	# These are just crazy high to ensure stuff works
	HorizSync   5.0 - 1000.0
	VertRefresh 5.0 - 1000.0
EndSection

Section "Screen"
	Identifier "dummy_screen"
	Device "dummy_videocard"
	Monitor "dummy_monitor"
	DefaultDepth 24
	SubSection "Display"
		Depth 24
		Modes "3840x2160_20.00" "3440x1440_20.00" "2650x1440" "1920x1440" "1920x1200" "1920x1080"

		# Not sure why, but 3440x1440 won't work when the Virtual is set to "3840 2160"
		# However it will complain in the Xorg.log when you didn't comment out the 3840x2160 resolution at the top
		#
		# Uncomment this for 21:9 4k
		Virtual 3440 1440

		# Uncomment this for 16:9 4k
		#Virtual 3840 2160
	EndSubSection
EndSection