FreeBSD, MySQL out of swap space

My FreeBSD server is giving a lot of swap errors. Every day the daily reporting email was filled with kernel messages like these:

my.server.com kernel log messages:
+swap_pager_getswapspace(31): failed
+swap_pager_getswapspace(32): failed
 ... and many more ...

After figuring out which process was causing this, it seemed MySQL was to blame.
That was odd, because these are the number of mysqltuner is reporting about my configuration:

[--] Physical Memory     : 8.0G
[--] Max MySQL memory    : 845.4M

Max 850 MB

After a lot of googling, reading MySQL, it seemd the performance schema could be to blame. Because is is completely stored in memory.

Now for several days in a row, I don't have the swaperrors ☺️
To disable the MySQL permance schema place the following in /usr/local/etc/mysql/my.cnf

[mysqld]
performance_schema = 0

Btw. I'm not sure if I need the performance scheme ..

⚠️ Followup a few weeks later

Unfortunately this isn't the solution. MySQL is still running out of swap space. I simply don't get it, because the memory usage should be limited

Followup more weeks later ;-)

ZFS ARC is taking a lot of memory...
Somehow MySQL checks the 'free' space and decides by itself to use swap. So the ZFS ARC memory is never released and MySQL .

To solve it limit the zfs arc memory /boot/loader.conf

vfs.zfs.arc_max="4G"

Bundle Hangs on bundle update

I finally found a reason why sometimes my 'bundle update' hangs infinitely.. It keeps showing dots... and takes forever..

bundle audit
Resolving dependencies...........   *sigh* ...

I have some gems that are available only for authorized users.
When the ssh keychain is still locked (haven't entered a password for my ssh-key yet) Bundler keeps running infinitely without showing any password entry...

ZFS cleanup of snapshots

Just a small note for this oneliner to cleanup snapshots

First list all snapshots with a given pattern.
(This eaxmple searches 12m)

zfs list -t snapshot | grep -F '12m' | cut -d " " -f 1 | xargs -L 1 echo zfs destroy

Remove the word echo to perform a cleanup

zfs list -t snapshot | grep -F '12m' | cut -d " " -f 1 | xargs -L 1 zfs destroy

Explanation:

zfs list -t snapshot
List all zfs snaphots

grep -F '12m'
Only match the lines containing the string 12m

cut -d " " -f 1
Only fetch the first column of the output. (This is the snapshot name)

xargs -L 1 zfs destroy
Execute the command 'zfs destroy' for the found lines
The option -L 1 means it calls 'zfs destroy' per item
without it, xargs wills supply multiple items to 1 call. (Which isn't support by zfs destroy)