freebsd

There are 23 posts tagged freebsd (this is page 3 of 5).

FreeBSD, MySQL out of swap space

My FreeBSD server is giving a lot of swap errors. Every day the daily reporting email was filled with kernel messages like these:

my.server.com kernel log messages:
+swap_pager_getswapspace(31): failed
+swap_pager_getswapspace(32): failed
 ... and many more ...

After figuring out which process was causing this, it seemed MySQL was to blame.
That was odd, because these are the number of mysqltuner is reporting about my configuration:

[--] Physical Memory     : 8.0G
[--] Max MySQL memory    : 845.4M

Max 850 MB

After a lot of googling, reading MySQL, it seemd the performance schema could be to blame. Because is is completely stored in memory.

Now for several days in a row, I don't have the swaperrors ☺️
To disable the MySQL permance schema place the following in /usr/local/etc/mysql/my.cnf

[mysqld]
performance_schema = 0

Btw. I'm not sure if I need the performance scheme ..

⚠️ Followup a few weeks later

Unfortunately this isn't the solution. MySQL is still running out of swap space. I simply don't get it, because the memory usage should be limited

Followup more weeks later ;-)

ZFS ARC is taking a lot of memory...
Somehow MySQL checks the 'free' space and decides by itself to use swap. So the ZFS ARC memory is never released and MySQL .

To solve it limit the zfs arc memory /boot/loader.conf 

vfs.zfs.arc_max="4G"

FreeBSD Set Timezone and Time to chmos clock

The following commands can be used to change time zone.
When the file /etc/wall_chmos_clock exists, it wil interpret the chmod time as local time.
Else it interprets the systemtime as UTC time

cp /usr/share/zoneinfo/Europe/Amsterdam /etc/localtime
touch '/etc/wall_cmos_clock'
adjkerntz -i

Adding a raw mail-file to the postfixqueue

Today I had an issue that I needed to resend quarantined emails.
You can simply execute the following command to inject the mails to postfix


sendmail -t < name_of_the_raw_email_file

Source:
Thanks Wietse (Author of Postfix)
http://postfix.1071664.n5.nabble.com/best-way-to-send-quot-raw-quot-email-files-td45960.html

Encrypted offsite backups via rsync

I love rsync to simply send files to other machines for backing up my data.
Unfortunately sometimes you can't trust the other side. So you would like to encrypt your data..
Rsync (via ssh) is encrypted during transport but not on disk...

I tried several solutions but they all have some issues:

duplicity
It's nice but doesn't do what rsync does. It just makes an encrypted initial full-backup and sends increments. It's also adviced to do a full backup regularly. That's not nice thing when you have to send >200GB over the internet..

rsyncrypto
Rsyncrypto encrypts files in such a way they are rsync friendly. (Big files don't completely change when a byte is changed).
Problem with rsyncrypto is that it requires an encrypted copy of all your data. I've got a LOT of files, and keeping this copy up to date costs too much time. I need to run rsyncrypto just before rsyncing the data. And my experiences with it is that it was pretty slow. And I even haven't taken into account the issue that I'm required to store my data on disk 2 times..

Meet encfs

Thanks to a reaction on serverfault ( http://serverfault.com/questions/160014/encrypted-remote-backups-via-rsync )
I've found encfs. Which makes it possible to create and mount a virtual filesystem via fuse-fs which shows an encrypted representation of your files..
Exactly what I want...

Install encfs on FreeBSD

It's required to enable fuse in FreeBSD. This is a kernel module
so add the following line to '/boot/loader.conf'

fuse_load="YES"

Next install the port (or package) encfs. (I love ports)

cd /usr/ports/sysutils/fusefs-encfs
make install clean

Creating a Backup

# mount read-only encrypted virtual copy of unencrypted local data:
encfs --reverse --idle=60 -o ro ~/data/ ~/.tmp_encrypted/

rsync -ai  ~/.tmp_encrypted/ name@example.com:backup/

umount ~/.tmp_encrypted/

First time a menu appears. I choose the following options:

  • Configuration mode: x - expert configuration
  • Cipher algorithm: AES
  • Key size: 256
  • Block size: 1024
  • Filename encoding: null*
  • Per-file initialization vectors: No
  • Password: *****

*I don't have the requirement to encrypt my filenames

Restore

To restore you data..

  • Take your encrypted file(s) Only the one's you need :D
  • Copy them into an empty folder ~encrypted-stuff
  • Copy your .encfs6.xml key to the same folder

Mount it:

encfs ~encrypted-stuff ~decrypted-stuff

Encfs asks for the password and behold: you can acces your files again in the ~decrypted-stuff folder :D

WARNING

You should backup your plain text .encfs6.xml file (which is on the unencrypted volume).
on a really safe location. It's your only key to decrypt your data.

Notes

A possible extra safety measure could be to NOT sync the encrypted .encfs6.xml file...
So use --exclude=encfs6.xml

For me the filenames aren't really sensitive so I don't encrypt them.
It makes it much more easy to find the correct file in my encrypted data..

Btw a scripting tip could be the following '--sdtinpass' and supply your password:

echo 'PASSWORD' | encfs --reverse --stdinpass --idle=60 -o ro ~/data/ ~/.tmp_encrypted/

If you have suggestions or advise, please drop a line!!