ActiveStorage SVG Analyzer

For My Hero's Journey, we're building a new feature to show/hide layers of an SVG image depending on the progress you make. It's used for pretty large/complex images. (see previous post for a custom svg variant transformer)

The backend of the app, attaches layer names to certain tasks that need to be done. When finishing the task, the layer becomes visible.

The frontend is going to show this SVG, with all layers hidden that aren't available yet. (Pretty easy to do with CSS classes)

To make this performant the layer names need to be extracted from the SVG. A good place to make this happen is via an ActiveStorage Analyzer.

A Custom SVG Analyzer

The ActiveStorage::Blob model contains a metadata field, that receives the metadata. The default image analyzer only extracts width and height from the SVG.

Registering a custom anlyzer can be done with an initializer. During development I noticed ActiveStorage only runs one analyzer. The first analyzer that's valid for a given file, is being used.
That's the reason our custom analyzer is prepended to the analyzers array. (The delete/prepend squence is for development-mode reloading)

config/initializers/active_storage_svg_analyzer.rb

ActiveSupport::Reloader.to_prepare do
  Rails.application.config.active_storage.analyzers.delete ActiveStorage::CustomSvgAnalyzer

  # Important! Needs to be prepended to be placed before the othe analyzers. It seems active storage only runs a single analyzer
  Rails.application.config.active_storage.analyzers.prepend ActiveStorage::CustomSvgAnalyzer
end

The interface for an analyzer is pretty simple, the accept? class method needs to return true if the analyzer supports the given file.

The metadata method should return a hash with all metadata, this metadata is placed in the active storage blob record.

A SVG file is just a XML file, so parsing this is pretty easy with for example Nokogiri. In the example below all id's are extracted. (Affinity Designer exports the layer names as id's).

Because only 1 analyzer is run, I also included the width/height metadata attributes. (Though I don't think these are being used)

app/lib/active_storage/custom_svg_analyzer.rb

class ActiveStorage::CustomSvgAnalyzer < ActiveStorage::Analyzer
  def self.accept?(blob)
    blob.content_type == 'image/svg+xml'
  end

  def metadata
    download_blob_to_tempfile do |file|
      doc = Nokogiri::XML(file)
      width, height = extract_dimensions(doc)
      { width:, height:, layer_names: layer_names(doc) }.compact
    end
  end

  private def layer_names(doc)
    doc.xpath("//*[@id]").map { |e| e[:id] }
  end

  private def extract_dimensions(doc)
    view_box = doc.root.attributes["viewBox"]&.to_s
    return [] unless view_box

    left, top, right, bottom = view_box.split(/\s+/).map(&:to_i) # => 0 0 4167 4167

    [right - left, bottom - top]
  end
end

The result

The active_storage_blobs.metadata record now contains the following data:

{"identified":true,"width":4167,"height":4167,"layer_names":["island","island-plant-3","island-plant-2","island-plant-1","island-color-overlay","house","mask","water-well","drum2","drum1","vase3","vase2","vase1","woman-and-child","tree","racoon-tower","racoon1","racoon2","racoon-top-back","_clip1","racoon-middle-back","racoon-bottom","racoon-middle","racoon-top","single-orange","oranges"],"analyzed":true}

Data that is is pretty easy to acces, via the blob.metadata hash. See the asset model below.

models/asset.rb

class Asset
  has_one_attached :file

  def image_layer_names
    content_type_svg? ? Array(file.blob.metadata['layer_names']) : []
  end
end

That's all, thanks for reading!

Install ruby 2.7.8 on FreeBSD 14 (with rbenv)

Yeah I know, it's very old. But unfortunately i needed to use this ruby version for running errbit. (the self hosted error handler)

Trying to install install an older ruby-version on FreeBSD results in the following error (is found in the ruby-build log file )

util.c:236:1: error: expected identifier or '('
ruby_qsort(void* base, const size_t nel, const size_t size, cmpfunc_t *cmp, void *d)
^
./include/ruby/util.h:59:21: note: expanded from macro 'ruby_qsort'
-# define ruby_qsort qsort_r

This is an issue in the ruby code compiling on FreeBSD 14. The issue is mentioned for ruby 3.1 See: https://bugs.ruby-lang.org/issues/20151.
Problem is that the solution isn't backported. There isn't a ruby 2.7.9.

Quick Solution

Create the file ./rbenv/plugins/ruby-build/share/ruby-build/2.7.8-freebsd-14, with the following content

install_package "openssl-1.1.1w" "https://www.openssl.org/source/openssl-1.1.1w.tar.gz#cf3098950cb4d853ad95c0841f1f9c6d3dc102dccfcacd521d93925208b76ac8" openssl --if needs_openssl:1.0.1-1.x.x
install_package "ruby-2.7.8" "https://www.blommersit.nl/downloads/ruby-build/ruby-2.7.8-freebsd-14.tgz#58beea1e9e954efb2e7c27a3dcf5817d739049b50ae718c78d4fffe9a1e11c0b" warn_eol enable_shared standard

And install it:

rbenv install 2.7.8-freebsd-14

Further changes to run errbit:

  • remove the ruby-version from the Gemfile
  • change .ruby-version file to match 2.7.8-freebsd-14
  • bundle update puma to a later version (issue with nio4r)

Details

The issue happens because util.c file contains some q_sort function logic, which resolves incorrectly in FreeBSD 14.

The source contains the following patch (Note this is not a correct solution for all platforms, but a quick hack to make it work in FreeBSD 14).

Around line 222 a few defines are undefined

#undef HAVE_BSD_QSORT_R
#undef HAVE_QSORT_S

The patched ruby version is here for download:

The download https://www.blommersit.nl/downloads/ruby-build/ruby-2.7.8-freebsd-14.tgz#58beea1e9e954efb2e7c27a3dcf5817d739049b50ae718c78d4fffe9a1e11c0b

FreeBSD Update all php packages to a new version

When PHP is installed on a FreeBSD system, you have a lot of packages, for example:

pkg prime-list
php80-ctype
php80-curl
php80-exif
...
php80-tokenizer
php80-zip
php80-zlib

To update all these packages to newer version the following can be used :-)

# first note what was installed (optional but recommended)
pkg prime-list > /tmp/installed-packeges.txt

# then execute the update command php80 => php83
cat /tmp/installed-packages.txt | grep php80 | sed 's/php80/php83/g' | xargs -o  pkg install

Freebsd bastille console not working after upgrade 13 to 14

After updating my host system to FreeBSD 14.0, it was time to update the bsatille jails.
The jails I've running are thin jails , with symlinks to the release of the given jail.

Summary of the update

The jail have been updated following the instructions found at the bastille manual

# ensure the new version is bootstrapped and update to the latest patch release: 
bastille bootstrap 14.0-RELEASE update

# and change the 13.2 to 14.0 mount
bastille stop TARGET
bsatille edit TARGET fstab

# force reinstallation and upgrade all packages
# the pkg boostrap  isn't in the manual but IS required
bastille start TARGET
bastille pkg TARGET bootstrap -f
bastille pkg TARGET update
bastille pkg TARGET upgrade -f
bastille restart TARGET

bastille console fails

Next the jail seemed to be running, but unfortunately bastille console TARGET just stops without any errors/warning etc.

Though bastille console didn't work, It was still possible to enter the jail via jexec

jexec TARGET

Incorrect /etc configuration files

The problem is described in the following post: https://forums.freebsd.org/threads/newbie-upgrade-problem-13-2-release-14-0-release-merge-conflict.91219/

The reason is that pam_opie.so seems to have been removed. it can be fixed by simply moving that auth method as describe there.

A much better way is to update the outdated /etc configuration files! (These haven't been updated yet!)

Update the /etc configuration with etcupdate

First make sure the freebsd sources are available in /usr/jails/bastille/releases/14.0-RELEASE/usr/src

I solved this by copying the sources from my host system to the release.

cp -Rp /usr/src/ /usr/local/bastille/releases/14.0-RELEASE/usr/src/

You can also download the source directly via this link: )
https://cgit.freebsd.org/src/snapshot/releng/14.0.tar.gz

Next perform the etcupdate in the jail.

jexec TARGET
etcupdate
etcupdate resolve # if there are conflicts

Fix merge conflicts if they happen.
After that bastille console should work again!

FreeBSD update 13 to 14

I've just updated my server from 13.2-RELEASE to 14.0-RELEASE. This when pretty flawlessly, I had an issue with slow updating of the userland part. (The update after the first reboot)

It was extremely slow, update userland was already running for more then an hour. Pretting Ctrl+t to view what it was doing, i noticed it was [wait]ing most of the time.

Searching for this problem I found a solution speed it up dramaticly.
Setting the following control variable (in another terminal) made the update finish almost instantly.

sysctl vfs.zfs.dmu_offset_next_sync=0

After the next reboot this flag is turned back to the original value.

Thanks to the contributors of the following references: