gamecreature

Install ruby 2.7.8 on FreeBSD 14 (with rbenv)

Yeah I know, it's very old. But unfortunately i needed to use this ruby version for running errbit. (the self hosted error handler)

Trying to install install an older ruby-version on FreeBSD results in the following error (is found in the ruby-build log file )

util.c:236:1: error: expected identifier or '('
ruby_qsort(void* base, const size_t nel, const size_t size, cmpfunc_t *cmp, void *d)
^
./include/ruby/util.h:59:21: note: expanded from macro 'ruby_qsort'
-# define ruby_qsort qsort_r

This is an issue in the ruby code compiling on FreeBSD 14. The issue is mentioned for ruby 3.1 See: https://bugs.ruby-lang.org/issues/20151.
Problem is that the solution isn't backported. There isn't a ruby 2.7.9.

Quick Solution

Create the file ./rbenv/plugins/ruby-build/share/ruby-build/2.7.8-freebsd-14, with the following content

install_package "openssl-1.1.1w" "https://www.openssl.org/source/openssl-1.1.1w.tar.gz#cf3098950cb4d853ad95c0841f1f9c6d3dc102dccfcacd521d93925208b76ac8" openssl --if needs_openssl:1.0.1-1.x.x
install_package "ruby-2.7.8" "https://www.blommersit.nl/downloads/ruby-build/ruby-2.7.8-freebsd-14.tgz#58beea1e9e954efb2e7c27a3dcf5817d739049b50ae718c78d4fffe9a1e11c0b" warn_eol enable_shared standard

And install it:

rbenv install 2.7.8-freebsd-14

Further changes to run errbit:

  • remove the ruby-version from the Gemfile
  • change .ruby-version file to match 2.7.8-freebsd-14
  • bundle update puma to a later version (issue with nio4r)

Details

The issue happens because util.c file contains some q_sort function logic, which resolves incorrectly in FreeBSD 14.

The source contains the following patch (Note this is not a correct solution for all platforms, but a quick hack to make it work in FreeBSD 14).

Around line 222 a few defines are undefined

#undef HAVE_BSD_QSORT_R
#undef HAVE_QSORT_S

The patched ruby version is here for download:

The download https://www.blommersit.nl/downloads/ruby-build/ruby-2.7.8-freebsd-14.tgz#58beea1e9e954efb2e7c27a3dcf5817d739049b50ae718c78d4fffe9a1e11c0b

FreeBSD Update all php packages to a new version

When PHP is installed on a FreeBSD system, you have a lot of packages, for example:

pkg prime-list
php80-ctype
php80-curl
php80-exif
...
php80-tokenizer
php80-zip
php80-zlib

To update all these packages to newer version the following can be used :-)

# first note what was installed (optional but recommended)
pkg prime-list > /tmp/installed-packeges.txt

# then execute the update command php80 => php83
cat /tmp/installed-packages.txt | grep php80 | sed 's/php80/php83/g' | xargs -o  pkg install

Freebsd bastille console not working after upgrade 13 to 14

After updating my host system to FreeBSD 14.0, it was time to update the bsatille jails.
The jails I've running are thin jails , with symlinks to the release of the given jail.

Summary of the update

The jail have been updated following the instructions found at the bastille manual

# ensure the new version is bootstrapped and update to the latest patch release: 
bastille bootstrap 14.0-RELEASE update

# and change the 13.2 to 14.0 mount
bastille stop TARGET
bsatille edit TARGET fstab

# force reinstallation and upgrade all packages
# the pkg boostrap  isn't in the manual but IS required
bastille start TARGET
bastille pkg TARGET bootstrap -f
bastille pkg TARGET update
bastille pkg TARGET upgrade -f
bastille restart TARGET

bastille console fails

Next the jail seemed to be running, but unfortunately bastille console TARGET just stops without any errors/warning etc.

Though bastille console didn't work, It was still possible to enter the jail via jexec

jexec TARGET

Incorrect /etc configuration files

The problem is described in the following post: https://forums.freebsd.org/threads/newbie-upgrade-problem-13-2-release-14-0-release-merge-conflict.91219/

The reason is that pam_opie.so seems to have been removed. it can be fixed by simply moving that auth method as describe there.

A much better way is to update the outdated /etc configuration files! (These haven't been updated yet!)

Update the /etc configuration with etcupdate

First make sure the freebsd sources are available in /usr/jails/bastille/releases/14.0-RELEASE/usr/src

I solved this by copying the sources from my host system to the release.

cp -Rp /usr/src/ /usr/local/bastille/releases/14.0-RELEASE/usr/src/

You can also download the source directly via this link: )
https://cgit.freebsd.org/src/snapshot/releng/14.0.tar.gz

Next perform the etcupdate in the jail.

jexec TARGET
etcupdate
etcupdate resolve # if there are conflicts

Fix merge conflicts if they happen.
After that bastille console should work again!

FreeBSD update 13 to 14

I've just updated my server from 13.2-RELEASE to 14.0-RELEASE. This when pretty flawlessly, I had an issue with slow updating of the userland part. (The update after the first reboot)

It was extremely slow, update userland was already running for more then an hour. Pretting Ctrl+t to view what it was doing, i noticed it was [wait]ing most of the time.

Searching for this problem I found a solution speed it up dramaticly.
Setting the following control variable (in another terminal) made the update finish almost instantly.

sysctl vfs.zfs.dmu_offset_next_sync=0

After the next reboot this flag is turned back to the original value.

Thanks to the contributors of the following references:

ActiveStorage SVG Variant Transformer

In a project I'm working on (My Hero's Journey), we use a lot SVG's for the artwork of the application. The application sometimes needs a full colored image of the SVG. Other times it needs change a certain accent color. We would love to use the ActivateStorage variant support for this. It would be a good use case for these images. Just generate it once, store it and return the different variations.

The original image. (In this image the islands are colored full red (#FF0000 with transparency))

A variant of this is image is to replace the red color (#FF0000) with green.

Or changing the hue to a given color.

The problem

Active Storage supports variants for images. Though this doesn't include SVG's, because it's a vector image instead of an rasterized image.

Under the hood the ActivateStorage uses a transformer class. ActiveStorage::Transformers::Transformer

When you look at the code, at first glance it looks like it should be able to support multiple transformers. At the moment using a different transformer is not (yet) available in ActiveStorage.

The cause of this, is the definition of the transformer in the
ActiveStorage::Variation class. It's always the ImageProcessingTransformer:

def transformer
    ActiveStorage::Transformers::ImageProcessingTransformer.new(transformations.except(:format))
end

My Solution for Now

A workaround for not being able to change the transformer class, can be done by patching the ActiveStorage::Variation#tranformer method. Make sure the following class is loaded by an initializer.

lib/active_storage/variation_custom_formatter_extension.rb`

module ActiveStorage::VariationCustomFormatterExtension
  def transformer
    return ActiveStorage::SvgTransformer.new(transformations.except(:format)) if content_type == "image/svg+xml"
    super
  end

  ActiveStorage::Variation.prepend(self)
end

Just invoke the constant on initialization to make Zeitwork load it.

config/initializers/activestorage_custom_formatter.rb

ActiveSupport::Reloader.to_prepare do
  ActiveStorage::VariationCustomFormatterExtension
end

A general implementation could to add a factory, which creates a tranformer based on a given mime-type.

To use SVG's as images in ActiveStorage, and allow variations the following configuration options are required.

config/initializers/active_storage_svg_support.rb

# SVG shouldn't be served as binary.
Rails.application.config.active_storage.content_types_to_serve_as_binary -= ['image/svg+xml']

# Make the default content-disposition of the svg's inline. (just like another image)
Rails.application.config.active_storage.content_types_allowed_inline += ['image/svg+xml']

# Make the svg content type 'variable', so variations are enabled
Rails.application.config.active_storage.variable_content_types += ['image/svg+xml']

ActiveStorage Transformer

An ActiveStorage transformer in rails uses a hash with transformations that are applied to the given image.
For example, a variant of rasterized image can be defined by passing several operations/settings:

image_tag asset.file.variant(
  resize_to_limit: [100, 100],
  format: :jpeg,
  saver: { subsample_mode: "on", strip: true, interlace: true, quality: 80 })

A simple SVG transformer

We could also define all kinds operations for SVG files. In our application I've defined two operations (for now). replace_color, to replace a color by another, and set_hue, to change the hue of an image.

image_tag asset.file.variant(set_hue: "#FF0000", format: :svg)

Below is the implementation of this transformer class. (It uses the color_conversion gem, to perform rgb to hsl calculations)

The main entry point that's being called by ActiveStorage is process. This method is called with two arguments, file and format. The file is a temporary file which contains the original data. When you don't rewind the tempfile, you get ActiveStorage::IntegrityErrors. Because ActiveStorage requires the file to be open at the first byte in the file.

The transformations applied to this formatter can be accessed via the method transformations.

lib/active_storage/svg_transformer.rb

class ActiveStorage::SvgTransformer < ActiveStorage::Transformers::Transformer
  def process(file, format:) # :doc:
    content = file.read
    transformations.each do |name, data|
      case name
      when :set_hue then content = set_hue(content, data)
      when :replace_color then content = replace_color(content, data)
      else raise "SVG Transformation not supported #{name}"
      end
    end

    out = Tempfile.new(["svg_transformer", ".svg"])
    out.write(content)
    out.rewind
    out
  end

  # { hue: #EE00AA }
  private def set_hue(content, color_in)
    hex_color_regexp = /#[A-F0-9]{3,8}/i
    color_tone = ColorConversion::Color.new(color_in).hsl
    content = content.gsub(hex_color_regexp) do |hex_in|
      hex, alpha = extract_alpha(hex_in)
      color = ColorConversion::Color.new(hex).hsl
      new_color = ColorConversion::Color.new(h: color_tone[:h], s: color[:s], l: color[:l])
      "#{new_color.hex}#{alpha}"
    end
    content
  end

  # Extract the hex and alpha part of a hex color
  # - #RGBA => #RGB, AA
  # - #RRGGBBAA => #RRGGBB, AA
  private def extract_alpha(hexa)
    case hexa.length
    when 5 then [hexa[0..3], hexa[4] * 2]
    when 9 then [hexa[0..6], hexa[7..8]]
    else [hexa, ""]
    end
  end

  # { replace_color: { "#FF0000" => "#4455aa" , ... } }
  private def replace_colors(content, key_value_map)
    key_value_map.each do |key, value|
      content = content.gsub(/#{key}/i, value)
    end
    content
  end
end

The SVG images are drawn with Affinity Designer and exported with hex color codes. At the moment the colors of the images are simply adjusted via a regexp replace. (Full XML parsing/building could be used for complex operations)

And that's all!

I'm pretty happy with the solution. The usage of variants makes use of the groundwork of ActiveStorage handling. Which generates it just once and store it on disk. And automaticly deletes the variants when the main image is deleted.

It would be very nice if ActiveStorage added support for multiple Variant transformers for different content-type. Audio transformers, PDF transformers, CSV transformers, or docx variations. (There is a related PR request about this feature. But it seems to be stalled)

Btw, I'm aware you can do very much with SVG on the browser side, like CSS coloring and JS dynamics. This works very well when rending an SVG inline in an HTML file. When using external SVG's as images this becomes a lot harder!