Happy Friday! 🥳

Today I merged one of the biggest PRs I’ve worked on for Visualizer. It fundamentally changes how Visualizer stores data about your coffee. Up until now, all data was stored directly on a shot. With this change, you’ll be able to optionally extract data into Roasters and Coffee Bags. This will allow you to change all the coffee data of all the shots by a single bag or roaster at once. 🤩

Even though it might have seemed like not much was going on with Visualizer development lately, I can tell you that this work started way back in March. I’ve been working on it most Fridays and many hours during the weekends. I’m able to do that because I’ve started working with Cliniko, and the 30-hour work week allows me to dedicate significant time to Visualizer, for which I’m super grateful. 🙏

This kind of feature has been requested pretty much since day one, and I’ve tried to implement it in many different ways. Each time I eventually found a wall I couldn’t get past and then gave up. This time I believe I cracked it, and I’m very happy with how it all turned out.

However, you cannot try this yet! 😬

Like I said, it’s a big feature with lots of code changes. I changed many interfaces, introduced several new views, created a combo-box component from scratch, and spent a ton of time reworking Airtable integration to support 2 new tables, touching pretty much everything in the codebase. To be exact: 2,459 lines of code were added and 1,150 removed. 🙈

I’m quite confident in all the changes, and I’ve tested them thoroughly. But in a couple of days, I’ll be going to Indonesia for 3 weeks, and doing bug fixes and customer support is not on the list of things I want to be doing there. 😄

You can expect this to be available to Premium subscribers in the beginning of August.

The reason I merged it today and announced it here is simply that I didn’t want to hold it back any longer. Besides the big feature, this update brings a lot of UI/UX updates to all users, like:

• Fixing zooming on iOS when tapping into a search field
• More compact edit/delete buttons to have more row space
• Not taking valuable row space if no shot has an image attached to it
• The aforementioned combo box for selecting grinder, roaster, and bean type
• Dark mode improvements
• Standardizing how links look

And probably some more that I’ve frankly forgotten about, given how long I’ve been working on this. 😂

Thanks for your time, thanks for using Visualizer, enjoy your summer, treat yourself to some amazing coffee, and expect another announcement in early August. 🌴

A couple of weeks ago, gillesbeesley opened an interesting issue on GitHub basically asking this:

As a Pressensor user with a Flair 58 and modified Breville dual boiler, I’d like to download my shot data in CSV format and upload it to the Pressensor app for reference when pressure and flow profiling on my machines.

Since I’ve never used Pressensor, it took some back and forth, to figure out the exact format. Eventually I found a way to add the export functionality, so now, you can easily export any shot as a CSV file by clicking the table-shaped button on any shot page.

Updates RSS

Since I support RSS on people’s pages, there’s absolutely no reason there shouldn’t be a feed for Updates as well. So now there is: https://visualizer.coffee/updates/feed.

Misc

As usual, there were many other minor improvements, most notably switching the JavaScript CDN provider from Skypack to jsDelivr. I didn’t have any issues, but Skypack seems abandoned, and importmap-rails recenty switched logic to always download. I decided to use the new approach wherever possible, and, since I was already working with importmaps, I took the opportunity to switch to jsDelivr for packages that are not yet compatible with the new approach due to their complexity.

Enjoy the remainder of your Easter! 🐣

This is going to be a more technical post than what you’re used to reading here, but it’s quite a big change, so I believe it’s the right place to share it.

Solid Queue

I, like many other Rails developers, was using Sidekiq for background jobs. It’s a great gem, but unless you’re paying for the pro version, it’s quite unsafe. When Sidekiq starts processing a job it removes it from Redis. If the worker is interrupted and/or killed before the job is finished, that job will be lost. The solution is upgrading to the pro version, but that is quite expensive for a small project like Visualizer.

So when David introduced Solid Queue at Rails World 2023, it looked like a perfect solution. It was lacking a few crucial things, though. The first one being a simple web interface to see the jobs. That got quickly resolved with Mission Control - Jobs. It’s very simple, but it gets the job done. No pun intended.

The other big thing was scheduling jobs. I was using sidekiq-scheduler for this before, and it’s a very important feature to have. So I was closely following the PR#155 and yesterday Rosa merged it in. 🥳

Literally minutes later Visualizer was running it in production, and it’s been working great since.

The Migration

At first, I had to point directly to the GitHub repository, but the updates have now been released, so all you need is to replace sidekiq* entries in your Gemfile with:

gem "solid_queue", ">= 0.3.0"
gem "mission_control-jobs", ">= 0.2.0"

Then you need to run bundle install, bin/rails generate solid_queue:install, and bin/rails db:migrate, to prepare the database and generate the configuration file.

After that you should set the queue adapter in config/environments/production.rb:

config.active_job.queue_adapter = :solid_queue

Finally, you need to expose jobs in your config/routes.rb. Ideally behind some kind of authentication:

authenticate :user, ->(user) { user.admin? } do
  mount MissionControl::Jobs::Engine, at: "/jobs"
end

At this point, you can find and delete the rest of sidekick related code in your project, and you should be good to go.

Configuring Recurring Jobs

The scheduling is done in config/solid_queue.yml and it’s quite flexible. The schedule key can take anything Fugit::Cron can parse. The class is the job you want to run and the optional args are the arguments you can pass to the job. It can even take kwargs as the last parameter. That all goes under the recurring_tasks inside dispatchers. Here’s an example from Visualizer:

default: &default
  dispatchers:
    - polling_interval: 1
      recurring_tasks:
        shared_shot_cleanup_job:
          class: SharedShotCleanupJob
          schedule: "@hourly"
        duplicate_stripe_subscriptions_job:
          class: DuplicateStripeSubscriptionsJob
          schedule: "@daily"
        airtable_webhook_refresh_all_job:
          class: AirtableWebhookRefreshAllJob
          schedule: "0 0 */6 * *"

Error Handling

Because Solid Queue is a new gem, you can’t assume your error tracking service will support it out of the box. For example, I’m using AppSignal, and they don’t support it yet. There’s an open issue, but not much progress yet. At least not publicly.

Luckily this gem is build by Rails core team, so they provide a very simple on_thread_error configuration option. In config/environments/production.rb, right under the queue adapter configuration, you can add:

config.solid_queue.on_thread_error = ->(error) { Appsignal.send_error(error) }

And for errors that happen in jobs, you can hook in to ApplicationJob and ApplicationMailer adding this snippet:

rescue_from(Exception) do |exception|
  Appsignal.send_error(exception)
  raise exception
end

This will send all Solid Queue and job errors to AppSignal, but you can easily change it to your own error tracking service.

Conclusion

It has not even been a day, but I’m already very happy with the switch. If nothing else it decreases my dependence on Redis. Not that I have anything against Redis, but it’s one more thing to maintain and keep an eye on.

Now the last piece of the puzzle will be switching to Solid Cache, and then I can finally say goodbye to Redis. Less service dependencies, fewer things to worry about, better uptime, happier users. 🚀

In Other News

As usual, there were many other changes since the last update, with the most significant being the introduction of a daily cap of 50 shots on the free plan. This adjustment is expected to affect only a very small fraction of users. However, those who exceed this limit are encouraged to upgrade to the premium plan, as their usage places considerable strain on the database.

Thank you! 🙏