Thai-ing Up Loose Ends

It’s only been a few days since the last update, but I wanted to give you a heads-up: I’m off to Thailand 🇹🇭 and will be there until November 20th.

I’m taking my laptop with me, so I’ll keep an eye on everything and deal with whatever comes up. Replies might take a bit longer than usual though, so please bear with me. 🙏 Visualizer itself doesn’t take holidays, so everything keeps running as usual, Premium included. When I was in Japan this summer, there wasn’t a single incident, so fingers crossed for another boring few weeks. 🤞

Before leaving, I fixed something a few people ran into: downloading a shot into DYE with its share code failed for shots uploaded from Decaid, the new Decent app formerly known as Reaprime. Decaid only uploads the newer JSON version of the profile, without the TCL fields DYE reads, so DYE got an empty profile and choked on it. Now Visualizer builds the TCL profile from the JSON one, so Decaid shots download fine, even ones uploaded before the fix, and the profile download button on the shot page gives you a proper TCL file too. Shots without any profile no longer offer an empty one either.

If you build on the Visualizer API, OAuth apps can now use PKCE. It’s optional and S256 only, so apps that sign in with a client secret don’t need to change anything. Mobile, desktop, and browser apps can’t keep a secret, though, so they should be registered as non-confidential and use PKCE instead. One catch: Visualizer used to ignore PKCE parameters, and now it checks them, so an app that sends them incorrectly will start failing. I emailed everyone who registered an OAuth app with the details, and you can try the PKCE sign-in right in the API docs.

And for the RSS fans out there, the updates feed now includes the full posts instead of just the excerpts, so you can read them right in your feed reader. 🤓

Behind the scenes, Visualizer now builds on the Ruby core team’s own Docker image. Ruby 4.0.9 took about 16 hours to show up in the image I used before, and only minutes in this one, so Ruby security fixes can reach Visualizer much sooner. The switch went through without any downtime. Plus the usual dependency updates, all in the full changelog.

As always, if something looks off, please let me know through GitHub issues or email. I’ll still read everything, maybe just a bit later than usual. 😅

Thanks for reading, enjoy the rest of your week, and please try not to break anything while I’m gone! ☕


Haunted Tables 👻

It’s been two weeks since the last update, and this one is smaller. No big new features, just making things faster and fixing a few things that were quietly failing.

Turns out Journal was a bit haunted. Each page of shots loaded slower than the one before, so the deeper you scrolled, the worse it got. That one was on me: a cleanup right before the last update broke it. 🙈 While I was in there, I also found that every page of 30 shots set up about 180 little JavaScript controllers. Now a single one handles them all, pages show up about twice as fast, and the next page starts loading well before you reach the bottom.

On the API side, apps can now search community shots. This came from a PR for Decenza, so it can show what everyone else brewed with the coffee you’re about to dial in. I went with new parameters on GET /api/shots instead of a separate endpoint: community switches to community shots, and all the Community filters plus the new canonical_coffee_bag_id work with it, or on your own shots without it. Add full and you get the shot details too, just without the telemetry and with stricter rate limits. Same as on the website, only shots uploaded in the last month show up unless you have Premium. It’s all in the API docs.

Airtable got some attention too. When a base hit its plan’s record limit, new shots just stopped syncing, and the only one who knew was me, staring at thousands of errors. 😬 Now you get an email about it. Or you can skip the sync altogether with Journal. Connecting without picking a base where you’re a Creator now tells you so, and shots that still pointed to an old base after reconnecting now upload to the new one instead of failing.

A few smaller things: uploading a GaggiMate or Meticulous profile instead of a shot now tells you exactly that, GaggiMate shots without a timestamp upload fine (dated when you upload them), and picking files from Google Drive on Android should work now. Linking or unlinking a bag’s canonical coffee in Coffee Management now updates the shots you already brewed from it, filtering Community by notes, profile, or grinder is faster, and fonts now come from Visualizer instead of Google, so loading a page no longer pings Google. Everything else is in the full changelog.

As always, if something looks off, please let me know through GitHub issues or email.

Thanks for reading, and have a spooky weekend! 🎃


Autumn Cleaning

It's been two weeks since the last update, and this one is a bit different. After Journal landed, I had an AI-assisted review go through the whole app, hunting for ways things could go wrong. It found quite a few. 😬

Most of this post is about that, because you deserve to know what was wrong, what is fixed, and what changes for you.

Security fixes

I checked the logs and found no signs of any of this being abused. Still, I'm sorry these were there in the first place. The silver lining is that they're fixed now, and I know a lot more about where the sharp edges are.

Some of it was about things being visible to the wrong people. Hidden shot times, for example, could leak through related shots and pagination links. Then there was the account side: resetting your password now also revokes OAuth access, passkeys, and push subscriptions, so nobody who got in earlier keeps a back door. Passkey sign-in now asks for your PIN or biometric and is rate limited. OAuth secrets are properly hashed, Airtable tokens are encrypted, and Airtable sync and custom fields now check for Premium on the server, not just in the UI.

The scariest category was keeping the site up. A single crafted upload could have kept workers busy long enough to take the whole site down. Parsing is bounded now, files over 10 MB are rejected, and web push endpoints and webhook events are validated too.

Many other small things were fixed along the way, and you can find them all in the full changelog.

What might break for you

Tightening things up means a few API behaviors changed, so if you build on it, please check these:

  1. Access tokens in query or body params no longer work. Send them in the Authorization: Bearer header.
  2. Tokens without the read scope see public data only. Shot reads treat them as anonymous, with no error.
  3. Unconfirmed and disabled accounts get a 403 from the API.
  4. Premium and authorization failures are always a 403 JSON error.
  5. After a password reset, integrations must be authorized again.
  6. Old OAuth app secrets can't be recovered. They're only stored hashed now, so if you never saved one, you'll need a new app.

Sign-ups

I noticed a lot of bots and fake accounts when I sent the newsletter and got a bunch of bounced emails and spam reports. Not a great look. 😅

So new accounts now need to confirm their email, sign-ups from domains without MX records are rejected, and sign-up attempts are rate limited. All existing users were marked as confirmed. Accounts left unconfirmed for 7 days are deleted, or disabled if they have shots.

Uploads, API, and behind the scenes

Invalid files now get a clear message (a 422 in the API) instead of a 500, and dropping lots of files uploads them in batches. Smart Espresso Profiler CSVs parse faster, and Decent .tcl files about 9x faster with a new hand-written parser. I also fixed a .tcl export bug that produced files the Decent app couldn't load.

On the API side, archiving coffee bags was requested, so integrations can now do exactly that. Roaster and coffee bag images can be uploaded too, and the documented upload scope, which was actually being refused as invalid, works now. The OpenAPI spec got a full rewrite while I was at it.

Images are now decoded and resized in a locked-down sidecar container called Hot Cell, introduced by 37signals, instead of inside the Rails app, so a malicious image file stays contained there.

Grind suggestions

Last time I said I was experimenting with Jev for grind suggestions. I removed it, since it was only enabled for me and wasn't as useful as I hoped it would be. I even tried replacing it with Cloudflare's Clef, but it was even worse. 🤦‍♂️

If something looks off after all these changes, especially in the API, please let me know through GitHub issues or email.

Thanks for reading, and enjoy your weekend! ☕