October 15, 2022

The Big Migration

This week, I did one of the biggest things I wanted to do for a long time: extracting all the JSON data out of the main shots table. With the popularity of Visualizer, the table grew a lot, and it keeps growing faster and faster every day. The simple database design I started with didn’t cut it anymore. The Your Shots page got slower, search got slower, people listings got slower, all because that table grew over 7 GB in size.

I knew I needed to do something about it, but I also knew I can’t screw this up. It had to go super smooth without downtime, without data inconsistencies, and at every step giving me a way to revert in case something doesn’t go quite right. Migrations like this demand time and focus.

I recently became unemployed when the company I worked for has shut down. While that was really unfortunate, it also gave me the time necessary to finally tackle this thing. And, I’m happy to report, it all went super smoothly, with no hiccups, and most of the pages are now way more responsive.

There were quite a lot of other changes since the last update:

I also recently switched from PointDNS to Cloudflare, since some users were experiencing issues that could only be explained by DNS. They seem to have resolved for those users for now, so maybe it actually is always DNS.

Thanks for reading, and enjoy your weekend! ☕

August 22, 2022

RSS feeds, Private notes, and more

Even though it’s still summer, there have been several improvements since the last update.

Because of the popularity of Visualizer, there are scaling pains. Sometimes pages time out. In order to deal with this, I have to move from Heroku to another provider. Currently, I’m looking at Render, but I’m not completely set on my decision yet. Having little-to-no DevOps knowledge is definitely one of my pains. 😅

To move anywhere, however, I needed to improve several things on the backend. The biggest thing I achieved was completely removing my reliance on chromedriver to make chart screenshots.

In case you weren’t aware - every shot page comes with a tiny little meta image. So that if you paste the link to that page in Discord, Slack, Twitter, or some other tool that supports meta images, a tiny screenshot preview of the chart will appear there. Before this was done by a Rube Goldberg machine kind of process which was extremely error-prone, and taxing on my backend and deploys. Now it happens via Highcharts Export Server and is both much faster and way more reliable.

Speaking of images: as Christoph discovered if you’ve attached an image to a shot, there was no way to remove it. This has now been fixed by adding a simple Remove Image link in the shot edit form. Whoops. 🙈

There were several great ideas via GitHub Feature requests that were also added recently:

RSS feeds

Every public user now comes with an RSS feed of their shots. So you can subscribe via an RSS feed reader and be notified of any new shots they make.

Private Notes

Sometimes you want to write something down next to the shot, but don’t want the public to see that. If you’re a Premium user you’ll now have a new field under Bean notes and Notes called Private notes. Anything you enter here will only be visible to you and no one else.

August 7, 2022

Improved Decent People Page

It’s been a typical European summer for me, so I’ve been out and about a lot.

Still, there were many changes in the past month or so.

I’ve vastly improved site protection from script kiddies and other “hackers” who keep trying random URL attacks. Bless their souls. 😅

I’ve added a new upload scope to OAuth. This enables app developers to have a “low danger” scope so that even if a token is stolen, the worst that can happen is that the attacker can upload shots. But they won’t be able to delete them.

There has been a change to default chart temperature colors, making them easier to read, especially for people with certain disabilities. I want to do the same for the main chart, but I am yet to find a good palette. Suggestions are more than welcomed.

I’ve also created custom error pages, so you don’t land on the Heroku one if/when something goes wrong.

But the most significant change has been to the Decent People page. Since there are so many of you now, the page became very slow to load, but also it was hard to find anyone. Now you have a search there and can quickly narrow down and find exactly who you’re looking for.

New Decent People Page

PreviousPage 3 / 9Next