Some more details, for anyone interested in the nitty gritty about this transition.
ConversionFor starters, how difficult was it to actually physically convert the forum from IPB (Invision) to SMF? Well, there is a converter, but it's absolute garbage. I used it but ended up writing about 2000 lines of my own code just to make it halfway decent.
I then ended up writing about 3000 more lines to supplement and further fix the converted data. Just about everything was subtly wrong and had to be corrected, from member names being improperly escaped to forums being ordered wrongly. IPB also only stores posts in HTML in the database (and converts to and from BBcode when editing), but because IPB uses subtly different code than the older forum software I had to write my own code to convert HTML back to BBcode. Finally there was additional data from the Wayback Machine that I needed to scrape and integrate into the database (more on that below).
I'd say that the regular converter got things maybe about 75% correct, but that last 25% necessitated rewriting almost everything because you can't fix improperly converted data after the fact.
It was just an absolute slog, and every single old forum software we've ever used had left its own unique mark on the database that needed to be accounted for. There were also tons of edge cases and unexpected problems that only occurred for some posts. For literally months my workflow was just: convert the forum, browse random old posts, find new problems I hadn't seen before and then write code to fix them.
QOL improvementsOne of the things that really bothered me about the IPB forum was how we'd regularly have old topics get bumped, mostly by spambots. There's no reason why that should be possible, so
from now on old topics actually get auto-locked after a while. Same with old polls.Speaking of
spambots, we now have a
much better way of dealing with them through a single form that instantly bans and deletes all their posts.
Another interesting thing is that SMF normally logs every topic view regardless of whether you're logged in or not. That worked fine in the 2000s, but today web crawlers are so unbelievably ubiquitous and aggressive that this led to even the most completely unremarkable posts having hundreds of thousands to millions of views. Using historic data, I was able to figure out roughly how view counts and reply counts interact over time, and used that to "restore" more accurate view counts. These figures are mostly completely made up, but I feel like it's fine considering crawler view counts aren't particularly "real" either. From now on only views are counted if you are logged in.
In general, the forum has been tightened up a lot compared to the old version. Still, there's more work that needs to be done, and feel free to let me know if you think anything specific needs addressing.
Old imagesThere's decades of old demo and screenshot topics on GW, but by now most of those images are now long gone from the internet. Fortunately, the Wayback Machine still had a ton of them, and so I decided to leverage that to grab copies, host them locally, and change old posts to utilize the new URLs.
Similarly, Photobucket was used as image host by many people, but today it displays watermarks on all images when linked directly. I was able to figure out how to request the images without watermark and ended up replacing them too.
In total, 14,081 images in old posts were brought back from the void.Post iconsAs the Invision forum had no concept of post icons, the database did not include any data related to them. Fortunately I was able to replace a lot of them using the Wayback Machine data.
Still, that left every single post since 2013 with only the default post icon, plus thousands others that weren't part of the archive. Setting new post icons manually isn't exactly feasible, so I opted to run an experiment to have ChatGPT set them for me. Finally, a good use for AI. I wrote a big prompt that basically said "here's our post icons, and here's our topic names and this is what forum they're in: pick the best one that matches for each". This seems to have more or less worked out OK. Again, this is all completely made up data, but it's better than nothing.
The post icons themselves aren't actually 100% complete. We actually had two sets: the originals, and a set of completely remade post icons by Konix, created somewhere around 2010. The latter have been
completely lost, save for a single one that happened to be in a screenshot I took. (If anyone has these, let me know.) I've made new post icons for a couple of missing ones, but still a few of them are placeholders for now.
What still needs to be done?- Probably the most important thing is that, right now, the design doesn't work well on mobile at all.
- There's a couple of areas that haven't been skinned properly yet, mostly admin/moderation areas.
- There's currently no visual editor, just BBcode.
- At the moment, you can only register regular accounts and OpenID does not work. (If you were using OpenID login before and have been locked out, contact me on Discord.)
- I'm sure a bunch of things are broken purely from moving it to my dev environment to the live server.
- Obviously some graphics are still missing (like a couple of post icons).
- Currently old links to Salt World are not working properly. I'm going to be writing a redirect script that fixes this.
- The email sending functionality doesn't work. Oops. If you can't get into your account (you should be able to login using your old data, though), let me know on Discord.
- Tons of other things, no doubt.
Just let me know if something isn't working.
I'd like to help!At the moment, the most helpful thing is to just use the forum. See if it works for you, try to find out if anything doesn't work well. Tell me what you think! I'm open to suggestions and I have some ideas about things to add from here as well.
Gaming World Forums
Messages
