General Category > General Talk

Welcome to Gaming World

(1/8) > >>

Hi everyone and welcome back to Gaming World! tl;dr is below.

You're probably wondering why the forum looks like it did back in 2006. For the past two years I've been working on and off on converting the Invision forum back to SMF, the software we used prior to 2013. It's actually, almost to the minute, exactly 11 years ago since we launched the last incarnation of the forum (Jan 10, 2013).

So why go back to such an antiquated version of the site? Well, when you think about it, this is the most iconic period of our community. This is when the site was at its most active and when it was the most relevant in terms of reach and impact on game dev. I don't think I'm wrong to say it's probably the version that we have our most fond memories of.

This is also not actually the "real" old forum from back then—it has been largely rebuilt. It uses a much newer version of the same forum software (although not the very latest, and an upgrade will happen eventually) and the theme has been rebuilt from scratch. A ton of QOL updates have been made and while there's still some things that need to be improved, it should be much better and more functional than the old Salt World forum.

It wasn't easy to get to this point. When I inherited the Salt World forum, the database was already pretty fragmented. A lot of very oldest posts are also just missing. I don't know where they went. There was a gargantuan amount of legacy data that was never properly converted. Did you know that the Invision forum was actually the 7th forum software we used? All that data ended up getting transferred from software to software with mostly extremely cookie cutter conversion scripts.

Most of my work, and the reason it took so long to get this project up and running, was just cleaning up the database and filling in all the gaps. Literally everything was a jumble, and every past incarnation of the forum required a subtly different approach to bring it up to a good standard.

Tons of data was also just gone: old post icons, member titles, signatures, avatars, member groups. In fact, there was no direct backup whatsoever of the old forum theme and all its associated graphics. All that has been essentially lost to time.

Yet a lot of it has now been brought back from the void. I cobbled together the old theme from my own backups and old screenshots, and got a ton of old data from the Internet Archive. As it turns out, GW was actually a pretty well scraped website and so there was a surprising amount of useful data on the Wayback Machine. I ended up getting a complete copy of everything on there and writing custom scrapers to extract the data from the raw HTML.

I wanted to bring back lost old posts from this backup as well, but that proved too difficult for the moment. While there is a lot of data, it is sparse and incomplete, and when we moved to Invision all the topic and post IDs changed and got recounted from zero, which significantly hinders any effort to use the data.

None the less, things are more or less the same as they were before 2013, and old accounts have been given a copy of their old avatar and signature. Post icons have been restored, premium member groups are back, and I've even restored thousands of broken images from old screenshot and demo topics and the like.

tl;dr: after a herculean effort, the old GW forum has been restored for great justice. RIP Salt World.


Some more details, for anyone interested in the nitty gritty about this transition.


For starters, how difficult was it to actually physically convert the forum from IPB (Invision) to SMF? Well, there is a converter, but it's absolute garbage. I used it but ended up writing about 2000 lines of my own code just to make it halfway decent.

I then ended up writing about 3000 more lines to supplement and further fix the converted data. Just about everything was subtly wrong and had to be corrected, from member names being improperly escaped to forums being ordered wrongly. IPB also only stores posts in HTML in the database (and converts to and from BBcode when editing), but because IPB uses subtly different code than the older forum software I had to write my own code to convert HTML back to BBcode. Finally there was additional data from the Wayback Machine that I needed to scrape and integrate into the database (more on that below).

I'd say that the regular converter got things maybe about 75% correct, but that last 25% necessitated rewriting almost everything because you can't fix improperly converted data after the fact.

It was just an absolute slog, and every single old forum software we've ever used had left its own unique mark on the database that needed to be accounted for. There were also tons of edge cases and unexpected problems that only occurred for some posts. For literally months my workflow was just: convert the forum, browse random old posts, find new problems I hadn't seen before and then write code to fix them.

QOL improvements

One of the things that really bothered me about the IPB forum was how we'd regularly have old topics get bumped, mostly by spambots. There's no reason why that should be possible, so from now on old topics actually get auto-locked after a while. Same with old polls.

Speaking of spambots, we now have a much better way of dealing with them through a single form that instantly bans and deletes all their posts.

Another interesting thing is that SMF normally logs every topic view regardless of whether you're logged in or not. That worked fine in the 2000s, but today web crawlers are so unbelievably ubiquitous and aggressive that this led to even the most completely unremarkable posts having hundreds of thousands to millions of views. Using historic data, I was able to figure out roughly how view counts and reply counts interact over time, and used that to "restore" more accurate view counts. These figures are mostly completely made up, but I feel like it's fine considering crawler view counts aren't particularly "real" either. From now on only views are counted if you are logged in.

In general, the forum has been tightened up a lot compared to the old version. Still, there's more work that needs to be done, and feel free to let me know if you think anything specific needs addressing.

Old images

There's decades of old demo and screenshot topics on GW, but by now most of those images are now long gone from the internet. Fortunately, the Wayback Machine still had a ton of them, and so I decided to leverage that to grab copies, host them locally, and change old posts to utilize the new URLs.

Similarly, Photobucket was used as image host by many people, but today it displays watermarks on all images when linked directly. I was able to figure out how to request the images without watermark and ended up replacing them too.

In total, 14,081 images in old posts were brought back from the void.

Post icons

As the Invision forum had no concept of post icons, the database did not include any data related to them. Fortunately I was able to replace a lot of them using the Wayback Machine data.

Still, that left every single post since 2013 with only the default post icon, plus thousands others that weren't part of the archive. Setting new post icons manually isn't exactly feasible, so I opted to run an experiment to have ChatGPT set them for me. Finally, a good use for AI. I wrote a big prompt that basically said "here's our post icons, and here's our topic names and this is what forum they're in: pick the best one that matches for each". This seems to have more or less worked out OK. Again, this is all completely made up data, but it's better than nothing.

The post icons themselves aren't actually 100% complete. We actually had two sets: the originals, and a set of completely remade post icons by Konix, created somewhere around 2010. The latter have been completely lost, save for a single one that happened to be in a screenshot I took. (If anyone has these, let me know.) I've made new post icons for a couple of missing ones, but still a few of them are placeholders for now.

What still needs to be done?
[*]Probably the most important thing is that, right now, the design doesn't work well on mobile at all.
[*]There's a couple of areas that haven't been skinned properly yet, mostly admin/moderation areas.
[*]There's currently no visual editor, just BBcode.
[*]At the moment, you can only register regular accounts and OpenID does not work. (If you were using OpenID login before and have been locked out, contact me on Discord.)
[*]I'm sure a bunch of things are broken purely from moving it to my dev environment to the live server.
[*]Obviously some graphics are still missing (like a couple of post icons).
[*]Currently old links to Salt World are not working properly. I'm going to be writing a redirect script that fixes this.
[*]The email sending functionality doesn't work. Oops. If you can't get into your account (you should be able to login using your old data, though), let me know on Discord.
[*]Tons of other things, no doubt.
Just let me know if something isn't working.

I'd like to help!

At the moment, the most helpful thing is to just use the forum. See if it works for you, try to find out if anything doesn't work well. Tell me what you think! I'm open to suggestions and I have some ideas about things to add from here as well.

first post lol

Excellent work my dude! Nothing quite hits that nostalgia button like 2006 era GW forums. You’ve clearly put a lot of time and love into this.

How dare you double post :gunz:

And yeah I'm really just glad that v1 is out, because it took much more work than I expected lol


[0] Message Index

[#] Next page

Go to full version