This was an absolute nightmare and I want to thank @fessyfoo for spending the last seven or eight hours helping me diagnose, resolve, and isolate the multiple issues causing it. Digital Ocean’s support was hilariously abysmal. They literally ran me around in circles for two days, and those days were days I was not off of work and my job now is both physically demanding and long in hours during which I have no access to computers. What precious little time I did have, they wasted. Discourse support is essentially limited to a dead forum, and I was about to pay a professional when fess stepped up and offered to help. I was convinced that the issue was on Digital Ocean’s end but it turned out to be a number of small obscure issues with Discourse itself that fess and I found while working together over google meet. At this point I think we know more about Discourse’s inner workings than any sane person should.
I really appreciate everyone’s patience and understanding. Everything should be pretty smooth from here on out. Yes we have backups, that was never an issue. The forum’s data was never in danger and if we had ended up having to nuke the site and start over we could have done so and restored the data. I think, and I believe fess agrees, that understanding these issues was more important. Knowing what happened means the issues can be checked for and fixed more quickly in the future.
However, we came out on the other side with a fresh upgrade to the latest version and more disk space for very important things like cat pictures, dramatic arguments over esk8 builds, and, most importantly, admin bashing.
Now, if you will excuse me, I have a fresh bag of Doritos to attend to.
Running a forum like this is often a thankless job but for guys like myself, I like to read and laugh with/at the community that has been built here. I know I speak for a lot of us when I say thanks for keeping it up and keeping our small community out of the hands of corporations.
I’d also like to mention that the backup frequency has been increased to daily from weekly, and I’ll be working on a script tomorrow that will automatically download those backup files from S3 to my external drive at home (which is also backed up to dropbox) so that if they are needed, I’ll have them on hand and can more rapidly restore from them.
I may also be looking into alternative hosting solutions. As reliable and fast as DO is, their support is terrible. I didn’t expect them to fix an app issue, but I did at least expect them to read the ticket history before responding. I could probably do better by putting a box in my closet on my gigabit fiber connection with a UPS on the server, fiber node, and switch, but I’m not itching to do that. However, reducing hosting cost is a concern. The current setup is a bit pricey, which brings me to another issue.
I have been looking into running ads on the non logged in traffic. if you are a registered user and logged in, you wouldn’t see them. Patreon has never covered the cost of hosting, it covers roughly half the cost. As much as I abhor ads and what the ad networks do with the data, I may have little choice. And no, this site would not generate the kind of revenue I would need to roll around in cocaine and hookers. But it might fill the gap or at least not hurt as bad.
It’s called Discourse. It’s very complex system wise but in general kind of easy to install and use. Unless something goes weirdly wrong, then you’re dealing with a lot more than permissions issues. There’s a number of individual internal system components and daemons and all have configuration files and mappings, so it’s not just a directory of Perl or php scripts with some code to drop into the web server config. The install script does everything for you though. Usually. It’s also pretty portable and they maintain it constantly. Portable because it’s containerized with its own stripped down version of Ubuntu with all utilities and services included and packaged inside.
I’ve used vbulletin in the past as well. Also phpBB, which was much simpler than this in the background. It’s probably been 20 years since I’ve used either of those.
Not sure what’s happening but the forum just choked again and I had to restart the container to bring it back. and now it’s slow af. Trying to figure out what’s going on this time.
Discourse backup files are roughly 8 gigs compressed and uploaded, with a maximum of five files. And that’s strictly just post, template,settings, etc data. The actual disk space used by hosting the forum is between 250 and 300 gigs, I’d have to look again for an exact number. And that does not include uploads and backups because those are on S3. It’s kind of massive.
As a recent user just wanted to say thanks for all the work getting the forum back up and it’s great that there’s backups of all the info.
I was thinking about how crazy the amount of VESC setup info (and vital literal files like the Jaykup firmware to get Flipsky 75100s as good as they can be) and general common technical knowledge around MTB type eskates I was able to learn just from the threads here in a short amount of time & just finishing up my builds mostly made of parts from sale threads here - right before the forum went down ha.
So it’s great it came back and that the data wasn’t in danger of being lost.