=][= ACCOUNTS PRUNED =][=

[Brother Tyler]

In an effort to keep the database lean, a lot of accounts were pruned (i.e., deleted) over the weekend. Our criteria for these account deletions:

• Member has zero content* (e.g., no topics, no replies, etc.).
• Member has not been active (i.e., they haven't logged in) in over 10 years.
• Member is in the + FRATER NOVUS + or + FRATER DOMUS + group.

All three criteria had to be met for the account to be pruned. If any of these were not met, the account was retained.

 

Some members had zero content because they never actually participated, only creating an account and then lurking. Some of these members never participated at all, dropping out of the hobby/community in time; and others created other accounts later which became active.

 

Other members had zero content because of the time the system crashed and our database was wiped out. That was in 2005 and we lost a lot of really great content.

 

Still other members had zero content because of an old practice which has since been abandoned. When the site was first created, we (the staff) would periodically delete old discussion topics. That tapered off, though we occasionally deleted rumors and event topics once the products were released (or disproven) or the events had passed. When we were preparing for the software update in 2022, we deleted a lot of discussion topics in an effort to keep the database lean. However, we discovered that the removal of this content had a negative impact on members' achievements - the system could not count content that no longer existed. So we dropped that practice in 2022, instead creating a location for obsolete content which would remove it from public visibility while ensuring that the system would preserve the contributing members' achievements.

 

So some of these members may have been active one time, but all records of that activity (i.e., the content they submitted) has been lost.

 

Members in other groups that met the other criteria were not deleted for record-keeping purposes. Anyone who served as a staff member has both activity (even if you can't see it) and a unique title. Likewise, it is important to preserve the members who have been permanently banned for misbehavior so that we have a record and can refer to that content if they ever return to the site and we are able to associate the new account(s) with the previous misbehaving accounts, which has happened and will likely happen in the future.

 

The pruning of these accounts is only one part of a larger effort that will include moving to new (and more capable) servers, fixing our database, updating to the new Invision 5.0 software (happening Soon^TM, though TBD), and some other technical things that require someone who's not a knuckledragger (i.e., not me) to explain. In truth, this may have little impact on its own.

 

If we repeat this pruning in the future, it will likely only affect members who join and then lurk. The only exceptions might be if we have a repeat of the Great Crash, but our backup process should mitigate that to the greatest extent possible (i.e., we'll only lose a fraction of content between the time of the most recent backup being taken and the crash).

 

* Note that the "zero content" description may be inaccurate. While these members had zero topics, discussion comments/replies, blogs/blog entries, articles, and files, some may have had images/albums in the Gallery. Some definitely had avatars/personal photos and some may have had signatures/About Me pages. Since these members don't have any other content (topics, comments, etc.), however, any images they may have had in the Gallery were either used at external sites or simply resided in our Gallery (i.e., in our database) with no other use. If so, these images were populating our database and putting a pointless load on our server. In the limited sampling we took of the members who met the criteria identified in the bullets above, however, none of the members had such images.

 

If you have any questions/concerns about this (especially if there are things we should incorporate for the future), feel free to post them here.

Edited Thursday at 07:28 PM by Brother Tyler
Added clarification that all three criteria had to be met

[grailkeeper]

I remember a load of accounts that hadnt been active in ages suddenly got a load of awards recently. Is this something to do with that?



I clicked one randomly. It had one post from 2009 with some pictures, saying it would post more soon. I was tempted to ask where the new pictures were!

[Inquisitor_Lensoven]

Damn, I was gone for like 10 years lol, didn’t do any pruning back then?

[Sothalor]

Ten years ago was... 2016.

 

What.

[Brother Tyler]

2 hours ago, grailkeeper said:

I remember a load of accounts that hadnt been active in ages suddenly got a load of awards recently. Is this something to do with that?



I clicked one randomly. It had one post from 2009 with some pictures, saying it would post more soon. I was tempted to ask where the new pictures were!

No, anyone with achievements other than longevity (e.g., ) has content, so they didn't meet all three criteria.

 

2 hours ago, Inquisitor_Lensoven said:

Damn, I was gone for like 10 years lol, didn’t do any pruning back then?

This is the second time that we've done this sort of pruning. In all cases, the members had to meet all three of the criteria I listed.

 

21 minutes ago, Sothalor said:

Ten years ago was... 2016.

 

What.

Indeed. Keep in mind, however, that members had to meet all three criteria for their accounts to be pruned. If someone had any existing content and hadn't been active in ten years or more, their account would not have been pruned.

[mel_danes]

How many accounts were pruned in this action?

Did it achieve the goals you needed it to, lighter load, fewer accounts to track, etc?

[Brother Tyler]

4 minutes ago, mel_danes said:

How many accounts were pruned in this action?

Did it achieve the goals you needed it to, lighter load, fewer accounts to track, etc?

It was somewhere around 8,000 (we went from 22,000+ down to 14,000+ members).

 

We won't know how much of an impact the pruning had until Monday (or @WrathOfTheLion is able to perform some analytics).

[roryokane]

The idea that we lost a load of data in some sort of big crash is so completely appropriate for the setting. Way to really live the hobby, Admins! ;)

[kabaakaba]

@Brother TylerI'm sorry, but could you share your/our server configuration? I mean 20k accounts shouldn't break db on load if you don't run it on pentium3. Is an indexes are built for all heavy tables like users and posts? As far as I know Invision have solid code base and all requests should be optimized.

Also how often you do db optimisation?

Do you use redis for caching?

Edited Thursday at 10:39 PM by kabaakaba

[Adeptekon]

2 hours ago, kabaakaba said:

@Brother TylerI'm sorry, but could you share your/our server configuration? I mean 20k accounts shouldn't break db on load if you don't run it on pentium3. Is an indexes are built for all heavy tables like users and posts? As far as I know Invision have solid code base and all requests should be optimized.

Also how often you do db optimisation?

Do you use redis for caching?

Regardless it's kind of a security measure. On a slightly related note, I also like the fact that instead of mods harping about necro threads just close after a long period of time. 

[Brother Tyler]

18 hours ago, kabaakaba said:

@Brother TylerI'm sorry, but could you share your/our server configuration? I mean 20k accounts shouldn't break db on load if you don't run it on pentium3. Is an indexes are built for all heavy tables like users and posts? As far as I know Invision have solid code base and all requests should be optimized.

Also how often you do db optimisation?

Do you use redis for caching?

@WrathOfTheLion will be able to provide much better answers to these questions. We are working on getting new servers, however, as well as some other technical improvements.

 

Ultimately, I doubt that this member pruning will have a significant impact on our performance. Getting rid of 1/3+ of our accounts is of a little help, especially in cases where the system recalculates members' achievements, but these unused accounts provide almost no other demands on the system. And while my very limited sampling of these accounts showed no images, there's a chance that, hidden in the 650+ pages of these members, some might have had images in the Gallery. I think it's a very slim chance, if any. If any of these accounts did have images in our Gallery, that cleans out a little more of our database. That is probably insufficient to realize any real performance improvements, but it cleans up a little bit of noise.

 

The real benefit that members have is that some display names that were previously used are now available again. The more important things that will improve our performance will be in other areas. This was simply one of those small things that I'm capable of (i.e., knuckle-dragger level work as opposed to work requiring real tech-adepts).

[WrathOfTheLion]

As far as performance goes, there are a couple sources. The first is that Invision 4 has performance issues with recalculating badge statistics, which is the main cause of periodic slowdowns. This may be fixed in Invision 5. Pruning accounts should mitigate that issue somewhat, but I also may have to manually patch the Invision codebase.

 

The other thing is that there may be an issue with cloudflare, but I haven't checked yet.