=][= Adjustments to Achievements and Ranks in Development =][=

[Brother Tyler]

Behind the scenes, the staff has been working on adjustments to the achievements and the ranks. The current structure (described here) was a "best guess" that was developed when we underwent the software upgrade a few years ago. We have identified several areas where this structure can be improved. The intent with the achievements is to encourage/reward positive behaviors, recognizing members' contributions to the community. Key changes:

• Increasing from 5 steps to 20 steps
• Flattening curves
• Adjusting goals to be more achievable
• Adjusting achievement points values
• Increasing the ranks

The current achievement steps are green-blue-bronze-silver-gold. In increasing from 5 to 20, we are adding a few colors, but are creating the additional steps by having multiple levels of each color. The green and gold badges will each have a single level. The green badge indicating to a member that the behavior is one that is recognized as an achievement and for which continued instances of that behavior will result in more badges. The gold badge is the highest badge in any progression and indicates that the member has hit that top milestone (and likely exceeded it). We are adding purple, red, and black in between the blue and bronze badges. The blue, purple, red, black, bronze, and silver badges will all have three levels, with Roman numeral I/II/III indicators. So after receiving the green badge, a member will receive the blue I badge at the second milestone, the blue II badge at the third milestone, the blue III badge at the fourth milestone, the purple I badge at the fifth milestone, and so on.

 

The current progressions are decidedly non-linear, sometimes drastically so. The modified progressions are much more linear, the curves largely being flattened. In part, this is due to making milestones more achievable. We examined how many members had earned each of the existing achievements and determined that some of the gold badge milestones were too high. A few of the behaviors are currently single-milestone, but those have been incorporated into the 20-step progressions. Previously, members were limited to a single blog, so there was only a single badge. In evaluating the blogs, however, we determined that allowing members to create multiple blogs was much more useful. Similarly, the award for creating clubs was previously limited to a single badge because we didn't want to encourage members to create useless clubs. With our awareness of how clubs can be used on the site, however, it makes sense to encourage the creation of constructive clubs, and to award achievements for that behavior. Here are some examples of a few behaviors and how they are being changed. The red line shows the current progression (along 5 steps) while the blue line shows the modified progression (along 20 steps).

 

 

Members are able to create discussion topics in the FORUM feature (obviously), but they are also able to create discussion topics in the CLUB feature (assuming the club has one or more discussion forums). As of 14 February, only 1 member had created 1,000 or more topics and 6 had created 500 or more topics.

 

 

Under the current rules [for achievements] comments/replies in all areas are counted in this progression. This includes replying in topics (obviously), but also includes comments on events, gallery images, files, downloads, blog entries, etc. As of 14 February, only 35 members had posted 10,000 or more comments/replies and 119 had posted 5,000 or more comments/replies. While we are flattening the curve, we are also breaking the comments/replies out into different progressions for different areas. So there will be a progression for comments/replies in BLOGS, another for comments/replies in the GALLERY, etc. The blue progression shown above is for comments/replies in the FORUM (where the majority of comments/replies are currently located).

 

 

As of 14 February, 30 members had uploaded 1,000 or more images and 94 members had uploaded 500 or more images. The only images that count towards this progression are those that are uploaded into the GALLERY feature. When a member directly embeds an image into content (e.g., by pasting an image or off-site URL into a post), that image (a) doesn't count towards the progression, and (b) will be automatically deleted by the system after a period of time (this is not something that we have any control over).

 

As the comment/reply progression shows, we are also creating a few new progressions (mostly breaking them out from currently consolidated progressions). This necessitated the creation of a few new designs. I also made some tweaks to create an overarching system, so everything to do with the FORUM feature shares design cues, everything to do with the ARTICLE feature shares design cues, etc. Most of the designs had to be tweaked so that we could fit the I/II/III indicator (I chose the center bottom). Also, standardized iconography for comments/replies and reviews was created. So all review achievements feature a design of five stars across the middle/bottom of the badge, above the I/II/III indicator and superimposed over the feature design (e.g., the badge for a review of an album features the five stars superimposed over the four image icon of the new album badge).

 

I spent all day yesterday working on the badges. At this point I've created the basic designs. I'm considering creating some images for the green and gold badges to fill the space that other badge colors use for the I/II/III indicator, and I may tweak a few of the images a little (I'm not satisfied with the review image). After that it will just be a matter of creating the new backgrounds (purple, red, and black) and then creating the new badges. That will take some time.

 

Under the current system, each achievement milestone is worth 1 point, and this includes the longevity achievement milestones (1 year, 5 years, and then every multiple of 5 years thereafter). With longevity achievements worth points, members can advance in rank simply by logging in without otherwise participating in the community. We also observed that it was very easy to reach higher ranks. In one example, a new member that had been here for less than a month reached the Veteran rank. That seemed counterintuitive. Under the modified system, achievements will be worth 0-4 points (unless we decide to modify that to 0-3 or 0-5). The concept is that the baseline achievements are worth 2 points at each milestone. "Importance" of achievements is based on their ease/difficulty of performing (level of effort) and their contribution to the community. Some behaviors are extremely easy, but contribute little to the community (or their contribution is reflected in other ways). For example, voting in a poll is relatively easy and the contribution is reflected in the outcome of the poll. Also, a more meaningful contribution to the poll is often posting a comment/reply to help others to explain differing viewpoints on the issue being voted upon. So voting in a poll is worth 0 points (that may change as we refine things). You'll still receive badges for voting because we want you to vote and help the community to make decisions, but voting in polls doesn't contribute to rank progression (see below). Meanwhile, a fully realized participation in the hobby involves assembling/painting miniatures and playing games with them, which is best covered in the BLOG feature. So submitting blog entries is worth the highest amount of points (whatever that ends up being, currently 4). We have only started working on the weighting of achievements, but we're trying to keep a sort of bell curve (few 0/max achievements, most somewhere in the middle).

 

The current ranks are Aspirant, Novitiate, Battle-Brother, Veteran, and Venerable Brother. We are definitely adding the Honoured Brother rank (though I've been wishy-washy about whether it should be immediately before or after Venerable Brother). We may add one other rank (Revered Brother?), though I'm not sure that's necessary. We are deliberately avoiding actual ranks (e.g., Sergeant, Lieutenant, Captain), and though we've discussed lore-base steps through a Codex Chapter, we're probably going to stick with the simpler progression. Yes, we're going to stick with an Adeptus Astartes theme based on the site's mascot Chapter, the Legio, and our origins as a Space Marine discussion forum. So I need to create at least one new rank badge, and perhaps a second.

 

Only a few achievements will be considered "core" or "pacing" for the purpose of calculating ranks. Other achievements will earn points and those points will contribute towards a member's rank, but we're only using the core/pacing achievements in calculating the points required to reach each rank. In addition, we are working on notional timelines for members to reach each rank. Our current working numbers are 3 months to reach the Battle-Brother rank, 1 year to reach the Veteran rank, 4 years to reach the Honoured Brother rank (assuming that one comes next), and 7 years to reach the Venerable Brother rank. We'll then make an assessment of which milestones a member would achieve in each of the core/pacing behaviors at those points in time, assuming regular moderate participation over that span of time. The total points a member would earn at each of those points in time, then, become the points for the rank. Since different members participate in different ways, however, they may have more/less participation in those core/pacing behaviors. The other behaviors offer alternatives. In addition, members can lose achievement points as a result of disciplinary action; and having core/pacing achievements and other achievements offers a way for these members to recover rank. So while we may determine that it requires 50 points to reach the Battle-Brother rank based on the core/pacing achievements, members can earn those 50 points via the other achievements.

 

Ultimately, we want members to have flexibility. Each of us participates in the hobby in our own way, and we each participate within the community in our own way. Achievements, achievement points, and rank don't really do anything in that they don't confer authority or privileges. However, they serve as rough indicators of a member's contributions to the community over time. They don't necessarily speak to quality, which is reflected in reputation (earned from the reactions other members assign to one's content), but are simply an indicator of the quantity of one's contributions to the community. They are, as one member put it, imaginary Internet points. They are not pointless, however, as they both encourage and recognize constructive behaviors within the community.

 

To reiterate, however, this is still a work in progress and we have not made our final determinations on everything. Once we get everything done, we'll have to add the new badges, adjust all of the rules, add the new rank(s) and icon(s), and adjust the disciplinary rules (and figure out where we'll need to fairly adjust the achievement points of those members that have been subjected to these disciplinary measures). There is a lot of work that has to be done in one fell swoop, which will be time-consuming. And we may go through several iterations of the points assignments for each rank, just like last time.

[Brother Tyler]

A few more updates as we continue to work on this...

• Longevity is back in - members will receive 1 achievement point at each longevity step (1 year, 5 years, and every 5 years thereafter).
• The current top rank is "Venerable Brother" and that will soon be changed to "Honoured Brother." Once we make the changes, there will be two ranks above Honoured Brother - Venerable Brother and Ancient Brother. Why change "Venerable Brother" to "Honoured Brother" and then create a new Venerable Brother rank? Math. If we don't do that, many of you will think that you're being "demoted" which isn't what's happening. I spent over 8 hours yesterday and a few hours today analyzing this (and now my head hurts). And "Revered Brother" sounded arrogant, so it has been changed to "Ancient Brother" since that sounds a bit less smug.
• We're looking at ways to offset the great crash that took place in 2014. A ton of content was lost, meaning many members' achievements don't reflect that lost content. The amount of content varies from member to member, however, so we're going to look at a reasonable number of achievement points (along with a spiffy badge). It will be something along the lines of a "Great Crash Survivor" achievement. I'm looking for a way to have it awarded automatically so that we don't have to go through a database of over 50,000 members to figure out who gets it and have to manually award all of those members.
• We're also looking at ways to offset our now-abandoned policy of deleting content after it became obsolete. This was something that we did to "clean up" periodically when the site was first created during the ezBoard days, and we continued that practice (albeit sporadically) up through the software update/server migration in 2022. At that time, however, we realized that deleting content screwed members out of achievements. For example, many members' post counts went down (some of us lost over 10,000 posts), so we have adjusted that policy so that no content is removed (minus content that isn't appropriate to the site, and that will get you disciplined). Since content generally wasn't deleted unless it was over 1 year old, we're looking at a way to give members who joined prior to May 2021 a few points. Again, I'm looking for a way to have it awarded automatically to avoid the pain of manually awarding a ton of members. This is also intended to offset the fact that we once had blogs, but removed that feature and all associated content (my original blog was lost ). Naturally, the loss of the old blogs feature doesn't affect a whole lot of people, but there were quite a few of us that had blogs way back when (and it's great to see them back).
• The points values for achievements have been adjusted to 0-3 (vice 0-4). The swing up to 4 points created too large of a gap, especially since one of the core features is also a new feature and many members have yet to leverage it (BLOGS). This adjustment makes it easier for members to make up for points in one area by contributing content in other areas.
• Finally, more work is taking place to smooth some of the progressions. The initial progressions sounded fine "in my head," but looked bad once I saw them visually. For example, comments/replies (see graphic in post above) might be adjusted with the top end remaining the same, but the middle values raising slightly to get a less dramatic curve. All in all, the goal is to establish reasonable and achievable progressions.

For what it's worth, the core features of the community are the FORUMS, the GALLERY, and the BLOGS. Achievements/behaviors in these areas drive the requirements to reach each rank. So creating topics, replying in discussions, creating blogs/blog entries, commenting on members' blog entries, creating albums, and uploading images set the pace. Members can easily earn points in many other areas, however, so there's no requirement whatsoever to use all of the core features. Things like creating events in the CALENDAR, submitting ARTICLES, creating CLUBS, and uploading files in the DOWNLOADS are great and will earn points, too. The existing topic describing the achievements and ranks will be updated to include all of the relevant information. At this point, it's too soon to describe what things will look like since this is still a work in progress and things may change.

[Brother Tyler]

Okay, it looks like we'll be kicking this into gear soon, with the work beginning next Thursday and the final recount being completed either Friday or Saturday. Of course, my wife has the week off so all of my planning might be for naught. When the work starts, things are going to look wonky. This means that your current awards will disappear/change, and your rank may change. Once the recount is completed (and a couple of small details are adjusted*), things will settle into their final form. For those of you who are concerned, I recommend taking a screenshot of your current awards and rank.

 

Note that all of the achievements are changing, including the milestones (not counting the green/1 milestones) and the badges. In most cases, the new badges look similar enough to the old (current) ones that you'll be able to make the associations. With the increase from 5 to 20 levels for each achievement, everyone will have more achievements. In addition, many (most) of the achievements will see the gold level milestone reduced from the current level. There are a few where the gold level milestone decreases, and one that stays the same, but the majority are decreasing. Where ranks are currently based on all achievements being worth 1 point, the new system bases rank requirements on only a few of the achievements, driven by our core features of the Blogs, Forums, and Galleries. So all other achievements are in addition to those core features, making it easier for members to reach ranks in their own way. Achievements are worth varying amounts of points, however, so each achievement type can be more or less helpful. We used a control group of members to evaluate how well the new system preserves ranks, and it appears to do a pretty good job (especially with the increased number of milestones and the survivor achievements). It's possible that your rank will change, either increasing (more likely) or decreasing (less likely, but firmly within the realm of the possible). Keep in mind that seeing brand new members with less than 30 days in the community reaching the Veteran rank was one of several reasons that we decided to revise the achievements. Our goal was that the majority of members who are currently in the Honoured Frater rank (used to be called the Venerable Brother rank) should be in that rank or higher when the new system is implemented.

 

Here is the current sequence of awards for posting a reply to a discussion topic:

 

    

 

Here is the new sequence that will be implemented:

 

                   

 

(Double-click on any image above to see the full-size version)

 

The current gold-level milestone for posting a comment/reply is 10,000 posts, while the new milestone will be 5,000 posts. We looked at the number of members that had reached that 5,000 milestone (the current silver milestone) and determined it was more fair to make that the gold-level milestone. So you can imagine how the other 18 levels (green is always awarded for the first instance of a behavior) look (especially if you look at the posts above this one that show the actual numbers ).

 

The green, blue, bronze, silver, and gold background colors have remained the same. We've added purple, red, and copper between blue and bronze. The purple, red, and copper colors are based on color swatches from the Citadel range of paints. In addition, all of the colors except green and gold now have three levels. You can see a small black circle with an icon or Roman numerals at the bottom center. All green badges have the thumbs-up icon, intended to convey encouragement to members to continue performing that behavior to continue receiving awards. The gold badge, meanwhile, has a crux icon, which resembles a +. This is intended to convey that the gold badge recognizes the highest milestone of that performance and all higher numbers. The other badges have either I, II, or III to indicate the level.

 

As far as the main iconography goes, you'll see two elements. The upper portion is the main icon for the feature. All achievements for discussion topics feature the large speech bubble coming from the left (the old version didn't include this). In the achievement for starting a topic (not shown), the large speech bubble is black and white. Similarly, the main badge for each behavior (starting a topic, creating a blog, creating a blog entry, creating an album, submitting an image, submitting an article, submitting a file, and creating a calendar event), the icon will be black and white; and other achievements related to that (e.g., posting a review of a file, posting a comment/reply to an image, etc.) will feature that icon greyed out. Below the main icon, many badges feature an augmenting icon that shows the specific type of behavior that is being rewarded. The series of achievements above is for a comment/reply to a discussion topic, with the smaller speech bubble (coming from the right). All other comment/reply achievements (e.g., comment/reply to an article, comment/reply to a blog entry, etc.) feature the same comment/reply speech bubble in black and white, with the appropriate grayed-out upper icon for the feature. The other common augmenting icon is for reviews, which feature a series of five stars (where the smaller speech bubble appears above). Some of the achievements don't have augmentations so the icons for those achievements tend to fill the whole square (with room for the discs at the bottom).

 

Two notable changes concern comment/reply achievements and review achievements.

 

Currently, all types of comment/reply behaviors are counted in the same achievement (the one shown above). In the revised system, only comments/replies in discussions will be counted in that achievement progression. That includes the discussions in the Forums feature as well as discussion forums in the Clubs feature. Meanwhile, comments/replies in other features will be counted towards separate achievements. So if you post comments/replies to blog entries, those will count towards a distinct achievement progression. Meanwhile, comments/replies to files, articles, images, and events will also be counted towards other distinct achievement progressions. Discussion is the key form of engagement with other members of the community, but discussion isn't limited to the Forums; so breaking comment/reply achievement progressions out into separate awards will (hopefully) encourage members to engage in those other areas of the site.

 

Review achievements, meanwhile, are woefully underutilized. Also, they all currently use the same images. Just as we want members to engage in discussions in other areas of the site, we also want to encourage members to provide reviews of content in those other areas. Reviews help other members to know what content might be helpful, and they also help the members who submitted the content to know where they can improve. The system uses 5 stars for reviews, along with text. Submitting a review is almost exactly like composing a comment/reply or topic, so it will be very easy for members to learn. The key limiting element is whether or not members are willing to take the time to compose and submit reviews, so we're hoping that rewarding this behavior might incentivize you. Reviews must be approved, however, and we'll probably provide some level of guidance so that people don't just spam reviews with little or no feedback (my kneejerk instinct is to require 100 words or more, but the staff hasn't even begun discussing this issue yet, so stand by). There will definitely be a requirement for the feedback to be constructive (which doesn't preclude it being negative), and non-constructive reviews will not be approved.

 

A few of the badge icons were adjusted for (hopefully) more clarity. For example, the old (current) icon for voting in a poll is a pillar. That will be replaced with an icon of a hand placing a ballot in a ballot box. Similarly, the old (current) icon for starting a club features five people, but that can easily be confused with the icon for gaining followers, which features three people. So the new icon for starting a club is a clubhouse while the new icon for gaining followers is four people. Likewise, the icon for a file (Downloads) has been changed from a piece of paper with a folded corner (white for uploads, black for downloads) to a file folder.

 

Overall, we hope that the updated images will provide both clarity and consistency, improving visual recognition of the behavior that each rewards.

 

I'll post an update here once I start the actual update work so that you (hopefully) won't freak out if things look too wonky.

 

 

 

* Disciplinary action occasionally involves loss of achievement points. In revising the system, we want to ensure that anyone affected by such disciplinary measures isn't treated unfairly, so we'll have to make sure that their achievement points are adjusted appropriately. This is the key task remaining before we can begin the update. Luckily, there are only a handful of members to whom this applies, so it shouldn't take long to figure out how their recounted points will have to be modified to get them back at the right level.

[Brother Tyler]

True to form, my wife boss has informed me of her our plans, and that means that the work will not begin this Thursday as I had hoped. Instead, the work will begin next Monday.

 

Carry on.

[Brother Tyler]

The work starts now.

 

Here's the basic sequence I have planned (plans always subject to change upon contact with the enemy...):

1. Delete the old rules
2. Delete the old images
3. Upload the new images
4. Create the new rules
5. Update the ranks
6. Force the system to rebuild everyone's achievements
7. If numbers of members in ranks are off, adjust ranks and repeat step 6 (this step may happen multiple times)
8. Update the Achievements & Ranks topic (the current version will be replaced by a new one that is in development)
9. Tell y'all when everything is done
10. Tie up loose ends (manually awarded badges might need to be redone)

Everything will look wonky until step #5 is completed.

 

Note that this will only affect the automatic rules, except for the Longevity rules (which aren't changing). All manually awarded achievements such as those for supporting the site, participating in events, having your game content published in White Dwarf Magazine or Warhammer Visions, etc. will remain unchanged.

 

[Brother Tyler]

The rebuilding of the achievements is in progress. While that is taking place, things might slow down a bit. Once it is complete, everyone's new achievements will show up. If the numbers are off, I'll tweak things and rebuild again. I'll let you know when that is complete.

[Brother Tyler]

The rebuilding has been in progress for over 20 hours and it is 6.43% of the way through the process. I anticipate having to make some adjustments and going through the rebuild process [at least, but hopefully only] one more time.

[Brother Tyler]

32 hours into the rebuilding and the system has made 10.47% progress.

[Brother Tyler]

42 hours in and we're 15.46% along.

[Bouargh]

If reconstruction rate is linear, 10 more days to go . If it is exponantial, we will be in the frame in a couple of days. I will vote for the optimistic schedule.

courage! It could be worse…

 

Edited 12 hours ago by Bouargh

[Brother Tyler]

48 hours on and we're 17.67% of the way through.

 

This is us...