The Most Absolutely Absurd RoN Bug I’ve Ever Found

For most of this year, I’ve been working on getting the next version, Alpha 7, of the Community Balance/Bugfix Patch out. One bug in particular relating to Barks and Triremes was especially difficult to pin down, but last night I was finally able to figure out the bug’s trigger and get a 100% reproduction rate.

Update:

It seems that more than just data files are able to trigger it (so long as the file is loaded – dud files don’t trigger it). That makes the bug significantly more effort to work around, as ALL loaded custom files (e.g. art files) will have to be changed from using the local mod format to using the direct mod format. The rest of the information still appears to be correct. Original article follows.

 

Getting it wrong

I first became aware of this particular bug much earlier this year, and in fact I shipped what I thought was a fix for it to Misc Fixes in early March.

The bug seemed to be that Barks and Triremes are treated as Modern Age (VII) units for the purpose of damage modifiers, making them deal way too much damage against low-age units, and take far too little damage from high-age units. This could be treated by pulling some numbers, doing some maths, and then applying counter-modifiers in balance.xml, basically offsetting the effect of the bizarre “hardcoded age” that the units seemed to have by respectively lowering and raising damage dealt and taken against other units based on that unit’s age.

At the time, I had a reliable 100% reproduction rate of the bug, and 100% success rate with the fix. It did seem a little odd that nobody had noticed this before now, but considering that:

• It took about 11 years for someone to notice that Dutch unique ships have bugs which make their DPS generally awful.
• It took about 6 years for anyone to notice — at scale — the object masks bug (even top level players who played both bugged and unbugged versions of the game barely noticed every unit in the game dealing/taking the wrong amount of damage!).

..it didn’t seem that unreasonable that nobody had noticed this either.

And so after creating and testing the fix, I stuck it in Misc Fixes, and made a mental note to add the fix to CBP at some point as well.¹ I then went on with working on CBP Alpha 7 and other things in life such as not dying.

Getting it wrong, again

Fast forward a few months and I’m getting close to having Alpha  7 ready for pre-release testing.

One of the biggest things that Alpha 7 will be including is a now-seems-ready-for-public-use version of CBP Launcher, something I’d always wanted to have with the mod but had generally struggled with justifying the required development time to get done (which is why development stalled for a number of months between v0.2 and v0.3).

Anyway, I’d buckled down and gotten it ready for some private testing. Then, despite my hesitance about having people I didn’t know testing the pre-release version of CBP (since bugs in the pre-release could damage the main mod’s reputation) HnZ roped in a few people from his friends list and public lobbies into playing some games with the pre-release.

As a result of these live tests (and some extra followup testing on my part after noticing the problem(s)), we found the following:

• The Bark/Trireme bugfix seemed to work normally for the first game played in a session.
• The bugfix appeared to be applying twice on all subsequent games, i.e. the Bark and Trireme would be far too weak – the opposite of the original bug in the first place.²
• The discrepancy would cause games to go out of sync when they had a mix of players on their first game in a session vs not.
• The damage for just those two units was affected, as confirmed by extracting the game’s internal damage/balance table a few times.

With this new data on hand, my first theory was that the slightly-lazy approach I’d used in the bugfix was itself bugged.

Changing RoN’s damage modifiers is an entire topic in and of itself, but in short, you can:

1. apply damage modifiers on a per-unit basis (on both ends of the dealt vs receiving equation), or
2. apply damage modifiers on a per-group basis (e.g. all Heavy Infantry deal extra damage to all Classical Age (II) units), or
3. do half of each of the above, with an individual unit on one end of the equation and a group on the other (so e.g. an individual unit could be made to do more damage vs all units in a specific unit group, or the reverse).

Applying damage modifiers using groups where possible (via properties such as object masks, those things that are bugged to hell and back specifically in EE) is the most convenient because it means you don’t need to individually tweak between hundreds and tens of thousands of modifiers.

The fix I had was using predominantly group-based modifiers (based on unit age), modifiers which appeared functional in my limited testing,³ but are actually never used in this manner by the game itself. So I figured okay, probably those not-normally-used group modifiers are bugged somehow, so it’s time to switch to using individual modifiers.

The thing about using individual modifiers is that while automated programs can change these easily enough, you have to actually write the program which does that in the first place. For a few hours I looked at both how hard it would be to learn Rust and modify mjn33’s existing program for this new purpose⁴ and how hard it would be to write something of my own in python or C# (the only two languages I’m vaguely familiar with, so that less extra learning would be required). After about an hour on the first option and a few more combined on the latter two, I evaluated that the fastest method would be to redo all the relevant modifiers manually.⁵

And so I did. I spent the majority of a day going through and manually changing every modifier for and against Barks and Triremes to the appropriate (fixed) value. I even streamed the soul-damaging experience on the RoN Discord server, and amusingly got a viewer for an hour or two of the experience who commented on things as I went. They were particularly impressed that I could drop a huge stream of hexadecimal data into Excel⁶ and quickly convert it into adequately formatted human-readable decimal.

Changing a thousand or two modifiers by hand, calculated individually for each unit based on its age (and accounting for any existing modifiers) was error-prone, as expected. Having to manually cross-reference an external file (which has a different internal naming scheme for many units, sometimes duplicating them multiple times (!)) to get each unit’s age wasn’t helping, and at least three times after going through the full list, I extracted the damage modifiers from the game to check for errors and then had to go back and fix things up. After carefully balancing “pay enough attention to make as few mistakes as possible” and “pay less attention to reduce the urge to end your life rather than complete this” for a few hours it was done and I would never have to deal with the bug again, thank god.

And then I went to test and verify the new fixes in-game.

They didn’t work.

The same problems as before were occurring. Barks and Triremes would work as expected for one game per session, but were consistently too weak for games after that in each session.

Getting confused

After some amount of despondence and faffing around, my next theory was that the game was loading balance.xml once in the first game, and then a second time on top of that in subsequent games, regardless of whether using individual or group-based damage modifiers..  and only for the Bark and Trireme (very weird, but not completely insane given the mess that RoN:EE is). I thought it was being caused by the local-mod format, as obviously the default balance.xml (in the game’s standard /Data/ folder) doesn’t seem to double-dip damage modifiers.

And so, a few weeks ago, and after over 2 months of slogging away on the launcher, I added yet another thing to the to-do list: convert CBP to the direct-mod format, and rewrite core parts of CBP Launcher to handle the change seamlessly for the end-user. Yesterday I finally completed that task and quickly tested the damage of Barks and Triremes so that I could bask in the satisfaction of being the meanest motherfucker that ever existed by fixing this stupid fucking bug.

The bugfix didn’t work.

Barks and Triremes were still regularly too weak, just like when using both versions of the fix in the local mod format. For a few moments I got a fun influx of creeping dread, realising there was a chance I’d just subjected myself to a huge amount of not particularly fun work for nothing. I had no more theories left about what was causing the problem.

Getting very confused

So, in short: what the everliving fuck was going on? Did I personally offend someone important in a past life? Was it that time I made a joke about Brian Reynolds? Was CBP development fucked, yet again?

The path forward, as per usual in the RoN modding journey, was to do the work that nobody else had done or was generally willing to do. In this case that meant more braindead testing: controlling multiple different factors, run the same mind-numbingly mundane test(s) in-game repeatedly while changing external factors to record which of them correlate with what result (whether Barks and Triremes are bugged, etc). It’s like using machine learning to solve a problem, except you have to turn yourself into the machine.

For the first slew of tests, I tested multiplayer games against myself in 1v1s (controlling my laptop remotely over unstable WiFi for “opponent me”). To start off in almost the most vanilla state possible, for these tests I verified RoN’s game files on both machines and emptied the local mods folder on both installs. I then manually applied only a modified balance.xml file to each of them that included the most recent version of the Bark/Trireme bugfix. After a few games of this (and getting the same result each time) I switched over to singleplayer, which was faster and easier to test with.

In a variety of nation matchups (including a mirror matchup), in both first-session and not, Barks were too weak⁷ in all eight tests.

To get one step closer to maximum-vanilla, I then did more testing but now while using the default balance.xml file. Maybe my baseline idea of the bug itself was incorrect? Three tests showed no change, so I added the local mods back on my desktop install (where I was doing the tests) and the bug occurred. Maybe it was a nation-specific bug? The local mods in question were basically just scenarios and UI / cosmetic files so it seemed just coincidental that the bug happened at the same time. I was very sure about the contents of the mods, as I checked them during testing to make sure.

I tested a couple more times then went back and manually picked the nations which I had RNG’d when the bug tripped. The bug didn’t happen. Maybe it only happened under specific RNG rolls? I kept testing, tripping the bug a couple more times but taking more than a couple of tests to do it.

At this point the three times (out of more than 20 tests) that the bug had appeared was while local mods were loaded instead of removed, so I began to hone in on that. I had an inkling from a combination of previous RoN reverse engineering adventures and RoN modding experience in general that the game seemed to load files differently depending on whether not they were being loaded from a mod or not, so this seemed kind of promising.

I kept testing, starting from a baseline of all local mods removed and steadily introducing an additional local mod into the mix every few tests. I didn’t have much luck until I got to a UI mod where I was prototyping some features for CBP (basically CBP status readouts on various menu screens). On first-game-of-session and with that mod loaded, I could reliably trip the bug with the vanilla balance.xml file loaded.

One last thing to test.

I loaded up the entirety of CBP Alpha 7 PR5, the version where I had converted all the mod’s data files to the direct-mod format  and tested that with the UI mod unloaded. No bug.

Then I added the UI mod back into the mix.

The bug tripped.

The worst RoN bug in the history of RoN bugs

So the bug, in my currently-entire understanding:

• Affects specifically the Bark and Trireme, and no other units, not even ones that share traits such as other Ancient Age (I) units (and this is confirmed with extracted damage/balance table).
• Affects the game’s internal damage/balance table for those two units, comparably to if they were set to be Modern Age (VII) units, consequently modifying hundreds/thousands of damage modifiers in-memory.
• Only affects the first game in a session (if triggered), but no subsequent games.
• Works in both singleplayer and multiplayer.
• Is “hard” enough (as in “hard data”) to cause out-of-sync issues in multiplayer between first-game and not-first-game players.
• Is not triggered by simply loading mods.
• Is triggered by loading external /Data/ files, even if those files never even mention Barks or Triremes and have no direct impact on game balance or any other gameplay elements.
• Has absolutely no clear relation between the trigger and the effect, like at all, in any capacity.
• Has a 100% reproduction rate in the triggering circumstances.
• Has a 100% fix rate when negating the triggering circumstances.

I have fixed many RoN bugs. This is not the worst one⁸ (that honor obviously goes to RoN:EE’s bugged object masks), but it is — by far — the most completely, absolutely absurd one that I’ve encountered. It has an extremely specific trigger condition, doesn’t affect every game (and yet does affect games in a consistent manner), and can change something that’s completely unrelated to the triggering file(s) in a way that does meaningfully affect game balance.. but not often enough to be a problem every game either because it’s naval combat.

I would complain that I’m not getting paid enough for this, but in fact I’m not really getting paid for this at all. So I guess all I can do is simply complain. As mentioned in one of the footnotes, this is the only RoN bug that has ever given me a physical headache, an experience I hope is not repeated. If there was a way to do so without actually, you know, doing it, I would really like to slap whoever’s responsible for introducing this bug.⁹

In other news, CBP Alpha 7 is currently going pretty swimmingly. Expect it to be publicly released in the coming weeks / month-ish as final development and testing is done on the launcher and I do final QA on the mod itself.

 

---

 

1. (the implementation has to be slightly different in each mod, so it wasn’t something I could copy-paste)
2. This is the only RoN bug that has ever given me a physical headache.
3. Unlike object masks in EE
4. If I didn’t think I could feasibly do it myself via any available method I would’ve asked him about tweaking it for me, but I didn’t want to bother him otherwise.
5. The pitfalls of having limited coding competence. Nice dodge of the sunk cost fallacy there though ::fingerguns::
6. Yes, this is the same link as before.
7. (i.e. the “bugfix applied twice” category)
8. So the header isn’t literally true I guess OMG CLICKBAIT!!1!
9. I do also wonder if it’s present in T&P or if it’s EE-exclusive as well, but loading up T&P on my system is a chore so I didn’t bother checking this out.