Snagging failed splits

Published: 2020-10-25

Author: Teddi

Back in May of this year (so recently?! feels like forever) we released the first version of the Timetrial Split system, this was a heavily sought after feature from many within BB to give data and granularity to their runs. A further update was then deployed so not only could a player identify whether or not a split was better or worse than before; they could also see if it was better or worse relative to their current Personal Best time. Five months on the system has been for the most part a roaring success!

There was one thing that was apparent and that was a problem existed with the system and that somehow players were managing to miss the checkpoints that dictate when a split starts and when it ends. In this case the both how and why are important but what’s even more important is how can we expose this to us as developers? Fundamentally we want players to have a good time and there’s nothing more frustrating to get a new Personal Best (especially if you’ve been grinding it) only for the system to turn around and go “we got your time, but we didn’t get your splits!“.

Furthermore we weren’t getting the data of where a split was going wrong, players were crying out it was happening and all we could do at the time was revisit each and every split in a given map which is a huge waste of time and doesn’t really tell us anything other than “maybe lets rejig this bit here, that bit there”.

Before we dive in to how we’ve mostly resolved this issue, lets take a look at some of the most common reasons a player may not hit a split zone.

Unexpected Player Movement

In an attempt to make splits accurate as possible for a given stage (or linears) a fair bit of effort went in to making them precise and in most cases this worked out okay. However what had been bit of a downfall to this system is there were expectations about how the average player launched themselves on to a ramp - we tend to expect most people go straight on and down (with maybe some air-strafing) but some players instead prefer to go from the side getting an air-strafe in early.

How they were missing: They were taking non-obvious / non-intended paths.
Why they were missing: To be as fast as possible.

Fix: Lets loosen up the requirements and give a wider coverage / additional split capture zones. This might incorrectly give a split of +/- 1 or 2 seconds in some maps that have non-standard geometry, but better to give a slightly different split time rather than no data at all. Plus it doesn’t affect timetrials whatsoever. Until we can get mesh checkpoints out it’ll have to be an acceptable margin of error.

Unexpected Map Behaviour

As I’ve complained lightly about in the previous blog post Debugging Map Quirks, maps aren’t always the cleanest in terms of their feature set / functionality and this can be for whatever number of reasons. A big example of this is surf_fortum, where the exit teleport takes you from stage 2 to 3. Something there can go a bit funky when “teleporting you” and even though for all intents and purposes we usually capture players when they land on the end platform, sometimes Source goes “nah” and you evade the split end checkpoint.

The resolution in this case is to add a second split zone end on the start of stage 3. Best case: you hit the correct endpoint as normal and get an accurate split. Worst case: you don’t and you get a split delayed by a second or so.

How they’re missing: Maps have inconsistent behaviour and it means players may not end up touching what they need to.
Why they’re missing: Maps have inconsistent behaviour and it means players may not end up touching what they need to.

Fix: Generally try to work out smart ways to fix it that isn’t to the detriment of players, e.g. multiple split zones for a given checkpoint may be required.

Player Speeds

This one is tricky to explain but it turns out some checkpoints were too “thin” so that if players moved through them fast enough; they would never actually trigger the zone. The way the server sees movement is lots of “teleports” as opposed to linear movement that you or I experience when playing. The server does some fancy maths to interpolate how you’re moving so the faster you’re going, the larger these “teleports” become in terms of distance apart.

How they’re missing: players going zoom zoom because gotta go fast.
Why they’re missing: Gotta go fast for top times.

Fix: Make checkpoints larger. There’s a risk they “hit” the box later, but their overall time will be lower so it doesn’t matter. It’s a quirk we can’t really fix unless we up the tickrate of the server (which is a contentious issue for many reasons).

Resolving the actual reporting of missed splits

So when we first started getting reports of missed splits the output to players wasn’t the most informative other than “we failed to record (your splits != total splits). In an effort to make this better and so players could report the issue better the output was updated to show a list of all splits that had been recorded so when reported to someone at BB, we at least knew which split had failed.

This was far better than it was before as we could immediately identify which split was going wrong and this meant faster turnaround on reports! This then revealed a second issue - we were reliant on players to actually report these issues. What started off (and still continued for a while) was a dedicated few reporting the split failures when they cropped up with the vast majority of players either mentioning there had been a failure (without keeping a record of the splits) or failing to mention outright entirely.

At some point there was a recognition that we can’t expect players to report these issues reliable or consistently. I mean we already knew this given we have Lua errors automatically report themselves but a key problem was this automated Lua reporting system wasn’t fit for split errors. We need to fix these split issues as soon as possible as opposed to sitting in a backlog and we need a way to notify ourselves exactly when they happen.

What resulted is a system that when an error is detected it automatically gets reported to our GitHub “issues” bot who then posts it on behalf of the server.

Oh hey the only picture in this stupid long blogpost

The nice thing about this system is everything was wrote from the ground up and allows us to insert any issue we want in to GitHub allowing us to catch any future issues that we need immediate turnaround on, not just split issues! Even more important however it’s meant that typically when we get a split issue now, the average turnaround time from initial reporting has gone from maybe being around ~30 hours to roughly 4. To clarify - that doesn’t mean it takes 4 hours to resolve and instead the time of which it’s picked up and closed by.

Split failures still happen but we’re starting to see their frequency already less and less. They weren’t that common previously but it did seem to be specific maps that had the issue. This tells me that in most cases maps were set up correctly and it’s mostly maps that were set up in the early stages that generally have this issue.

So overall we’ve gotten three core benefits out of this. One is we now have a way to immediately raise and report issues from within the gamemode itself which secondly leads to better triage and bug fixes. The third is players get a better experience out of this overall. A win-win-win!