January 17th, 2012
Hoovaloov

(Click for UPDATE #1 | UPDATE #2 | UPDATE #3)

 

ABSTRACT:

Since the Title Update for Halo: Reach’s multiplayer went live in the fall, there have been many discussions on the changes it implemented. The Title Update most notably impacted the mechanics of the DMR and the Needle Rifle, specifically their “bloom.” The question is, does 85% bloom encourage or discourage “trigger spamming” compared to default Reach settings?

 

INTRODUCTION:

Anyone who has played Halo: Reach has experienced weapon bloom. Bloom describes the change in a weapon’s ability to fire accurately as the player adjusts the rate of fire. Firing many shots in quick succession will cause the aiming reticule to expand, indicating more randomness in the accuracy of each shot. To guarantee maximum accuracy, a player should wait until the reticule shrinks back to normal size before firing the next shot. This mechanic was designed by Bungie as a trade-off of pace and accuracy in an effort to balance the DMR in the overall sandbox.

If a player fires the DMR as fast as possible (known as spamming the trigger), he is making a decision to trade accuracy for more shots fired. This is more likely to work at close range where even after many shots at maximum pace (where the reticule has reached maximum bloom), the spammer may out-duel a player who chose to pace his shots to preserve accuracy. The tactic of spamming is somewhat frowned upon by some members of the matchmaking community as they feel spammed kills are cheap, due mostly to a lucky random bullet instead of skilled accuracy and a proper firing cadence. Others say spamming is part of the game, and therefore a valid tactic; a kill is a kill, in other words.

The Title Update that 343 Industries implemented into multiplayer matchmaking changed this mechanic for certain gametypes and playlists. If we designate maximum bloom as 100% bloom, the TU reduces that maximum to 85% of the original. Whether this means the “cone of accuracy” is 15% smaller, the aiming reticule resets 15% faster, or the bullet is 15% more likely to hit the target, I can’t say. But we have been told by 343 that the TU has reduced bloom by 15%.

Those who dislike trigger spamming have argued that 85% bloom encourages spamming by making spammed shots more accurate, and therefore more likely to result in a win against a pacer. Having played many TU games, I felt that 85% bloom actually punishes spamming more than 100% bloom because the player would not have to wait as long for the reticule to reset, meaning he could fire accurate shots more quickly than in 100%.

Without a proper experiment, this debate could go nowhere.

I devised an experiment to answer the question: In 85% bloom, does increasing the accuracy of a spammed shot outweigh the reduction in waiting time between properly paced shots? In other words, will 85% bloom result in more or less spammed kills than in 100% bloom?

 

METHODS:

In order to test this, I created a Forge World map with different sized firing ranges: short, medium, and long range (5, 10, and 15 Forge World units). On one side of the range I placed a Red Team spawn point, and on the other, one for Blue Team. Below is a screenshot of the firing ranges (click the picture for a bigger version).

Aiming at the head while unzoomed at all times, I shot and killed the stationary target 25 times by pacing all of my shots (letting the reticule fully reset before firing the next shot), and 25 times by spamming the trigger as fast as I could. I repeated this procedure for all three ranges with 100% bloom settings, and then again on all three ranges with 85% bloom settings.

Next, I captured video files of each of these tests onto my computer. Using video editing software, I counted how many frames elapsed during each kill, starting with the frame where the muzzle flash of the DMR’s first shot was visible and ending with the frame where the Team Slayer score updated on the HUD.

Using a frame rate of 29.97 frames/second, I converted these durations into seconds. I then entered all of these results into a spreadsheet to examine the effects of 85% bloom.

To better understand the results, I added some equations that make things a little clearer.

First, within each range, I compared paced and spammed kill times one at a time and chose the lowest value of each pair. Then I calculated the percentage of those 25 kills that was won by pacing. This I called “Percentage of Pace Wins.” This is useful in seeing if 85% bloom increases or decreases the amount of pacing wins.

Second, I averaged the time duration of the 25 kills in each test group. This is useful to determine whether pacing or spamming is faster on average for each range in 100% and 85% settings.

Third, within each range, I compared the average time of paced kills to the average time of spammed kills. This I called the “Margin of Win.” This is useful for determining the amount of time (and therefore shields and shots) to spare for the winner’s kill. A small Margin of Win means that both methods (i.e. pacing or spamming) were very close in the time it took to kill the opponent, so the winner of each battle will be more unpredictable. A large Margin of Win means that one method will win the majority of the battles.

 

RESULTS:

Here is a link to the spreadsheet containing my findings: DMR Test Results

 

CONCLUSIONS:

I included conclusions in the spreadsheet to minimize jumping back and forth between it and this article. I will summarize my findings here as well.

1) At short range, spamming is on average more likely to win in either bloom setting. However, 85% bloom increased the Percentage of Pace Wins from 28% to 40%. Also, 85% bloom reduced the Margin of Win of spamming from 0.178 seconds to just 0.036 seconds, further emphasizing the decreased advantage spamming has at close range in 85% bloom. Basically, at short range, 85% bloom increases the likelihood of pacing wins compared to 100% bloom.

2) At medium range, pacing is on average more likely to win in either bloom setting. But 85% bloom increased the Percentage of Pace Wins from 64% to 76%. Also, 85% bloom slightly increased the Margin of Win of pacing, further emphasizing the increased advantage of pacing at medium range in 85% bloom. Basically, at medium range, 85% bloom increases the likelihood of pacing wins compared to 100% bloom.

3) At long range, pacing is on average extremely likely to win in either bloom setting. Interestingly, the Margin of Win of pacing actually decreased slightly in 85% bloom. This can be attributed to the higher likelihood of body shots landing at long range. But this change is insignificant given that both 100% and 85% bloom have Margin of Win times of at least 1.902 seconds. That is such a huge margin of time that the number of paced wins was actually unaffected. In both 100% and 85% bloom, the Percentage of Paced Wins remained the same at 88%. Basically, at long range, 85% bloom keeps the same very high likelihood of pacing wins as 100% bloom.

4) By studying the number of shots fired in each kill, I noticed that in 100% bloom, a spammer would always win the battle unless he needed to fire 7 or more shots. This means the spammer has two chances at max bloom (5th and 6th shot) to kill the pacer, while the pacer can’t miss any shots. But in 85% bloom, if the spammer needed a 6th shot, he would always lose to the pacer. This means both the spammer and pacer have no margin for error, which gives further advantage to the pacer since his shots are more accurate.

5) I also compared the probability of a spammed headshot kill in each of these groups. At short range, 85% bloom increased headshot probability from 92% to 100%. At medium range, 85% bloom did not change headshot probability and remained at 92%. At long range, 85% bloom actually decreased headshot probability from 84% to 76%. However, this does not mean 85% bloom makes spamming the DMR better at short range. It is important to remember that based on the above conclusions, 85% bloom results in less spamming wins than 100% does at short range. I included this headshot result to show that even though at short range spammed headshot probability is increased, spamming wins are still reduced in 85% bloom at short range.  Basically, whether a spammed kill ends in a headshot is meaningless if paced shots will win.  And 85% bloom increases the likelihood of paced wins compared to 100% bloom.

Therefore, because 85% bloom increases the advantage of pacing at short and medium ranges, equals the advantage at long range, and encourages pacing by making a missed spam shot much riskier, those who prefer to have DMR pacing rewarded and spamming punished should choose 85% bloom.

 

FINAL THOUGHTS:

I have shown that according to my test results, 85% bloom actually rewards players who pace their shots more than 100% bloom does. I should note that this result does not conclude anything about weapon balance. That is an entirely different matter that falls outside the scope of this experiment. I conducted this experiment to clarify the effects of 85% bloom and help settle the debate about whether 85% bloom encourages or discourages spamming.

I am more than happy to answer any questions or comments about this experiment, please post them here. Thank you for reading and I hope you’ve found this informative!

 

 

UPDATE #1:

Thanks to everyone for the great feedback to this study.  I’m glad you have found it helpful!

Some quick notes that might not have been clear before: all my tests were performed with the DMR unzoomed for all shots.  Also, when I said that I “paced each shot” I meant that I waited for the reticule to fully reset before firing the next shot.  This allowed me to completely separate any effects of spam from the paced results.  I tried my best to anticipate each full reduction in order to minimize “down time” between shots.  My goal was to keep everything as consistent as possible across each test, and if you examine the pacing kill times in each group, you will find they are very consistent.

 

In my discussions with HBO forum member RC Master, I have seen that perhaps just including mean averages in my study may have skewed my results.  RC Master edited a copy of my spreadsheet to include more advanced statistics in an effort to help build a better picture of the raw data.  That is exactly why I made the data available publicly, and I’m glad he was willing to undertake this further analysis.

His additions can be found in a spreadsheet here: RC Master Additions

(Note that he actually added a second page to the document, look in the bottom left corner for the Frequency tab for more data and graphs)

 

Most notably, he has highlighted certain pieces of the data: the Minimum and Maximum of each range, the Median, and the 1st and 3rd Quartiles.  Perhaps you’re not sure how these statistical devices help us, so I will quickly explain them.

Minimum and Maximum – Fairly obvious, they are the smallest and largest values in each group of data

Median – Unlike regular Average, which can be influenced by large or small extremes, Median picks the one data point in the exact center of the “continuum” of data.  So after each range of data is sorted in order, Median chooses the middle of the line.

1st Quartile – If Median is the halfway point between Minimum and Maximum, the 1st Quartile is the 1/4th mark.  This is useful for excluding extremely small outliers in the data.

3rd Quartile – If Median is the halfway point between Minimum and Maximum, the 3rd Quartile is the 3/4th mark.  This is useful for excluding extremely large outliers in the data.

 

So hopefully now you can begin to see why RC Master brought these things to our attention.  Mean Average is too easily influenced by outliers in the data, and these insights help us look past the outliers to see what is really going on.

But now my question is, how will these new statistical insights affect the conclusions I have drawn?  Will my results be overturned?

 

To help answer this question, I decided that the new data (Min, 1st Q, Median, 3rd Q, Max) should be graphed to help visualize the trends in the data. Here is my graph of RC Master’s new data points (click the picture for a much bigger version):

The vertical axis is the time it took for the kill, measured in video frames.  The lower on the chart, the quicker the kill.  Higher on the chart means the kill takes longer.

I colored the paced shots green and the spammed shots red.

The vertical skinny lines in each plot are defined by the Minimum and Maximum data point of each set.

The box on the line is defined by Q1 and Q3 of each set (which is the middle 50% of the data).  This box discards most outliers and is less influenced by the extreme results.

And the little horizontal line across each box is the Median.

 

So let’s compare where these boxes are positioned relative to each other and see if we can draw some conclusions.

The first thing to notice is that none of the green plots appear to have boxes on their vertical lines, they look more like horizontal lines.  This is because pacing the DMR’s shots leads to extremely consistent results, so the data is grouped extremely tightly, squishing the box flat.

Second, spamming the DMR results in a wide range of outcomes.  These outcomes grow more and more varied from short to medium to long range.  This increase in variation can be easily seen in the increase of the vertical red lines’ height.  There’s so much variation at long range that even the red boxes at long range have stretched out.

 

Now, let’s take a closer look each range of the test.

 

>> Short Range <<

The graph makes it clear that spamming will win a majority of the time at short range in either bloom setting because most of the red box is below the horizontal green line.

In 100% bloom, the horizontal green line (pace) is in between the thin horizontal red line (spam median) and the top edge of the red box (spam Q3).

In 85% bloom, the horizontal green line is still in between the thin horizontal red line (spam median) and the top edge of the red box (spam Q3).  However, the horizontal green line is closer to the bottom of the vertical red line (minimum kill time of spamming).

Therefore, because 85% bloom puts pacing nearer the minimum kill time of spamming, it reduces the amount of times a spammed kill will win.  This leads to an increase pacing win percentage compared to 100% bloom.

 

>> Medium Range <<

The graph illustrates that pacing will win a majority of the time at medium range in either bloom setting due to how low the green horizontal line is compared to the red box.

In 100% bloom, the horizontal green line is near the lower third of the red box.

In 85% bloom, the horizontal green line is actually below the red box.

Therefore, because 85% bloom puts the entire green line below the red box, it definitely increases pacing win percentage compared to 100% bloom.

 

>> Long Range <<

The graph clearly illustrates that pacing will win an overwhelming majority of the time at long range at either bloom setting due to how much lower the green horizontal line is compared to the red box.

In 100% bloom, the horizontal green line is well below the red box.

In 85% bloom, the horizontal green line is still below the red box, but not by as much.  However, the horizontal green line is closer to the bottom of the vertical red line (minimum kill time of spamming).

Therefore, even though 85% bloom resulted in the green line being closer to the bottom edge of the red box, it also put the green line nearer the minimum kill time of spamming, reducing the amount of times a spammed kill will win.  This mix of results points toward 85% pretty much equaling the extreme majority of pacing wins as 100% bloom.

 

>> Conclusion <<

Using these new statistics provided by RC Master, I have arrived at the same conclusions I had in the first place.  At short range, spamming will win a majority of the time, but 85% bloom reduces that probability compared to 100% bloom.  At medium mange, 85% bloom definitely increases the probability of pacing wins compared to 100% bloom.  At long range, 85% bloom pretty much equals the results of 100% bloom.

Also, important to note is that at all three ranges, 85% bloom moved the green horizontal box closer to the bottom of the skinny red vertical line, especially apparent at medium and long range.  In other words, 85% bloom allowed less and less spammed wins to occur at these ranges, illustrated by how little is left of the vertical red line that extends below the horizontal green box.  This is direct proof that in 85% bloom, the reduction in waiting time for the reticule to return to size significantly outweighs the slight increase in accuracy experienced by spammers, especially at medium and long range.

Once again, I’d like to thank RC Master for taking the time to delve into some advanced statistics for us.  Thanks again for reading! If you have comments or questions, please post them here.

 

 

UPDATE #2:

Before I get to the update, I’d like to thank Bs Angel at 343 Industries for posting this article on Halo Waypoint.  Also, the feedback to the study has been great, I really appreciate everyone’s comments!

 

HBO forum member uberfoop notified me of a weakness in my analysis.  So in the interest of due diligence, I will explain what I did wrong, and then illustrate the method he described to fix it.

In my initial discussion of the data, I showed how I killed the target 25 times by pacing all my shots and then spamming all my shots at different lengths in both 100% and 85% bloom settings.  Then I explained the results by comparing a paced kill time and a spammed kill time to each other, determining a winner, and continuing on to the next pair of data.

But as uberfoop pointed out to me, this is not a good way to compare the data.  Just picking a time from the top of one list and comparing it to a time at the top of the other list is not the same as actually having both players fire at each other and seeing who actually won.

This is easy to illustrate.  Let’s say I have a group of three times for paced shots: 1, 3, and 5 seconds.  And I have another three times for spammed shots: 2, 4, and 6 seconds.  The way I compared them at first would show that pacing would win 100% of the time (left side of picture below).  But if we rearrange them, we get an entirely different result (right side of picture).

(Click the picture for a bigger version)

The best way would be to actually have two players fire at each at different pacing rates, obviously.  But since I didn’t have my experiment set up that way, uberfoop suggested that I completely jumble the numbers in each table by sorting them each randomly.  Then compare the values and determine the winner for each random pair and save the results as one sample.  Then repeat that process hundreds of times.  This would provide enough data to determine whether 85% bloom increases or decreases pacing wins over spamming wins compared to 100% bloom.

So I made another spreadsheet that jumbled and compared the data 1,000 times to ensure a large sample size.

This spreadsheet can be found here: Win Percentage of Paced Shots

Then I examined the data using all the statistical techniques I used before and graphed them to see the new trends.

(Click the picture for a bigger version)

The vertical axis is the probability of paced shots winning over spammed shots.  Low on the chart means spamming wins a majority of the time, high on the chart means pacing wins a majority of the time.

I colored 100% bloom results blue and 85% bloom results orange.

 

CONCLUSIONS:

We can draw some conclusions right away from this chart.

First, at short range, spamming will win a majority of the time in either setting.  But the orange box is much higher than the blue box, meaning that 85% significantly increases the winning percentage for pacing.

Second, at medium range, it is clear that pacing will now win a majority of the time in either setting.  But again, the orange box is higher than the blue box, meaning 85% once again improves the odds of pacing beating spamming.

Third, at long range, the orange box is slightly lower than the blue box, meaning that pacing win percentage is slightly lower in 85% than 100%.  Does this mean 85% encourages spamming at long range?  Definitely not.  Look how high up the chart both the blue and orange boxes are at long range!  Spamming is extremely unlikely to win in either bloom setting at long range, so the slight reduction in 85% will not be that noticeable in real world outcomes.

 

HARDCORE STATS:

There are those of you out there that have probably read through this and thought to yourself, “This guy sure is getting corrected a lot on his statistics.  I bet he doesn’t really know what he’s doing.”  Well, I don’t compute and analyze statistics for a living, if that’s what you mean.

So I brought in someone who does.

My friend Joel is an actuarial scientist at a insurance company here in St. Louis.  We went to college together, and he definitely knows his stats.  I showed him my study, and he crunched some serious numbers.

Here’s what he came up with:

 

Just for fun: Modeling as a normal distribution.

Z going to represent the time difference between P (paced shooting) and S (spam shooting). A positive Z value means spamming wins, while a negative value means paced kills faster.

Z is approximately normal, with μZPS, and σ2Z2P2S.
P(Z < X) = (X – μZ)/σ2Z

So, givens for our six situations:

Short Range, 100%
μZ = 2.262 – 2.085 = .177
σ2Z = .00303 + .20014
σZ = .45075
=(0 -.177)/.45075 = -.3938

Plugging into normal distribution, spamming has a 65.17% chance of winning.

Short Range, 85%
μZ = 1.981-1.945 = .036
σ2Z = .00150 + .22236
σZ = .47314
=(0 -.036)/.47314 = -.0762

Plugging into normal distribution, spamming has a 53.19% chance of winning.

 

Medium Range, 100%
μZ = 2.350 – 2.628 = -.2776
σ2Z = .00321 + .40822
σZ = .64143
=(0 – (-).2776)/.64143 = .4328

Plugging into normal distribution, spamming has a 33.36% chance of winning.

Medium Range, 85%
μZ = 2.010 – 2.306 = -.2963
σ2Z = .00288 + .29903
σZ = .54946
=(0 -(-).2963)/.54946 = .5393

Plugging into normal distribution, spamming has a 29.46% chance of winning.

 

Long Range, 100%
μZ = 2.481 – 4.654 = -2.1728
σ2Z = .01771 + 4.71985
σZ = 2.17659
=(0 -(-)2.1728)/2.17659 = .9982

Plugging into normal distribution, spamming has a 15.87% chance of winning.

Long Range, 85%
μZ = 2.053 – 3.955 = -1.9019
σ2Z = .00144 + 6.23487
σZ = 2.49726
=(0 -(-)1.9019)/2.49726 = .7616

Plugging into normal distribution, spamming has a 22.36%chance of winning.

 

Before you say, “It’s all Greek to me,” let’s take a look at the percentages he provided.  He calculated the percentage of spamming wins, so lower is better.

At short range, 85% bloom decreased the percentage of spam wins from 65.17% to 53.19%.  Now instead of spammers winning the majority of the time in 100% bloom, their chances are reduced to basically 50-50 in 85% bloom.

At medium range, 85% bloom decreased the percentage of spam wins from 33.36% to 29.46%.  In other words, 85% bloom makes it even less likely that spammers will win.

At long range, 85% bloom increased the percentage of spam wins from 15.87% to 22.36%.  Yes, 85% bloom did increase the spammer’s chances at long range, but even 22% is still punishingly low.  It’s really not a good idea to spam at long range in either setting, ever.

 

 

 

 

FINAL CONCLUSIONS:

So after all of these recalculations, new statistics, new charts, and hardcore number crunching, we have still arrived at the exact same conclusions I originally posted:

At short range, spamming wins a majority of the time, but 85% bloom reduces that probability.  At medium range, pacing wins a majority of the time, and 85% bloom further improves that probability.  At long range, pacing wins an overwhelming majority of the time, but 85% bloom slightly decreases that majority.  This decrease will in all likelihood never be noticed in real world settings because spamming loses almost all the time at long range in the first place.

Therefore, I conclude that 85% bloom discourages trigger spamming in comparison to 100% bloom.

 

 

THANKS:

Once again, I’d like to thank everyone who has contributed suggestions, ideas, time, and brain power to this study.  I have thoroughly enjoyed the progression of the analysis as it took its various twists and turns.

Thank you for taking the (admittedly long) time to read this, and I hope you have found this article helpful and informative.  Maybe you even learned a thing or two about statistics along the way (I know I did!).

If you have any questions or comments, please post them here.

Thanks again!

 

 

UPDATE #3:

Last one, I promise!  :)

HBO forum member RC Master brought up another good point.  Joel assumed the data distribution is normal, but it most likely isn’t.  Normal distribution means it will make a nice “bell curve” when graphed.  But since our data has a hard lower limit ( around 50 frames) and is mostly bunched up around the lower end, perhaps his percentages aren’t quite applicable.

For example, Joel’s numbers at Short Range 85% bloom shows at 53% chance of spamming winning.  But if you look at the blue and orange chart I showed earlier, 85% bloom at short range doesn’t get very close to 53% at all.  It’s more like 40%.

This means our suspicions about the data not being a normal distribution are probably correct.

 

Therefore, my results in Update #2 where I re-examine the Win Percentage of Pacing using uberfoop’s suggestion is more likely to give us true, “real world” percentages. After all, I repeated the random pairing 1,000 times, so we should get a very accurate result of pacing win percentage.

Since we have such a large sample size, we can use Standard Deviation, Mean Average, and the Confidence Coefficient of 1.96 to determine our Margin of Error for these Percentages.

Therefore, with a 95% confidence level, the Pacing Winning Percentage results are:

 

Short Range: 100% bloom >> 28.00%  (+/- 0.05%)
85% bloom >> 38.70%  (+/ 0.15%)

Med. Range: 100% bloom >> 62.98%  (+/- 0.17%)
85% bloom >> 69.32%  (+/ 0.28%)

Long Range: 100% bloom >> 85.18%  (+/- 0.21%)
85% bloom >> 84.11%  (+/ 0.23%)

 
Basically, in 85% bloom, Pacing Winning Percentage was increased by 10% at short range (big improvement), increased by 6% at medium range (good improvement), and decreased by 1% at long range (you’ll never notice this difference).

Therefore, my conclusions are not only upheld, but improved.  Compared to 100% bloom, 85% bloom discourages trigger spamming at short and medium range, and there is virtually no difference at long range.

 

Thanks again for reading!