DIR Return Create A Forum - Home
---------------------------------------------------------
Airbattle Games
HTML https://airbattle.createaforum.com
---------------------------------------------------------
*****************************************************
DIR Return to: Wing Leader General Discussion
*****************************************************
#Post#: 875--------------------------------------------------
WL:Origins Test Status
By: pilotofficerprune Date: January 14, 2019, 2:32 am
---------------------------------------------------------
This will be the thread where I'll post updates on the test
status of Origins scenarios. For newcomers, the test status is a
list of scenarios which shows the number of tests on each
scenario and who has signed them off as complete.
Early in the project I will not tell you what to test. You can
pick and choose for yourself, and see from looking at the
updates what scenarios need testing.
I keep it open like this because each of you prefers to work in
a different way. Some like to start testing from the front of
the list, some from the back of the list and some in the middle.
Some of you are concerned only with testing scenarios that you
created.
Late in the project I shall direct people to focus on particular
scenarios that need attention. Keep an eye out on this thread
for those instructions.
Some things to keep in mind:
(1) I do not expect everyone to test everything. There are a few
guys like Elias and Gordon/Andrew who will try and test
everything, and I encourage others to do the same. However, for
most of you, it's okay to concentrate on a handful of scenarios.
(2) Please use the playtest report template as the basis for
each after action report. The most important information is: the
version being played; the result and breakdown of victory
scoring; your list of recommendations. If you simply posted
that, that would be sufficient. The template can be found here:
HTML https://airbattle.createaforum.com/wing-leader-playtest-reports/scenario-playtest-report-template/
(3) Please try to test a scenario more than once. Two or three
playings will help you calibrate a scenario better. Remember,
what I really want from you is recommendations for changes or
tweaks, and though one playing might suggest changes, two or
more playings will help refine those tweaks.
(4) Do not be afraid to suggest changes and fixes. No-one is
going to be offended by change suggestions, though sometimes
they can spark interesting discussions. You are welcome to
suggest changes and test them yourself immediately. That helps
validate your change suggestions more quickly.
(5) If you've tested a scenario a few times and you are happy to
sign it off, please say that you want to sign off. When I've got
enough signoffs (preferably two or more) I'll mark the scenario
as 'Done' and stop testing it.
#Post#: 876--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 14, 2019, 2:37 am
---------------------------------------------------------
[quote author=Lee Brimmicombe-Wood link=topic=60.msg875#msg875
date=1547454728]The result and breakdown of victory
scoring[/quote]
Oh, and this is important: in addition to the actual score and
result, tell me what the impression of the result was. If the
result was a draw but you thought it should be a German victory
because of some reason or other, this is important information
to communicate in your playtest report.
#Post#: 994--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 19, 2019, 5:00 am
---------------------------------------------------------
Announcement
Okay, I am declaring the Origins test open. Some notes:
(1) As discussed on the WL: Origins thread, at this early stage
you are free to test anything. However, please talk openly about
what you are testing, so that you don't step on each other's
toes.
(2) Once things have settled down I will start directing people
to focus on particular scenarios.
(3) Please do focus on one scenario for at least two or three
tests, so that you thoroughly understand it.
(4) Testing is an iterative process. It's important that you
make recommendations for changes to a scenario and then test
those changes.
(5) Feedback is everything. It's not enough to say 'I played it,
it was okay'. You need to break down any strengths or
weaknesses, and communicate any concerns or recommendations for
change.
(6) I have posted a loose feedback template for you to use. The
after action report does not have to be long; you can make it as
short as a few sentences. However, the following is the vital
information you MUST post for me:
- The version number (scenarios change rapidly and I must know
what iteration you have tested)
- Victory and the breakdown of VPs (I spreadsheet these results
for balancing purposes, so I need the numbers)
- Recommendation (please, PLEASE make recommendations for
change, or if you don't have any, say so; it's vital we change
and iterate scenarios, especially in the early stages, so as you
play and afterwards, please think about what might fix problems
or improve a scenario)
#Post#: 1025--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 20, 2019, 4:13 am
---------------------------------------------------------
So, below I've posted the first, near virgin, test status sheet.
As the project goes on, we will track the playings and status of
scenarios here. For the new bugs, here's how it works:
(a) We track the total number of playings, so you can see at a
glance what scenarios have had attention and which haven't. If I
haven't otherwise directed you to test a particular scenario,
this should allow you to make an informed choice as to what to
test next.
(b) The numbers of tests are broken down by whoever posted the
AAR.
(c) The status column, lists scenarios by 'Poor/Okay/Good/Done'.
The traffic light system in this column is not a measure of test
quality as such, but more a measure of whether we have done the
minimum due diligence in numbers of tests. My minimum bar for
'Good' is: five tests by at least two different hands.
(d) The signoffs column is important. This lists who has signed
off on a scenario and I use this to gauge when a scenario should
be marked as 'Done', though I largely go by a mixture of this
and the playtest reports. On average it takes around 8 playings
to sign off a scenario. Sometimes, it's much, much more.
[attachimg=1]
#Post#: 1061--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 21, 2019, 1:03 am
---------------------------------------------------------
An interesting question has arisen: at what point will I accept
a signoff on a scenario from a tester? At the moment, the answer
is 'it depends'. We have a few new guys who are unknown
quantities as testers, and in some cases are still learning to
master the game. However, if as with Dan they drop four tests of
a scenario on me, I'm probably going to be happy with them
giving me a signoff. With the super-experienced hands like
Gordon or Elias, it may be fewer--though those guys are probably
going to deliver at least three or more tests anyway, before
they sign off.
#Post#: 1180--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 24, 2019, 4:59 pm
---------------------------------------------------------
Thanks for all the tests so far. I really appreciate every AAR.
I get a bunch of things out of these:
(1) Errata and problems with scenarios. Already we've had
comments back on typos, how the special rules are worded and
countermix issues that hadn't previously been identified.
(2) Accounts of the gameplay. These sometimes reveal issues we
need to keep an eye on, such as bomber evasion, lack of radios
on unit fragility, as well as scenario-specific stuff.
(3) Victory data. The VP breakdowns are super-useful and are
going to be key to balancing scenarios. You'll find I'll
frequently crunch the numbers on VP outputs and other measures
to find the sweet spot for victory conditions. And I'm always
looking at this in context with your perceptions of whether
something was a win/loss for one side or another.
(4) Recommendations. Even saying you don't have any
recommendations is useful. Though I certainly see
recommendations as a healthy sign that we are improving
scenarios.
One thing I haven't asked for, but is important, is whether the
scenario is interesting and fun. Whether it supplies enough
challenges. I love to hear it if you think a scenario hits the
spot. But don't be afraid to say if something is boring, or has
too few ways to solve the tactical problem. That's a signal for
me to look at the scenario design again. And of course, if you
think you have a way to make it more interesting, or more fun,
then please make a recommendation. I'm all ears!
#Post#: 1242--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: January 27, 2019, 3:58 am
---------------------------------------------------------
Okay, here's the current status of the test. As you can see it's
been a gala first week with no less than 21 plays. Our regular
test rate is normally lower than this, but it's great to see a
surge of feedback.
Some quick notes if you are still new to testing:
Status. The scenario status column does not indicate how close
the scenario is to completion. Rather, the Poor/Okay/Good
traffic lights indicate whether we have done the minimum of due
diligence to testing that scenario. the minimum I define as five
tests by at least two test teams or solo testers. I don't sign
off anything that hasn't reached this minimum.
Signoff. The signoff column is more important, as it indicates
the mass of people who think the scenario is ready to ship.
Scott has signed off on Operation Zet, but we need more people
to sign off before I declare this done.
As you can see, two scenarios have received a bunch of tests,
but there are plenty that haven't been tested at all. Don't be
afraid to test a scenario that someone else has started testing.
We need multiple sets of eyes on each scenario to push them over
the line.
[attachimg=1]
#Post#: 1393--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: February 2, 2019, 5:51 am
---------------------------------------------------------
Okay, here's the current status of the test. We've had another
great week of feedback, all of which has been useful and has
filtered into the latest round of iterations.
[attachimg=1]
Test Direction. I don't yet feel the need to direct anyone
regarding what scenarios to test. I think you guys can look at
this and self-organise. However, I'd like a veteran tester (not
Elias) to play O02 Operation Wasserkante a couple of times and
give us an assessment, following the recent round of changes.
#Post#: 1613--------------------------------------------------
Re: WL:Origins Test Status
By: pilotofficerprune Date: February 10, 2019, 5:29 am
---------------------------------------------------------
So here is the current status of the Origins test.
Again, thanks for all the feedback. I swear you guys are going
twice as fast as I’d anticipated. We are iterating scenarios
really quickly. My estimate of 215 playings for the entire
module may need to be revised upwards.
[attachimg=2]
Though we don’t have many signoffs, I’d like, for now, to do a
shift testing away from those scenarios in the green, that have
had a number of plays. We’ll come back to them again later, with
fresh eyes.
For now, can people please test those scenarios that have had
just one or two tests? These scenarios have had enough test to
shake out initial bugs, but could do with a closer look.
#Post#: 1694--------------------------------------------------
Re: WL:Origins Test Status
By: Elias Nordling Date: February 12, 2019, 1:01 pm
---------------------------------------------------------
Pete and I are starting another round of Shall We Cancel the
Party anyway because we'd like to see the effects of the
changes. Other than that, I'm focusing on my special assignment,
and Nemesis next :-)
*****************************************************
DIR Next Page