To start this is not asking when the beta is as I'm sure that even the alpha is a while away and will gladly wait till its ready, instead this topic is to discuss the differant approaches of running a beta I've seen as well as their pro's and cons.
To me I see three differant sorts of beta type:
Beta Weekend -
In this form of beta the developers open up the beta servers for a duration of usualy 24-48 hours over a weekend period, though this has been known to be shorter in some instances.
Pro's: This allows developers to stress test select things like server loads and lag as well as key features, as well as being the cheepest method of beta.
Cons: Very limited time for testers to test all aspects of the game before release and can miss certain long term problems such as a data leak that will go unnoticed during the 48 hours timeframe but after a week would cause the game to become unplayable.
Opinion: Personaly I see no problem with beta weekends at the start of the beta development cycle to help fine tune major issues such as server lag and population limits, but later on when it starts getting closer to launch I'd rather see more full beta access.
Beta Week -
Similar too Beta Weekend only your keeping the servers up for at least a whole week at a time.
Pro's: Longer timeframe to locate more gamebreaking bugs and exploits as well as potential data leaks that could have been missed during a Beta Weekend.
Cons: Slightly more expensive than a Beta Weekend due to having to run the servers for a much longer time and while given more time beta testers may still not be able to cover all the content needed in time.
Opinion: This method seems to be a bit more of a happy medium and can allow for more bug finding than a weekend.
'Full' 24-7 Beta Access-
Pretty much the servers go up and do not come down except during scheduled or emergancy maintinance.
Pro's: Beta testers are given full reign to test just about every aspect of the game and potentialy find every bug, as well as more easily identifying long term data leaks and glitches.
Cons: Again the cost of maintaining the servers 24-7 goes up even more, as well as the potential idea that if you essentialy give beta testers full access to the game they might burn through all the content and descide not to buy the full release after they've 'done it all'.
Opinion: While probably the dream method from a beta testers perspective, may not be best from a developers perspective especialy with the added costs and risk to reward later.
Conclusion: As can be seen in my personal observations and opinions I would favour the Beta Week option of the three, namely from what I've seen of the big companies using beta weekends as their sole method of beta testing the end result has always been a extremely buggy mess at launch as their purely using beta weekends as a method of building hype for their game rather than actual bug/stress testing.
Note: This thread is for discussing the Pro's and Cons of various beta methods, please keep it on that topic, thank you.
Edit: Changed title to be more clear on subject matter
Sounds like an interestingidea! I'll pass it along to the team to keep in mind.
Just to point out that it is possible (and in my mind) recommended to do at least 2 of of the 3 listed here...
Although you did miss out on one form of "beta"...
The genuine stress test: The servers are up for just a *few* hours, and you try to get as many people on the servers as you can.
Oh, and I would say that its recommended that you have do at least the "small scale" and then a "large scale" form of testing as well... so going from the "stress test" style, a few times, and then have larger week long tests as well (even if its *EVERY* week for 3 months, there is a scheduled 24 hours-48 hour downtime on a regular basis).
The 3rd option is possible and a valid one... however, what I do dislike is companies selling beta access with pre-orders.
I always think these are going to be a good idea, they invariably are not.
Software release cycle
http://en.wikipedia.org/wiki/Software_release_life_cycle
It's very much a process, normally you don't see most of it as it's in-house/closed.
If people won't pay enough to finance its creation, it is not worth creating.
/Segev
Actualy the 'stress test' you mentioned is part of what I mentioned in the beta weekend section as stress testing the servers and is part of the reason I said that beta weekends are fine towards the start of the beta development cycle to iron out the major bugs and see just what the servers can handle and what if any population limits need to be put into place.
And yes the Pre-Order concept is a very tricky topic that may require its own thread of discussion to go into its Pro' and Cons.
And thanks VDG, realy do hope this discussion can help MWM when it eventuly gets closer to desciding how to put together the beta for CoT.
I think the important thing to realise is that most MMO's go through ALL of these phases... even if you dont necessarily *know* about them.
Tabula Rasa for example, it had a 4 day week during one phase, whilst interspersed with a couple of "stress test" events, and ended up with the 24/7 open phase in the couple of months up to release (this is all if i remember correctly),
Another MMO that I am testing right now, seems to be following the same set up. In fact, I believe that they have said that "Open beta" will ONLY be for one month, and that is the month before release.
This MMO has already had a couple of stress test events, and has had several "closed beta" phases (where the server might be up for 5 days out of 7 for a couple of weeks, and then down for a period of time whilst they work on other stuff, and then back up).
So whilst you might prefer *one* style of "beta"... it isn't all that uncommon to see companies use *all* of the "beta's" that you have mentioned....
Well, part of what we're doing, philosophically, is engaging in staged rollouts of this game. So, for example, when Avatar Builder is finally functional, parts of it will have tested the network infrastructure and authentication, in addition to the character creator, and so on. And as we continue to work towards the day when everyone logs into Phoenix Plaza, we're going to have a series of these 'beta weeks', for this sub-product and that sub-product, testing this and that subsystem as we go along. If we're very lucky, it'll all be tested by the time we're ready for the traditional beta. (It's never all tested by the time anyone's ready for traditional beta.)
Beta Weeks do seem to be the way to go, but we're being a bit Agile about things, and we're going to be working on an iterative cycle that's going to be a little different from how many MMOs do things. Not that we can't do it the other way, we just think this'll work better. If it doesn't, we can fall back to tradition.
But hey, if we see a way to improve how things are traditionally done, we're going to do it.
[color=#ff0000]Project Lead[/color]
If you'll excuse the pun (I wouldn't, but then I'm the one making it), this thread appears to be a case study in ... Premature E-Speculation.
/em [url=http://www.youtube.com/watch?v=uXILNncQwH4]rimshot[/url]
[url=http://en.wikipedia.org/wiki/Scrum_%28software_development%29]Scrum[/url]?
My first introduction to Scrum Development was delivered (in person!) by no less important a friend of the community than ... [url=http://paragonwiki.com/wiki/CuppaJo]CuppaJo[/url] at the [url=http://tabularasa.wikia.com/wiki/News/20080529]Tabula Rasa War College[/url] (I was [b]Line[/b] and Lin Chiao Feng was [b]Lanster[/b] among the listed attendees) in June 2008. Needless to say, ever since that explanation by CuppaJo as to how Tabula Rasa organized its development, I've always been rather partial to that sort of goal oriented structure for getting things done.
[center][img=44x100]https://i.imgur.com/sMUQ928.gif[/img]
[i]Verbogeny is one of many pleasurettes afforded a creatific thinkerizer.[/i][/center]
Gamers, changable of mind? Unpredicted challenges? No, not _our_ players... oh, wait.
Maybe a tiny bit. Of course, proper planning prevents poor performance.
[color=#ff0000]Project Lead[/color]
And planning doesn't survive first contact. Especially not this bunch.
Seriously, we're getting way ahead of ourselves. Let's let the devs make something that can be pushed to beta before worrying about the logistics of a beta.
Anyway, regarding Scrum, the thing that impressed me the most was that each scrum team had to give a presentation at the end of the two-week run, explaining what they did to everyone else. I've been on too many teams where you don't hear from the other folks until way late, and then you realize that your team wrote five slightly-incompatible libraries that do generally the same thing, and built a bunch of code on top of them so it's too much work to merge them...
[i]Has anyone seen my mind? It was right here...[/i]