I am sometimes pleasantly surprised at how devs can figure out what little things need to be polished in a game. It's nice when I see a particular functional update and think, "Oh yeah, that used to be a bit of a pain". How do the devs figure out what to polish?
There are some obvious answers. Some devs are also players. Some get feedback from forums. In some cases there are in-game feedback mechanisms. There are also metrics.
I was hoping the devs could speak a bit about plans for metrics and in-game feedback and that we could brainstorm on what mechanisms could work.
IN-GAME FEEDBACK:
I have a couple of thoughts on this one. First is that it should be quick and easy. Often these features are buried in a menu and require all sorts of fields to be filled out. The game client should be able to capture most of the relevant game state information. The player should just be able to describe, perhaps verbally, what's wrong or could be improved upon and move on.
Second is that the feedback mechanism should be available virtually everywhere. By definition it may be difficult to have this feature on a loading screen, but many games omit it from character creation, server selection, pre-login, post-logout, cut scenes, and whatnot.
METRICS:
What are the plans for metrics? What data do you plan to capture and how do you expect to use it? For instance if many players pause or take a long time at a certain point in the game this could indicate either good or bad player experience. A long time in the character creator could be good or bad for instance. How would you differentiate the two? Will chat records be data mined? For instance in my opinion "lft" = a failure of the teaming function. What other sources of data do you intend to tap and what do you expect to get out of it?
"THE TITANS ARE COMING! THE TITANS ARE COMING!"
I've been doing the alpha testing for Camelot Unchained for a couple years now. And yes, it has been true alpha testing. And I've also done closed betas on almost ten other games (some were mere cash-grabs but others took their beta status seriously). The best mechanism I've seen is a "Bug" command or button. Every game company does it a little bit differently, but the ones that work the best are:
type "/bug" in the chat window and a bug report window opens up.
open the game menu and select "submit report"
Usually there is a decision tree to go through that makes sure your issue gets categorized. For instance, there are connectivity issues, graphical issues, animation issues, grammar and text issues, balance issues, etc. etc., each correspondng to a developer's skillset.
One of the things that CU has done is open up a restricted forum that only testers can use to log their detailed reports. One of the most helpful things that CSE has done with CU is they sent a PDF file to all their testers telling them how to record a dxdiag log and how to attach it to your forum post, among other things.
Finally, I think it is important to focus the testers on what you want tested. When an alpha or beta player jumps in, they need to be coached to test things like "die and respawn as many times as you can," or "we need people to test crowd control abilities on Rook NPCs tonight," or "make as many different characters as you can and try this list of emotes on each." This kind of testing would be happily performed by the potential player base. But only those players who have been read into the non-disclosure and expectations of the tests.
I like to take your ideas and supersize them. This isn't criticism, it is flattery. I come with nothing but good will and a spirit of team-building. If you take what I write any other way, that is probably just because I wasn't very clear.
It's also there so you don't need a dozen people working full-time just to sort the flood. Because if one person finds a bug, chances are others will find it too, and it's tough to figure out that a set of reports are talking about the same problem.
Anyway, City of Heroes had a
/bug
command which pretty much did what you described. There was also a/petition
command that went straight to the GMs for problems that needed to be fixed quick (broken task force goals and the like). We'll have at least that much.Has anyone seen my mind? It was right here...