So, I guess my major disclaimer to start here is that my job is like 50% running surveys for a public sector agency that has a legislative responsibility to serve the entire state (the other half is convincing people that they do not need to do a survey which is maybe why I'm here, lol). Regardless, I really empathize with the whole it's impossible to satisfy everyone thing because it is my life. I am all about the value of participatory and collaborative idea generation, but my first question when designing an evaluation scheme is how do we want to measure success and I am always checking in to make sure that our measures are still appropriate. What are the metrics that the mod team is hoping to compare this against? Making the greatest number of current players happy? What if the desires of a slightly active majority are in direct contrast with the desires a really active minority? Is player tenure or avidity for the game considered?
DWRP games generally have different stakeholders groups too -- current players, potential players, and past players that may return. I often think of things in terms of recruitment, retention, and reactivation. If the game is run by popular vote, essentially we are just aiming to retain the majority of participants. If there is recruitment or reactivation, it's essentially going to be accidental. Are we going to conduct polls of these groups too? Arguably they have interest and stakes in the direction of the game.
I'm sort of exaggerating here, but these are the reasons that even a public sector agency whose responsibility is 100% serving our stakeholder groups does surveys to gather points of information about the social landscape rather than polls of what we should or should not do. Obviously, the stakes are vastly, vastly different here, but there is a lot of power in letting a team that has committed to the longevity and maintenance of the game have creative control, and I personally don't have a problem sacrificing some of my power and influence over game settings to allow a team to implement a bigger, more cohesive vision than a game created by a committee that can feel jumbled.
I also worry about expectations that are created through the nature of some of these questions. Does the modteam have the capacity (and importantly, interest) in carrying out some of these changes? There are also trade-offs in effort and different qualities/attributes of each piece of these which you can't get a valid read on from these questions alone. The nature and scope of this feedback project is really admirable, but I am afraid might actually slow mod operations down and make you guys less nimble to changing conditions.
Sorry I have a lot of thoughts about survey instruments 😬😬😬😬 I'd be happy to chat more one on one.
Re: AKITA: MISCELLANEOUS SUGGESTIONS, QUESTIONS
So, I guess my major disclaimer to start here is that my job is like 50% running surveys for a public sector agency that has a legislative responsibility to serve the entire state (the other half is convincing people that they do not need to do a survey which is maybe why I'm here, lol). Regardless, I really empathize with the whole it's impossible to satisfy everyone thing because it is my life. I am all about the value of participatory and collaborative idea generation, but my first question when designing an evaluation scheme is how do we want to measure success and I am always checking in to make sure that our measures are still appropriate. What are the metrics that the mod team is hoping to compare this against? Making the greatest number of current players happy? What if the desires of a slightly active majority are in direct contrast with the desires a really active minority? Is player tenure or avidity for the game considered?
DWRP games generally have different stakeholders groups too -- current players, potential players, and past players that may return. I often think of things in terms of recruitment, retention, and reactivation. If the game is run by popular vote, essentially we are just aiming to retain the majority of participants. If there is recruitment or reactivation, it's essentially going to be accidental. Are we going to conduct polls of these groups too? Arguably they have interest and stakes in the direction of the game.
I'm sort of exaggerating here, but these are the reasons that even a public sector agency whose responsibility is 100% serving our stakeholder groups does surveys to gather points of information about the social landscape rather than polls of what we should or should not do. Obviously, the stakes are vastly, vastly different here, but there is a lot of power in letting a team that has committed to the longevity and maintenance of the game have creative control, and I personally don't have a problem sacrificing some of my power and influence over game settings to allow a team to implement a bigger, more cohesive vision than a game created by a committee that can feel jumbled.
I also worry about expectations that are created through the nature of some of these questions. Does the modteam have the capacity (and importantly, interest) in carrying out some of these changes? There are also trade-offs in effort and different qualities/attributes of each piece of these which you can't get a valid read on from these questions alone. The nature and scope of this feedback project is really admirable, but I am afraid might actually slow mod operations down and make you guys less nimble to changing conditions.
Sorry I have a lot of thoughts about survey instruments 😬😬😬😬 I'd be happy to chat more one on one.