In the past few years there’s been an increased emphasis on data-driven management in the association management field. This is to be applauded. But data alone will not improve management decisions. At L&M we believe there are five basic steps to solid management studies to increase the likelihood that the right questions are being asked to produce the best answers that time and budget allow.
1. A Problem Well Stated is a problem half-solved1
A clear understanding of the problem to be solved, or the desired outcome of the decision to be made, sets up a question that can be more easily researched. Perhaps even breaking the problem into several discrete, but related questions, can reveal a series of smaller and more digestible research questions.
2. Selecting the data to gather
Many management studies do not have to involve paying for data or the collection of data. Start by asking:
- Do we have data on our membership from the last membership renewal cycle that can shed light on this question?
- What does our most recent (or past few) membership surveys tell us about the question we’re trying to resolve?
- Are their published studies or data sets at trade associations, professional societies, analyst firms or consulting firms that can shed light on the question under consideration?
- Lastly, if existing studies are not adequate, what data should we seek to collect to answer our key question(s)?
3. Data collection
This step can present a significant barrier to the exercise due to time and/or costs of data collection. Obviously, if a decision is required before data can be collected, then we might need to begin the data collection process with the understanding that it can be useful to evaluate the decision after it’s been made and use the data to make adjustments to the original decision down the road.
It often makes sense to conduct tests on both data collection techniques to ensure that the exercise is worth the effort, and to sample an initial data set to make sure assumptions about the value of the data are valid.
It’s okay to make mistakes in this process so long as learning takes place about how to do this better the next time.
4. Data Analysis
Don’t be confused with the word “analysis” in thinking that this step involves interpretation of the data. That step comes next. This step involves the normative presentation of the data.
For example, assume you want to study membership trends over the past 3 or 5 years (whatever period of time seems to make sense for your organization). The question might be: “How is our membership retention and recruitment going to change in the next few years – do we need to do something different?” Before you begin considering external forces that might impact membership trends in the future, you need a clear representation of membership trends over each year in the study period (e.g., how many per membership class, renewal rates, and new members). If the data exists, you might also try to identify each of these trends by member age if a professional society, or size of company if a trade association.
The point of this step in the process is to obtain the most detailed analysis of “what is today” — not why.
Here we begin trying to understand why the situation is the way it is today. In this example of trying to understand membership trends it’s tempting to ask members. The weakness in this approach is that members only know why they renewed or didn’t renew; they don’t have the overall picture that association staff should have. It’s much better to go through these five steps before surveying members. This process may yield a better set of questions to ask members and former members — if you’re going to ask a target audience to give you some of their scarce time to answer questions, make that precious time count!
Continuing with the example of membership trends, you might plot the data over time and begin adding organizational meetings/events, launches of new programs, releases of publications, changes in membership benefits or dues, etc. to the time line. Do you see relationships between these events and the membership trend lines? Is it what you expected? If not, why not? Do new questions emerge? Do you see how you can improve your knowledge by collecting different data in the future?
It is not necessary to conduct scientific studies to improve management decision-making over time. It’s also not necessary to be a statistician, although it helps a great deal to have a good grounding in introductory statistics. If it’s been a long time since you took your first statistic class, buy an introductory text book and you’d be surprised how quickly it comes back to you. If you’ve never had a statistics class, take one at a local community college or night school, or perhaps an online course.
Get in the practice of asking questions about the decisions you need to make, attempting short-hand ways of answering them, and always – always – be your own devil’s advocate! Once you’re posed a question ask: “How many ways can this be wrong?” “Does it matter if it’s wrong?”
It’s a process and the sooner you get started, the sooner you’ll become proficient at it.
1 Charles Kettering