After the Webinar: Beyond CompStat – The Role of EBP in Policing. Q&A with the Presenters

Webinar presenters Dr. Renee Mitchell and Rachel Tolber answered a number of your questions after their presentation, Beyond CompStat: The Role of EBP in Policing. Here are just a few of their responses.


Audience Question: Several of our audience members are asking for the citation for the Critical Incident Stress Debriefing study. Could I ask one or either of you to send me that citation and what we can do is we can put that link on our webinar recording page if that’s possible? If you have it handy, you can certainly share it verbally too if you like. 

Dr. Renee Mitchell: Give me a minute and I will find it. I have it somewhere. I usually have it on the bottom of that slide but I don’t think I have it in this edition of this PowerPoint. I will find it and give it to you and send it out.

Rachel Tolber: I think we also may have a research brief we can get. Let me follow it up to you on that.



Audience Question: How do you counter the argument that people sometimes say that evidence-based is just one more thing or it’s a new fad or it’s a new face. The counter to that is why aren’t the tried and true practices why aren’t they just as good if they have never been tested. How do you argue with folks who respond with that? 

Dr. Renee Mitchell: To me I try to explain that evidence-based policing is really the umbrella that all of the practices should be under. It’s not a fad. It’s really testing our practices. It’s not like a strategy. It’s not like problem-oriented policing where here I’m going to have a pop approach, or community-oriented policing. It’s not that. It’s testing if your pop approach works. It’s testing if your community-oriented policing works. It’s testing to see if your homelessness intervention works. It’s testing. It’s the same way when you think about when people talk about tried and true practices. The reason we were stuck with blood-letting for over 1800 years is because nobody actually tested if blood-letting was an effective medical intervention or not. They just thought it worked because in their experience they probably saw some people get better. They probably saw some people get worse. With cognitive biases, because they are doctors and they are supposed to do interventions for good, they just thought it worked. To me, and that’s just like the test study with arrhythmia. That study was a pivotal moment for medicine because everybody was sure that arrhythmia medication would help heart attack sufferers not die. They weren’t correct. When you are arguing that piece in people in some ways, it’s hard when somebody’s embedded in their practice for a long period of time. They’ve actually shown that if you go to a doctor and you have some major health issue, you should see a younger doctor because the younger doctor is up-to-date with evidence-based practices and new research whereas a lot of older people get embedded in how they do things. To me, in some ways, I try not to have the arguments because I don’t think they are going to change people’s minds. To me, we owe it to the public when, one, we are spending their money on our policing services and, two, we have shown that we have created harm to our community consistently in like every profession there is. Humans get it wrong all the time. Who are we to say in policing that what we are the only industry that gets it right all the time because how do we know?



Audience Question: Should the police carry-out the evaluation of the interventions or would it be better for an external party to conduct such evaluations? 

Rachel Tolber:  That’s a really good question and I really think that it could be either way, and frequently is. There is a growing trend for a lot of us that are practitioners to become educated, to be able to carry out the evaluations internally. One of the things that we really do speak to and a goal of ASEBP is facilitating partnerships between academics and practitioners. Like I mentioned earlier, I’m a practitioner, I go to work every day in policing and I’m not always thinking about methodology. Having that partnership to be able to bounce these ideas off of each other and work together I think really lends itself to a very good outcome and a good body of evidence and research.



Audience Question: How are we supposed to this when so many agencies are so short-staffed, they are already stretched thin? Am I assuming correctly Rachel then that your advice is best to win a partnership especially an academic partnership is still valuable?

Rachel Tolber: Absolutely. That is very much the case. All of us, our motto is do more with less. If you have a local college in town, reach out to them. If you have questions for ASEBP, we have members that are both practitioners as well as academics. A lot of academics are able to have these conversations, want to have these conversations and want to partner. I’ve been very fortunate to do some work with Dr. Natalie Todak and a variety of people, Renee as well. I think partnership really is key.

Dr. Renee Mitchell: Also I think sometimes when people talk about being stretched thin. To me, that’s when your effectiveness and efficiency comes in. If your organization is stretched thin in a way taking half your city, we did this is Grand Perry about going beatless versus patrolling by beat. We just took half the city and went no beats and the other half, we went and they stayed in their regular patrol duties and the beatless side responded the hotspot calls. We just compared the half of the city to the other half. That’s where field research is not perfect. The question about having outside entity yes that is absolutely the best way to do research: it is to have somebody from the outside examine your data, but if you don’t have that you can still examine your data on the inside. A randomized control experiment is the best way, the gold standard of experimentation or research but when it comes down to it if all you could do is say okay I’m going to test this intervention in this one little area of the city and then here’s my other area that is a pretty good comparison group and I’m going to run my intervention for six months or for however long to get enough data. That’s pretty good rough research. It might not be scientifically published in a journal article, but it gives you very good information about your own organization. That’s where the academics live in this kind of gray area and it’s where most of the pops live. It’s not a perfect science but it’s a better way than just doing what we have always done the same we’ve done it.



Audience Question: How do you do evidence-based policing and innovation at the same time? Can the former hinder the latter?

Dr. Renee Mitchell: I don’t think so. I think if you have a new innovation, you are just testing it. Create a way, go create all the ideas in the world, just test it. Randomly assign them. If it’s an intervention for your cop, randomly assign which cops get it. If it’s a patrol intervention, randomly assign areas. The other nice thing we’ve seen done throughout the world by cops, not just by the researchers is they’ll randomly assign the days of the intervention. If you don’t have a big enough of a sample or an area or what have you, you can have, depending on the intervention, they’ve done it with like hotspot policing, is you can have it on certain days and off certain days. You could compare the days that it is on and the days that it is off and determine if your intervention is working. I personally don’t think it hinders innovation. I think all you are doing with evidence-based policing is you are testing your creativity. That’s all. I don’t know Rachel, what do you think about that?

Rachel Tolber: I totally agree and I think it really goes back to the conversation we had earlier which is it is a mindset, right? We still want to be innovative and think of new interventions, new ways, there’s always new technology. Then, again, it is the mindset of, okay how do I know that what it is that I’m doing is working? Once you change that mindset and that’s the question that you are asking, then it would follow the point that you are building in this evaluative component that Renee is discussing.


Click Here to Watch a Recording of Beyond CompStat: The Role of EBP in Policing



Additional Resources
3 years ago
Thoughts about Research from Shon Barnes
We loved this statement Deputy Chief Shon Barnes made during his webinar, Implementing Evidence-Base […]
3 years ago
Implementing Evidence-Based Policing: Lessons from the LEADS Program
Evidence-based methods have been utilized in medicine and sciences for a significant time and it has […]
4 years ago
LEADS Scholars & Agencies: NIJ Initiatives to Advance Evidence-Based Policing
The National Institute of Justice (NIJ) is the United States Department of Justice (US DOJ) arm that […]
4 years ago
LEADS Scholars and Agencies: an Interview with Gary Cordner
Evidence-Based Practices have become a significant part of conversation throughout the justice commu […]