After the Webinar: Effectiveness Evaluation of Implicit Bias Training. Q&A with the Presenters

Webinar presenters Renee Mitchell and Lois James answered a number of your questions after their presentation, Effectiveness Evaluation of Implicit Bias Training: Research Study Results. Here are just a few of their responses.


Audience Question: What tests or methods are available for a person to see if he or she does have implicit biases, or how biased they might be? You mentioned 1 or 2 during your presentation. What were they, again? 

Lois James: The implicit association test or the IAT, is definitely the most widely used of all of the implicit bias measures and you can go online at Harvard’s project implicit, I believe. And you can take on any number of tests in, terms of just kind of knowing yourself and what-not. Another that I like that I think is quite a good one. It’s slightly different. But it’s the subtle and blatant prejudice scale or sub-scale. I think it’s called, and that’s one that I know that I use quite a bit in terms of other studies and measurements and what-not. I know the IAT is the one that is usually the go-to.


Audience Question: Is there any or more or less bias and policing practices as compared to other professions? So, such as teaching or nursing, or is it merely just because of the news coverage it seems that there’s more bias? 

Lois James: That is a great, great question. I would say, part of it is exposure range. So, I mean, we do, in several studies that we’ve kind of run, we seem to see slightly higher than then, kind of US general population, in terms of implicit racial bias with some officers that we’ve tested. Certainly, I mean, it’s not a critical mass that we have tested. And part of that, unfortunately, is exposure-based. So, as I said, if you have an officer who works in a predominantly Latinx community that’s quite low-income and high crime, it is going to be impossible for the officer, not to develop implicit associations between members of that community and threat or crime. Does that mean that that officer is going to discriminate or behave and appropriately, not necessarily. So, that’s one of the things with implicit bias, and certainly with the IAT, that I always urge a little bit of caution in terms of interpreting too much. It’s really useful for knowing yourself. It’s really useful for kind of bettering yourself, but not necessarily for monitoring or judging people if that makes sense.


Audience Question: What are the most important factors that determine any one specific implicit bias’ resistance to change? 

Lois James: I would say the more deeply ingrained it is. And I don’t know whether that means, that it’s been around for your whole life. So, something you were kind of exposed to as a child. We know that some of the implicit biases that you do form very early in life tend to be the ones that are the most salient but I’m not actually kind of sure beyond that.

Renée J. Mitchell: I’d be hard-pressed to answer that too, I would almost say like maybe answering it in the reverse. Like when you’re talking about like, officers in the context, because we do know that that your implicit biases are malleable. So, if you’re working in a bad area, or a certain group of people that your context or your implicit biases, can change. Right? If things are difficult to change, then it could be something in the context. So, more than thinking about like the person thinking, what it is that makes it difficult for that. So, if you’re always working around

Lois James: That consistency, you’re being exposed.

Renée J. Mitchell: And it can be anything, like, if you’re always working around like the elderly, and it’s a negative interaction all the time, and I don’t mean the negative could be any type of thing, right? Like, maybe the elderly with dementia. So, it’s constantly, like emotionally weighing to get out of that context of viewing the elderly in that way. Then you want to put them with the elderly that are doing fitness classes, so they can see that not every single person who is in this age range is at a level where they can’t take care of themselves anymore. You know what I mean? So, I’m trying, I’m working at it from the reverse, but that’s where I could see is like, it being the stickier. Besides what Lois had said about like something that’s really salient that you had from the moment you were born and exposed to, but also like the context of being immersed in it every day. I would think that would make something also very sticky.


Audience Question: So, are there any plans to have a longitudinal aspect to this to go back over a longer period of time a year later or two years later to see if the training sticks? 

Lois James: Well, that’s kind of the survival analysis piece that Renee was talking about. So, I mean, it’s longitudinal in the sense that we have kind of 9, 10 months of data following the intervention. So, we can assess it to that end how long does it take before the training effect wears off. So, that survival analysis is basically just looking at how long until it goes back to baseline. If it does go back to baseline. Right. Because at the moment, I’m just looking at we’re looking at aggregate performance before aggregation performance after, but that’s averaged across that, basically a year and three quarters of the year.


Audience Question: How does implicit bias training relate to the massive amount of anti-racist training or equity training, and related topics, that are taking place in many fields, and seem to be the focus of many academic conferences over the last few years? 

Lois James: Yeah, great, great question. And we do we’re seeing a lot of a kind of mindfulness, anti-racism kind of DEI. There are many, many different types of training. I would say the implicit bias training is predominantly focused on the human brain, like, kind of the mechanics of why it is that we think the way that we do, right? So, it’s by no means the only, or even necessarily the most important component. But I think one of the reasons why it’s stuck it out with lots of new trainings that are coming up, is that it really explains why we think the way that we do, how we develop, what implicit bias is, and why it’s so kind of challenging, and why we need to continually work on us. Right? That’s not to say that the anti-racism and mindfulness, emotional intelligence, and all of these other trainings aren’t important. They very well might be, but they’re just a little bit different if that makes sense.


Audience Question: In your coding, did you document those contacts with weapons or impairment?

Lois James: Yes, and yes. So, I mean, obviously, in terms of the body camera footage that we were accessing, I mean, if there was something that was still kind of under investigation, that I’m pretty sure we did not have access to that, but we didn’t have a proportion of our interactions, a very small number, but then that did involve weapons of some sort, so, yes, that was all documented in there as well.


Audience Question: What tool did you use for the BWC coding? 

Lois James: So, the tool that I described was a coding mechanism that we developed at Washington State University from a series of four different studies now. So, we have metrics that look at the use of deadly force. We have metrics that look specifically at crisis or CIT encounters. We have metrics that look at tactical social interaction. And what we did for this particular study is we selected a series of metrics from kind of each of those interval level metric tools and kind of collage of them for use here. So, they’ve undergone kind of a series of validation at this point, because one of the things that were very important to us is we need to, we needed to have a valid and reliable tool for measuring police behavior or performance.

Renée J. Mitchell: A large number of years went into the development of that.


Audience Question: So, often when we talk about bias and policing, it’s around police and citizen encounters. What bias studies or, what does the research say about bias that’s experienced by the officers within their own agencies or management structure? 

Renée J. Mitchell: I can’t think of like studies off the top of my head. But they’re going to experience the same bias

Lois James: There’s been a lot of anecdotal work, certainly.

Renée J. Mitchell:  Yeah, because there are women, there’s diversity, there’s everything, and we’re human beings, so the same biases that run throughout humanity are also going to run organizationally. So, as you become older, you’re going to suffer from those implicit biases and how you’re viewed, being a female officer versus male officers. Some of those are just explicit biases for some people. And so, but I don’t know, Lois, if there’s an actual study about that.

Audience Question: I know, Natalie Todak has certainly done some interviews with officers, with women officers, with officers of color, certainly, in terms of some of the kind of the barriers and the challenges faced. And even to some extent,  your study on women, in kind of the more extreme teams, you know? I mean, there is really, really excellent work. Although, maybe not directly addressed, that particular question has certainly gotten out of it. And I think that work is being used to feed some of the efforts around kind of increased DEI in recruitment and retention and so, on, which is a really, really critical area that’s gaining more, and more traction, I think.

Renée J. Mitchell: Yeah. That’s an area that I would say like research needs to be done because you think about my favorite French horn example, if you’re using those same parameters if you’re thinking about how you evaluate within the academy. And if it’s a male archetype that you’re using, women might be evaluated at lower rates because you’re seeing them through this male lens. Right? So, how do you get around that? Because you can’t put up a screen and say, Okay, let me see you do it, place somebody under arrest, or be involved in this scenario. And I’ve thought about those things, like, how do you develop a method that helps take away those implicit biases. But I’m not that smart to think about, like, how you could actually do that in the field.

Lois James: Plus, I think it is going to depend on shifting culture, right? I mean, part of it is that fundamental question of, Okay, well, what type of officer do we want? Right. And I think that is changing and has changed even in the last decade even the last eight years, I would say that is changing and so it’s not just a matter of, Okay, we need to be really objective and unbiased in that selection process. Because if you’ve got out of 100 people that show up, if 95 of them, are white men, that’s not really going to solve the problem. Right? You need to think upstream. Okay, well, how do we convince people who are, you know, to this point? Maybe would not have even thought a career in policing was a possibility, either because they didn’t want it, or they thought that they wouldn’t be welcome. How do we change it there? It’s the same with the orchestra piece, right? I mean, how do we get more women through Juilliard and so on to get to that point, where they actually can audition at all?


Audience Question: As a field training officer, how can I help open the minds of the trainees who come through my car in respect to the sort of righting the wrong? Many trainees are not prior law enforcement or military and are placed in a position that can greatly change the course of someone’s life based on their implicit biases. What’s your advice? 

Lois James: Oh, that’s such a, it’s such a great question. It makes me so happy to hear that because we know how massively influential the FTO is, in terms of young officers’ development. So, I think that the first major piece is kind of being aware of all of this yourself. By the sounds of it, you certainly are, if you’re looking to that mentorship role, in that positive sense. But, I mean, yeah, just kind of, going back to what I said before about it’s the why, and it’s the how. You have to start with the why and a big part of that and I know it’s challenging, I know it’s uncomfortable that is admitting that there are problems. Right? I mean, this is, we wouldn’t be spending this much money on implicit bias training, we wouldn’t be doing all this research if everything was perfect, right? I mean, there are significant problems that need addressing. So, I mean, kind of recognizing that, recognizing the police role in it, recognizing everybody’s individual role in it. And then just trying to promote that self-awareness among your people. And one thing that I always like to do, and I think from an SEO perspective, it could be fairly useful, is that where we often think about consequences only being related to intentions. Right?. If we think, well, I didn’t intend to cause any harm. I didn’t intend to cause offense. I didn’t mean it like that, right. We need to make a shift away from that, and think about the impact that we have, whether it’s intentional or not. If you don’t intend to cause harm, but you cause harm. You’re still accountable and responsible for that, right? And again, I know that that can feel super uncomfortable because you feel like you’re being blamed for something that’s not your fault. But the impact is really, really important. So that’s kind of what, I would say as a takeaway.

Renée J. Mitchell: For an FTO, I would just say, I think one way to maybe think about that is to listen to your trainees and the language they use, and you don’t have to be accusatory at all. I think you need to be gentle. And when they use language that’s, and I don’t mean like biased-biased, like you, when people use certain words, like, there are gendered words. And I don’t mean in like some woke way, I actually mean it in like this very gentle way of calling to their attention. Like, what do you mean when you say, like, she’s crazy, like certain terms that get attributed to women or certain things that get attributed to the elderly? Or certain things that get attributed to the mentally ill? I gently point that out to them, because then it will make them think for a minute, like, “Oh, why am I automatically assuming these things?” Just to make them aware of it, and you have to do it without judgment because we’re human beings. We all do it. It’s not like you’re trying to call them out to, be, like, “Oh, I’m better than you.” You’re trying to do a gentle awareness. So, and I think by doing that, you could help them view the world a little differently, hopefully, at the beginning of their career. If they become cynical and super biased, that’s on them, You did your best to try to gently move them down the path. That’s what I think I would say about that.


Click Here to Watch a Recording of Effectiveness Evaluation of Implicit Bias Training: Research Study Results



Additional Resources
7 months ago
Coach or Critic? Implement Strategies to Overcome Negativity Bias and Limiting Beliefs
Ever felt like you want to achieve something only to feel limited by your own thoughts and beliefs? […]
1 year ago
Understanding Implicit Bias: Research and Reality
Implicit bias is a hot issue that the Justice Clearinghouse has covered in past webinars. However, m […]
2 years ago
Thoughts about Bias from Michael Lieberman
Jessica Reaves and Michael Lieberman of the Anti Defamation League shared incredible insights and ad […]
3 years ago
Implicit Bias: What You Don’t Know Can Hurt You
The human mind is a truly incredible thing. How it processes information outside of our consciousnes […]