ÌÇÐÄvlog¹ÙÍø

leading experts from Harvard Kennedy School, who answer your questions on public policy and current events. Wiener Conference Calls recognize Malcolm Wiener's role in proposing and supporting this series, as well as the Wiener Center for Social Policy at Harvard Kennedy School.

 

- Hi, everybody. I am Mari Megias in the Office of Alumni Relations and Research Development at Harvard Kennedy School. And I'm so pleased to welcome you to this Wiener Conference Call, just kindly sustained by Dr. Malcolm Wiener. Today, we are joined by Iris Bohnet, who is Co-director of this school's Woman in Public Policy Program, and Albert Pratt, Professor of Business and Government. She's a behavioral economist who combines insights from economics and psychology to improve decision making in organizations and society, often, with agenda of cross-cultural perspective, her most recent research examines behavioral design to debias how we live, learn, and work. We're so fortunate that she's agreed to share her expertise today with the Kennedy School's alumni friends, Iris.

- Thank you so much, Mari, for having me and thank you to Malcolm and Caroline Wiener for sponsoring this series. I'm incredibly happy to have so many people join us today and be able to share some of the work that we are doing as Mari said, helping organizations to create more inclusive workplaces and therefore, a more diverse workforce. I'm gonna start by sharing my screen. So if you could just hold on for a second. Okay, does that work for you? I presume, Mari, are we good?

- [Mari] Yeah, that looks good, that looks good.

- Thank you so much. Thank you. Okay, great. So the title of my talk today is in fact also the title of a class that I am currently teaching on Behavioral Science for Inclusive Organizations. And as I said, my passion really is to bring more rigor and more evidence to this question of inclusion, diversity, and equity. So my passion is behavioral science. Behavioral economists by training, and of course, evidence and data. And so I thought I'm gonna start with a little image for you where I would like to invite you to compare squares A and B for me. And I presume that most of you see square B as being lighter than square A. And now I'm gonna cover the surroundings and imagine that about now, you realize that the two squares have the identical color. I'm gonna quickly go back because I presume many of you are puzzled by this illusion. I'm starting with this to suggest to you that we don't just put boxes or squares into categories where some have to belong to a dark category, some have to belong to a light category. But of course, we also do the same thing when we see people. This pattern thinking or categorical thinking, as we call it in psychology is very, very typical of human decision making, of human judgments. It often helps us make judgments logic quickly. It is part of how our minds work. Of course, it often leads us astray, and that's what you have just experienced. That's the pattern that you see a checkerboard, so to speak forced your mind to think that be just has to be another light square. And what I'm doing now is I'm literally liberating your minds and allowing your minds to adjust to square B, because I am taking the pattern away. Now, what does this teach us? This teach us, so sadly, something that is also happening in the real world. And many of you are probably familiar with of the early studies that used the audit design, where employers were sent resumes that even either had the name Lakisha Washington or Jamal Jones or Emily Walsh or Greg Baker, and where the researchers found that Emily and Greg were much, much more likely. In fact, 50% more likely to get a call back and be invited to an interview by the various employer these resumes were sent to. The resumes otherwise were completely identical. Now, of course, this is not just an American phenomenon. So here is another study that was conducted in Germany. And some of you who might have worked Germany or other German speaking countries will recall that in those countries, you still append your photograph to your application. So this is the photograph of Sandra Bauer who applied to a job. And her callback rate was 19%. Now this looks like Sandra again. And of course, it is Sandra again, who applied with the resume, the same photo. But now her name was Miriam Exchat. Miriam gets a callback rate of 14%, and then Miriam wears a head scarf. And now the callback rate drops to 4%. So clearly, and sadly, the stereotypes are still well and alive even today. And going back to the US, what you see here is that over almost 30 years now, these differences haven't actually changed much. What you see here as a one is the callback rate, the likelihood of the callback of a black person. And then you see that on average, white applicants get a callback rate of 1.5. So to 50% more likely to get a callback, you also see some variation there. And the size of these balls just tells us something about the sample size, so there's nothing magical here, but the takeaway is there really isn't much of a trend, maybe slightly improving, but it really is pretty stable. So why is that? Partly I would argue, it is because we have been focusing too much on raising awareness and not enough on giving people the tools to follow through on their intention. Society evidence on awareness raising, or you might also wanna think about diversity training per se is not particularly encouraging. In fact, we don't know of any diversity training that has been documented to work, even though they are the favorite tool of corporate America and increasingly of countries across the world to help us overcome some of these biases, some of these stereotypes. So awareness can be helpful. It might open some doors, but clearly we have to do more. And so what have we been doing? And what have you learned works? So here's just one example of what behavioral design could look like. This was an intervention that was started to be tested in the 70s, so quite a long time ago, in fact, starting with the Boston symphony orchestra, where many orchestras had musicians audition behind the curtain. Surely that's a story, and also research by Claudia Goldin of the econ department, the Cecilia Rouse now of the council of economic advisors that you are familiar with. It has been around for some time. We now have about nine replications in other contexts, in other countries. And generally anonymized resumes increased the diversity of the people who end up being hired, but not always. And you can already imagine if you are engaging in affirmative action, if I take away your ability to in fact, know the gender or the race or other characteristics of the applicant, then you can't use that diversity dimension in your evaluations, but anonymous resumes are one example of behavioral design. And also one of the reasons, in fact, I wanted to write this book to bring together the evidence on what works to move the needle. It turns out that the original curtain that then was used to buy all of our major symphony orchestras, almost all of our major symphony orchestras in this country has helped increased the fraction of women on the major symphony orchestras from about 5% in the 70s to now almost 40%, which is very different from let's say, the Vienna, the harmonic, or the Berlin harmonic, which are not using curtains or anonymized evaluation procedures at this point. So let me give you a bit of a sense of why I'm excited about behavioral design and why I think more organizations should go into their systems, into their practices and procedures rather than trying to change mindsets. Mindsets can change eventually, but not in a one hour or even a one day training program. So this is how maybe aloft the aspiration, but many of you have been in a hotel room at some point, but a room key card did not just serve the purpose of opening and closing doors, but also of turning lights on and off. And as we all can imagine, just a little bit of design, a little bit of technology makes it so much easier, even for the well-intentioned among us to actually turn off our lights and preserve energy, to actually follow through. So that's my aspiration. And of course, I can't promise that we will be there at the end of this presentation, but that's really what the Women of Public Policy Program in my own work at the Kennedy School is focusing on. I'll give you one more example before I wanna dive in a bit more into what organizations can do and specific questions of talent management. So this is more of a story, although it is directly based on our research, but in the end, it is more of a story in that in 2018, I got a call from the Nobel Prize Foundation that also had realized that about 96% of the Nobel laureates had been men. And they were wondering whether I could come to Stockholm and look at their procedures and help them think through whether they might be falling prey to some of the stereotypes that we've just looked at on the checker board, whether it's anything that we could do to make the procedures more accurate. So I went, and of course I had to sign the NDA and can't talk about most of the things that we did, but the example here I can talk about because the Nobel Prize Foundation itself wrote about it. And it's also something that I arrived with. Namely, every year I get a form that invites me to nominate a person for the Nobel Prize in economics. And my simple recommendation to the Nobel Prize Committee was to change that form. Namely, I suggest that instead of asking people like myself, to just nominate one person, ask people like me to nominate more than one person, because our research strongly suggests that if we make those bundle decisions, if we think about more than one person at a time, diversity and the beauty is accuracy is much more likely to emerge. 'Cause when you ask me about my favorite candidate or the original study actually was done with food choices, my favorite snack, I'm always going to go back to what I used to always eating, what truly is the one and favorite thing that I have in my life. But when you ask me about more than one person, I am starting to calibrate, I'm starting to think about, well, I was immediately thinking about a colleague at Harvard, of course, but now that you're forcing me to think about three people, I am starting to cast in a more widely, and I'm starting to think about people outside of Harvard. I'm starting to think about people who might not normally be in my network. And I'm also starting to feel slightly awkward if I would write down the names of three Harvard professors on this forum. So in fact, this is something that the Nobel Prize Committee introduced in 2019, and this is the new form. So it now asks people like me to make three recommendations. And there's nothing magic about the number three, by the way, our research suggests just has to be more than one person because naturally, and that's a very important insight from behavioral science, naturally, our minds and our judgments are relative or comparative. So they are very hard for us to make absolute judgements. So whether or not you like the coffee that you drank this morning, depending on where you are in the world, has something to do with the coffees that you are used to, the temperature in your room has something to do with the temperatures that you used. So in that sense, these comparative evaluations do help us make better judgements without having to rely on the state of time. Okay, so that's just a warmup that I hope I picked your interest in behavioral science in using it also in our talent management. And while I've worked quite a bit on the hiring stage at the entry level, I thought today, I'm gonna spend more time on career advancement because as we all know, that is really where the rubber meets the road, where we have much more diversity at the entry level. This is my advanced statistics here, but in much less at the top. But if you're interested in any of that, we have written a number of HBR articles, which really serve the purpose of guiding employers into how to, for example, use interviews in a more structured way, in a more debiased way. I've also had the pleasure of working with a number of startups. So not all companies here are startups, but many of them are. Some of them from Harvard Kennedy School graduates. And that's actually been one of my particular passions, in that I have been thinking hard about how to leverage the insights that are created, in our little ivory tower here and make them accessible to the world. And of course, one answer has to be technology. And many of these organizations that you have here have taken all of them, really have taken our insights and have turned them into, for example, a tech platform that helps hire candidates in a more unbiased way. So Applied is one of them that is founded by a Kennedy School graduate, Cape Glazebrook. Pymetricc was founded by, MIT graduates focused more on AI. And there are a number more here, so this is not to advertise any particular platform, but just to give you a bit of a sense of how academics like myself are increasingly also asking the question of how we can make sure our insights and fact reach the practice, the decision makers who need to know about this work. But of course, in addition to writing articles and working with tech platforms, I also do specific research with individual organizations. In fact, that is one of my particular pleasures to work with organizations and help them, from of the beginning to the end, from the recruitment phase, from the sourcing phase to the promotions state, to debias the procedures. And during the Q and A, I'm very happy to talk about other types of work that we have done, but I thought I'm gonna zero in some work that we have done on performance appraisals. And a particular passionate about performance appraisals, because much research before us has shown, that of course they are important in informing pay. They are important in informing growth opportunities, development opportunities, and they are important in informing promotion. At the same time in most jobs, typically these are subjective evaluations. In fact, they're often called subjective performance appraisals because performance is very rarely. We very rarely are able to measure performance in an objective way. And so that's a financial services company headquartered in the United States, a global company that was concerned about their performance appraisals as well, and gave us access to their performance ratings. And I was particularly interested in this company, not only because it had actually been on a journey towards more gender equity and then increasingly also towards more racial equity, but also because they used a scheme, a design feature that many, many organizations employ, and that I already, when I wrote the book in 2016, had been concerned about, and that is the following. So many organizations ask their employees to first self evaluate themselves and then share their self evaluations with their managers who then make up their minds. Now, a little bit of behavioral science who even just common sense would raise a red flag for all of us in that of course, my self evaluations might anchor. My manager might influence my manager's evaluation. And if there are differences in our ability or even cultural proclivity to assess ourselves highly or shine the light on ourselves, we will see those differences potentially affect the manager. And we did find differences in self evaluations in other work beforehand, including cross-culturally that there are certainly some cultures which are much more used to shining the light in themselves. And there are other cultures where this is inappropriate to do. So this particular company used this design feature, but then had a glitch in one year and could not share self evaluations. And that of course, for researcher is Mecca, in that hoping that we could in fact, really draw causal inferences from something that looks like a natural experiment. So here's what we did. So first of all, I'm just gonna show you the rating distributions. And many of you probably are in firms, which use very similar types of design features in that, in this company, the scale is from one to five, with not one meaning that you need improvement and five meaning that you have significantly outperformed the rest of your employees. Now, first I'm gonna show you manager ratings. They're pretty normally distributed, probably also not a big surprise to anyone. And then generally, we see something in this company that we generally find, and that is that individuals, employees are slightly more optimistic about their performance than their managers. So these are just average data where you see red being more likely to be happening for the four and the fives. So now the real question is, what does this mean for our distributions across gender and across race? And what I'm going to show you now, our aggregated data for men and women and for white employees and for employees of color. I'm doing this because I'm working with the global data set here where we couldn't disaggregate race, because race is not generalizable across the world. But in the paper we have two sections, one with the global data and one with the west data. So what you will see here on the left are male employees, on your right are female employees, and then at the bottom, you see that, on the left, we have white employees, and then on the right, we have people of color. And you will generally see that managers, as you just saw from my introductory graph will decrease employees self-evaluations. So here we have male employees. There's not much going on in terms of race, but I already, when you see women's self-evaluations, you can see that on average women give themselves lower self evaluations, that is controlling for everything that we know about these employees. So these are not just means, these are the coefficients in our regression analysis. And so controlling for seniority, controlling for the type of job they have, for how long they have been working in the company, et cetera. And in addition to a general gender difference, you also see an additional intersectional difference where in particular, women of color gave themselves lower self rates. So what do managers do with this? So now we have managers in blue again, and what you see is that everyone is being taken down, but white women in particular, and even women of color to some degree are taken down less. Having said this, the anchoring effect is most pronounced for women of color, which started out with this gap in their self evaluations. And then even though managers closed that gap just a little bit, they didn't do enough to close it completely. So women of color end up with the lowest self evaluations. Now, the big question is, does not sharing in these self evaluations do anything? And it turns out it helps. It helps in particular women of color in their first year of employment. So women of color who have no history of the firm in their first year do better than when self evaluations are shared. In fact, the race gap for women completely disappears for this cohort of women of color in their first year. So not sharing self evaluations certainly has some benefits and the company in fact decided to stop sharing self evaluations. But this graph also tells us we have additional issues to worry about, and the company does in fact, do three things. One is not sharing self evaluations, the second one is calibration meetings after managers have submitted their performance appraisals to make sure that they didn't inadvertently have gender or race dynamics going on. And then thirdly, they also try to do a better job even preventing these differences from occurring early on in that they now have many more check-ins with managers beforehand. Okay, so let me summarize, maybe on this one for just a second and pause, and say that the tools that I have described to you so far, whether that is anonymized resumes or whether this is removing self-evaluations from being shared with managers, have in many ways, two important characteristics, and I'm using an acronym that was coined by the UK's behavioral insights team EAST, namely a good tool makes it easy for managers to follow who on their intentions. So often that involves some technology, some different form where we just don't burden the decision makers as in denominators in the mobile pricing sample to do more work. So it has to be easy and it has to be timely. That's a very important message, for example, for training programs, that it just is not very sensical to have a training program in January for something that then might happen in the fall, such as performance appraisals. So we have to think very hard about how hard it is and when we intervene, depending on basically what managers have to do tomorrow. So it has to be pretty short distance from an intervention or from a training to the actual behavior. In the middle, you have the A and the S of EAST, and they stand for attractive and social. And in fact, I wanna give you a bit more of a sense of what I mean with those. So even though I now have hopefully raised your awareness and I have given you some tools that you might be able to use and to debias your organizations, in this image here, I'm suggesting that even though people might realize that the left beach is dirty and might understand dirt, and I might have given them a tool such as a shovel to clean up, they might not want to clean up, right? That's a different problem. So motivation is a different problem that we haven't tackled yet. So in addition to awareness and tools, we also have to think about motivation. And this work has actually been stimulated by other work that is not an intervention, that has found that fathers of daughters care much more about gender equity than fathers of sons. Now, again, this is not an intervention that I'm trying to test, but it gives us some hints at what might matter for particular intrinsic motivation to work. And that is that long history of research suggests that yes, families of course care for each other, but there's also affinity that we experience of all different groups. This could be the fans in a Iraq concert or in a sports event. And they have the umbrellas here because research shows that we even can create pretty randomly groups. It's called the minimal group paradigm where people associate with the red umbrellas rather than the green umbrellas, pretty randomly, because they've been placed on their red umbrella. And of course, we feel, and I hope you feel some affinity to the universities that you either have gone to or are working with. And that is a bit the inspiration for work on affinity across races or across genders, across nationalities, across other dimensions of difference. It's called the Intergroup Contact theory that suggests that if exposure to people who are different from ourselves, deep exposure changes our motivation to care for these people. And so new research now suggests that mentors and sponsors experience some of that. So this is now not looking at the mentee, whether they derive any benefits from the mentorship, but on those of us who sponsor some employees or some of my students that people like me, of course, working with a student from Saudi Arabia, for example, is a little example right now, start to better understand, and to empathize where they come from and develop an intrinsic motivation to actually care for them and try to advise the world for them. So that my first appeal, I have two appeals for you today, become a micro sponsor. Micro sponsors are people who play the role of a sponsor, not just in the grand sense of the work where I literally pick up a phone and make a call for somebody, but also in meetings in informal places where somebody might not have been given credit for a comment, might have been interrupted for people like us, everyone on this call can in fact step in. And for example, bring the discussion back to Jamila and honor John for having repeated the comment, but thank John for building on Jamila and bring it back to her. My second appeal to you is to become a norm entrepreneur. And this is actually something that Cass Sunstein who some of you might know, one of the co-authors of the book, "Nudge," a term that he has coined, in that, we create norms and we enforce norms, and we nudge people towards norms in many different instances, often unbeknownst to us. And so if we could become a bit more intentional about our norm entrepreneurship, of course, we could have real impact. And here you have an example of the Kennedy School that I think all of you probably now know about, those of you have been involved in our school for longer, that it is now about 18 years ago, that of the 50 portraits of leaders exactly zero were of male leaders. And in the meantime, we have commissioned many, many more portraits of women, of people of color, of people from outside of the US. Because as you know, 50% of our students are women, almost 50% are from outside of this country. And inadvertently, we have signal to these students that they were not made to be leaders. What you see here or who you see here is Jane Mansbridge former Professor of the Kennedy School at Meritan, and who noticed who was the one to notice that we had no diversity on our walls. And then Ellen Johnson, certainly the Former President of Liberia, the graduate of the school and who's portrayed out is on our walls as are many others. Now seeing is believing. And that's of course not just the portraits in our walls, but also who we see on our screens, whether that's on social media or on television. And this is just an interesting example of one person was Atkins on your left, who is the host of a BBC show who realized literally one warning driving to work, that he did not have many female experts on his talk show. And he launched a 50/50 project just for his talk show. So it was a total bottom up approach and it was soft accountability. In that sense, nobody held him accountable. He was actually private about this initially. Excuse me, he mobilized his team to work together. It then spread across the whole BBC, it then spread across the world. We now have about 200 organizations that are part of the 50/50 project, not just media organizations, also some companies, including Unilever for example, was one of the very early signers. And it's now a case study that I teach in my course on behavioral science for inclusive organizations to help my students see how individuals can literally be norm entrepreneurs, how they can figure out how to create those types of goals, how they can figure out how to track, how they can start sharing the data, how they can start celebrating success, how they can bring others along. So in fact, everything they did was totally building on the spreading of the norm on positive reinforcement, not so much on naming and shaming. Now in contrast, this is work that I have been involved in, where we use naming and shaming. In addition to other means, this is Vince Cable, former Secretary of Business of the UK. And this is work where I got a call in 2013, about what many of you are probably familiar with, the attempt to increase gender diversity on the Fortune 100 companies in the UK to 25%. The initiative was launched in 2011 with about 12% women on boards. And by 2017, 2013, excuse me, we had about 17% women on corporate boards, and that's what we see here. I got the call because at that point, the progress had sold for a couple of months. And the question was whether behavioral scientists like myself might help move the needle because very consciously, the UK actually did not introduce quota. So this wasn't a quota, but this was a soft goal and aspiration, but at the same time, public aspiration. So they definitely were very, very intrigued to want to reach it. And there was a whole coalition of players involved in this. It wasn't just the government, it was a private sector, it was senior executive search firms, it was other academics, the media. And we're also teaching a case on this that I wrote to help our students understand how something like this on a bigger scale can come together. But I just wanted to show you one example of the types of things that this coalition introduced. So when I got the call in 2013, they also shared the cover or the brochure with me that they had produced in that year. And that's literally what we see in front of you. And this was the cover. And I was concerned about the cover because there is quite a bit of research that describing how the world is, can turn into prescriptions of what the world should look like in our minds. So very, very often, we lament the absence of women or the absence of people of color, or the absence of people with disability in our organizations. But what I'm suggesting to you is that this is not how we start a social movement, 'cause that could actually create prescriptions or norms of what the world should look like. So one thing we did was we redesigned the cover, and the cover then looked like this. What you see here are, again, the same sample, Fortune 100 companies. In fact, the little boxes there with the male and the female icon represent company each. So we have 100 companies on the cover and 94 out of 194% at the time were already gender diverse. And that meant they had at least one or more women on their boards. What I'm now suggesting, right? Is that the train has left the station, that we are moving towards more gender equity. And if you are part of the club, you really are an outsider. The whole communication of course, strategy changed it off. That it wasn't just the cover of the brochure, but I'm just showing you this as another example of how we can use norms to shape behaviors. So this is my third conclusion. So in addition to raising awareness, and in addition to providing tools to people to follow through on their virtue intentions, we also have to work on motivation. And very often, we focus on extrinsic motivators, and there's nothing wrong with that. I'm economist by training. I absolutely believe in KPIs, I believe in incentives, I am encouraged by relatively newish movement on ESGs and how that also brought the S to the forum. How many companies now, because of shareholder pressures do disclose the S, the diversity in their organizations in many parts of the world, just gender in the US increasingly also race, and transparency, accountability, that's all important, but I'm suggesting to you that two other motivators of behavior might often be overlooked. And this is of course, the social motivator, the S in EAST. So think about the norms that you help craft in your environments and think about how you might be able to literally redecorate the rooms in your organizations. And then lastly, a hard one of course is intrinsic motivation, not easy for us to affect, but that's why I wanted to leave you with this metaphor, so to speak of fathers of fathers, that if we can create affinity between groups in a deep sense, not in a superficial sense, people are much more likely to be able to overcome difference and develop care for each other. Actually, one of the best experiments, interestingly enough, was done in dorms where many universities, including Harvard use a random mechanism to assign first year students, freshmen, two dorm rooms, and where the researchers exploited that to see whether being to somebody who looks very different from you, in fact, changes your attitudes and your behaviors towards people of difference. And that evidence is actually very, very encouraging. Okay, so wrapping up here, I'm leaving you with a short acronym that is normally used for something else, and that is the ATM of DEI, that yes, you have to raise awareness. We have to give people tools, and we have to motivate people to follow through. My last slides are just some references. If you are interested in learning more about anything that I have described today. So I've served as an advisor to the Behavioral Insights Team, the Government Equality Office in the UK for quite some time now. And we have created guidelines for employers, private sector, public sector, NGOs, to use these types of insights in their HR. So go to that webpage in case that interests you. I also have worked with a large group of academics, DEI experts, and tech companies for the last year. The Act Report, this is action to catalyze tech, where we, again take behavioral science very seriously and tried to make it EAST compatible. What we advise companies in the tech sector in particular to do. So then also openly available report. Then almost finally, if you're particularly interested in gender quality, I served on the G7 Gender Quality Advisory Council last year. We also came out with a report that focused, particularly on improving the state of women and girls across the G7 countries was quite influenced of course, by COVID-19. And the particular experiences that we have made in the gender dimension, for example, on care, but also sadly on domestic violence during COVID-19, and thankfully built on what works. And then now finally, another resource that I wanted to leave you with, again, if you're interested in particular and gender at the Women and Public Policy Program created an open resource where we translate research for practitioners, decision makers across the sectors on how to close gender gaps, not just at work or in work, in the economy, but also in politics, in education, and in health. So please take a look, if that interests you, it's the passion of mine to really build this gap between those of us who create those insights and those of you who want to use those insights. And so with that, thank you so much, and I wish you good luck designing change, and I look forward to your comments and your questions.

- [Mari] Great, thank you so much, Iris. We're going to open the floor up for questions now, to ask a question, please use a virtual hand raising feature, zoom and a true Kennedy School fashion. Keep your question brief and end it with a question mark. You'll be notified when it's your turn to speak and just make sure to unmute yourself and hear from our staff. So I'm gonna start things off just by asking a question in terms of specific types of learning and development. What types are useful for both, not only raising awareness, but also changing behaviors that are due to unconscious bias.

- So, Mari, is that question about trainings, about switch trainings?

- [Mari] Yeah, yes.

- So, yes, that's another interesting story really. I was approached last summer by the White House about President Biden's executive order on DEIA, and A stands for accessibility, that has lots of features, but also has the feature of providing diversity training to the federal workforce. And as you heard me describe today, sadly, we haven't really figured out, we in women, when I say we, I'm saying the world, hasn't really figured out how to train people effectively. So now a group of researchers and myself are working with the federal government, inspired by the potential of testing interventions with of course, a huge, huge number of employees and our main ideas at this point, are that again, very influenced by behavioral science is that trainings have to be much more targeted. So generic diversity training, as I said in January, where we hope that that might then affect our decision making on hiring or on promotions in the fall is very unlikely to have any effect. So we're thinking much more targeted. We're thinking it has to be targeted timely in terms of when do people hire, who hires, when do people make those promotion decisions, who makes those promotion decisions? It has to be targeted on people, on time, and then thirdly, also on the task, right? So these are generic interventions, but very specific. And we can also think of them, for example, they don't just have to be formal targeting formal procedures, but could also be informal. So we're doing some work now on meetings. Can we make meetings more inclusive by providing relevant training or information just shortly before the meeting takes place? And that could be both structural, could be reminders of desired behaviors, could be tips of how to overcome microaggressions. So that's where our thinking is going. We're still testing. So I can't tell you that we have found the proverbial silver bullet here, but that's where our best science suggests people find most impact.

- [Mari] Got it, thanks a lot. We have Sheryl WuDunn, up next. If you can ask your question.

- Hi, Sheryl.

- [Sheryl] Oh, hi there. It's Sheryl WuDunn. Actually, I just posted my question in the chat, but I will summarize it basically in the footsy case, which is really a very creative and interesting the way you solve that problem, that issue there, but more generally, what has been the impact of getting more women on boards? First of all, is it an appropriate question to ask? 'Cause you could always argue, well, why should we have to prove that it's good to have women on boards, but on the other hand, you would think that there should be a difference in the way companies are governed if you have more women on boards, because in the past we've argued that diversity is good and it should help. What is the result of all that?

- Yeah, that's a super good question, Sheryl, on the business case, and I'm actually gonna start with your first question because I think you and I are complete aligned on that answer in that it cannot be the only, or even the major motivation for us to care about gender equity, can't be the business case. So women don't have to prove to be better than men in order to be allowed into the game. And that's an incredibly important message that I repeat all the time. It's primarily, and of course your work also Sheryl has brought this home. It's primarily a human rights case. So that's be said, but secondly, of course I'm also realized, and I understand that working with private companies, the bottom line is important. And so many asked a question of the business case, and I feel it's a place where a lot of bad research has led to quite broad generalizations that I wanna maybe disentangle a little bit. So one is, is there a correlation between gender diversity on corporate boards and company performance? Yes, yes, that's a correlation. It's not super strong, but there is a positive correlation that now many studies have found looking at many different samples across the world. But having said this, correlation is not causation. So we have no way of knowing whether it is the women who made a difference or whether it is even the diversity on the board that made the difference, that could also be the man who are now behaving differently. None of that we know because it could be just something, a third factor. And that is that companies which generally have good governance, which do many things direct, might also have more effective boards. So I think sadly, it's a hard question to respond to. The best evidence that we do have comes from Norway because Norway, as you, of course know, and many people on this call will know, was one of the early countries to introduce quotas for corporate boards between 2006 and 2009, or no, wrong, between 2003 and 2006 just before the financial crisis. So companies had to have at least 40% of any single sex on their corporate reports. And so the question is, what difference does that make? It's not a perfect experiment, but it is a shock to the system. And I think that there's two takeaways, one is there were in fact companies which fled to the UK, which unincorporated in Norway because they thought that was the end of the world. So stereotypes are definitely well and alive. Did Norwegian's companies break down? No, not at all. Did the women have a huge impact? Not so much either. So, what we do see is that the women on the Norwegian corporate boards tended to be younger, tended to be better educated, and tended to have less executive experience than their male counterparts. So these were different people. And the second thing of course, that was true, and that's why I remembered the exact exact years, this law went into effect just before the financial crisis. So during the financial crisis and much of this research in fact started to take place. Many companies mode of survival was restructuring, which means laying off people, and that research on Norway suggested that women were slightly less willing to do that. But that's really the biggest finding at this point. So I think the main takeaway is adding women has not hurt the world, has not hurt these corporations. There is a small, positive correlation between company performance and gender diversity in corporate boards, but we don't have any causal evidence suggesting that magically these new women just make things better.

- [Mari] Thanks very much for that question and answer, just a reminder, if you have a question, please pop it in the chat or raise your hand using the zoom hand raising feature. So just wondering with many organizations advertising their diversity statements on their website and in job listings. And I'm wondering if these diversity statements affect representation in anyway, if there been any research on that, do they increase the likelihood that underrepresented groups will apply for jobs?

- Yes, that is a very, very good question. And sadly it's not very encouraging. So I fear talk is cheap and many applicants know that talk is pretty cheap. There is certainly evidence that some information on them job postings does matter, but that might be slightly different information than what you intuitively think about. So for example, there is very good research suggesting that whether or not it is made transparent, that pay is negotiable, is more likely to attract women to apply. It's quite a bit of research on gender differences in negotiations and women tend to be less likely to ask and tend to be less likely therefore to end up with the bigger salary, and Hannah Riley Bowles' research, who is the Co-Director of our Center for Public Leadership and also the Women in Public Policy Program in our faculty has actually shown that one way to overcome that gap is more transparency or a decrease in ambiguity on what's negotiable. So what you can actually ask for, and other researchers then have gone into the field and have tested that in thousands and thousands of job ads, again, similar to the audit study, varying whether an applicant saw an advertisement with negotiability explicitly mentioned or not, and that actually increase the likelihood that women would apply. So I think we have to go a bit deeper again into the psychology in our understanding of what works for different groups and use those insights in informing our job ads. So don't take my reluctance in jumping on diversity statements are suggesting that what we send job advertisements doesn't matter, it matters heavily. And it also matters for example, in terms of the language that we use. There are very good tools out there which help us, for example, debias, the language in our job advertisements to make the language more inclusive of everyone independent of their backgrounds.

- [Mari] Great, we had a question in the chat from Kim Norton and she says, we are measuring the gender and racial memberships on our city boards and commissions. Given your comments about norms and messaging, how would you suggest this be shared with the public, or has your point in the service talk?

- Okay, and that's a really good question, right? So I think there are two questions here. One is, is transparency good or bad? And so I think, yes, transparency is generally good, right? But it is an accountability mechanism for your constituents to see whether you're falling through on your intentions, whether you're generally moving in the right direction. But secondly, the question is whether you can add some framing, right? Whether you, for example, could talk about the increases, the changes in your numbers to suggest to the public, but also internally the direction of travel that you would like to take. So that's the question. Can you use some clever, I'm gonna call it clever or really behavioral science in infused framing of the information. I often give as example, something that probably all of you're familiar with, and that is the food pyramid that was used in the United States to inform people of healthy eating, so you probably all vaguely called food pyramid, least wasn't pyramid. And then it was changed into a plate where now, first of all, not deep, inside here, we don't eat of pyramids, we eat of plates. And secondly, where we have these different divisions of how many carbs, how many vegs, how many fruit, how many proteins should you eat is much more intuitive because that's normally what we look at. So that's what I'm trying to think about. Yes, share that information, but think hard about how you share it so that the message that you wanna communicate actually comes across.

- [Mari] Great.

- And I think we have one more question, I see Caroline, I'm reading it right now.

- [Mari] Go ahead. The question is about salary transparency and job post date. And increasingly, does that increase the likelihood that women and people of color will apply for jobs?

- Yes, so that's a deep question in a sense, Caroline, that a transparency is not created equally. That's why I'm a little hesitant. So we spend a whole hour in my class disentangling the different models that are out there. So generally, so can we start to general? Generally transparency is a good thing. Generally decreasing ambiguity increased the likelihood that women and people of color apply. Having said this, there are different models of transparency in the United States, for example, already, but also then across nationally. So there is a very precise transparency, that for example, the University of California is using where people like myself when I was Academic Dean can look up every faculty member's salary, and that has research suggests. So that kind of specificity has in fact created some unrest where literally people are comparing themselves of others and know what my colleague is making. And that might make me think that I'm much more productive and micro famous and whatever have more citation than that other person. So me as human beings definitely are inequity averse. So that specificity, I think, comes with some concerns. Now let me give you just a very different example. The UK has chosen a different model. Some states in the US have chosen a different model where we just present aggregate salary. So salary bans, and the evidence in that is actually quite encouraging that it both leads to more diverse applicants to applying, but also in fact, decreases pay gaps. And so, I do think generally transparency is a good thing, but the devil is in the detail, and we probably have about 10 studies right now. Canada has done some interesting work on that transparency as well. That is still trying to fine tune how exactly we wanna disclose salaries. I'll give you one last example, and I know we're almost out of time, so I'll give you this one last one, 'cause I know a bit more about it. So in the UK, due to my advisory role, probably those of you are from the UK or have work in multinational companies who know that companies before COVID with more than 250 employees had to report their gender differences in pay. And I can tell you that was an endless discussion of how that disclosure, what that disclosure should look like. Of course, the economists in me wanted it to be very precise as a no, no, no, we can't do just what an average man make and what an average woman make. That's just not precise yet need to control for educational backgrounds and seniority and experience and occupation and what we normally do, but it turned out the government chose something very different, something very simple and said, no, no, no that is just too complex. So going with EAST, too hard, let's just do on average, what men make in your company, what women make. And that had in very interesting behavioral consequence in that many, many companies then felt compelled to do their own gender pay gap analysis and disclose that too. And saying, we actually looked at while we have a 40% gap, and it is due to differences in occupation. It is due to difference in seniority, et cetera, et cetera. And I actually take that as a very positive outcome that companies started to pay attention and also started to realize that if we don't close gender gaps, for example, in promotions or in performance appraisals, we won't close this raw gender gap and just comparing the averages because so much is in fact motivated by differences in seniority.

- [Mari] Great, well, thank you very much to everyone who called in to listen to this Wiener Conference Call. Special thanks to you, Professor Iris Bohnet. Our next Wiener Conference Call will be held May 11th, featuring Sheila Jasanoff on the impact that science and technology have on society. Thanks very much, and have a great rest of the day.

- Thank you so much for having me.