Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

Archive for the ‘Program Evaluation’ Category

Survey Comments Highlight What Tenants, Landlords Think About Mediation Program

Jasmine Henry, September 16th, 2024

Since 2020, Resolution Systems Institute has administered the Kane County (Illinois) Eviction Mediation Program. Our team works with the 16th Judicial Circuit Court and community services to mitigate the negative effects of eviction filings. We conduct intake, schedule mediations, provide guidance for parties going through eviction mediation, manage the mediator roster, and provide assistance to the mediators as needed. After parties participate in mediation, we ask them to complete a survey about their experience, including answering multiple open-response questions.

Landlord and tenant comments on the program and their mediations illuminate what they value and what falls short for them. We have seen how the process makes parties feel respected and listened to, and what that means for them. We have also learned of issues that arise with power imbalances and with mediators who repeatedly facilitate cases involving the same attorney.

Mediation as a ‘Life Preserver’

Comments from the past year make clear that many parties have been profoundly affected by their experience with our program and their mediation. This is perhaps best exemplified by this comment from a tenant: “Everyone has their own reasons for difficulties. I got a chance to say how this happened and also how I can prevent it from happening in the future. I’m thankful for everyone who has helped me through this and being treated fairly and with respect. This program is a life preserver to a drowning person. Thank you.”

Another tenant shared a similar sentiment, saying, “When people go through a difficult time, this program gives them a chance.”

Mediator Role is Key

As with so many mediation programs, mediators are central to parties’ positive experience in the Kane County program, survey responses show. In fact, the presence of a knowledgeable, impartial mediator is one of the most appreciated aspects of the program. One participant noted that the mediator “listened to both sides of the story.” Another commented: “I really appreciated . . . the way the mediator helped to point out the facts of the situation and allow both parties to express themselves. As a tenant without a lawyer who’s been through a major financial crisis, it gave me the opportunity to be listened to and to get my head above water.”

Many survey respondents praised their mediators’ ability to help parties find compromise. As one tenant put it, “They really try to meet in the middle and make everyone happy in the end. It gives so much clarity and security.” Another tenant succinctly stated, “It gave a compromise to a no win situation.”

Participants noted that both the skills of the mediators and the structured nature of the mediation process ensure that all parties have a chance to contribute to the conversation. A tenant shared: “Both sides were given the opportunity to speak uninterrupted. Everything was explained clearly and the mediator was very pleasant to work with. I came in feeling anxious not knowing what to expect but was pleasantly surprised by the whole experience and outcome.” A landlord shared that it was “good to talk to the other party in the presence of some responsible people.”

“The meeting was not at all stressful once the call was started. Each party received ample time to discuss any offer, explanations of situation, possible resolutions to ensure both parties benefited.”

Tenants in particular mentioned the importance of feeling listened to. When asked what they liked about the mediation, tenants described mediators who “listened to my doubts and questions,” were “helpful and caring and want to hear what you have to say,” or gave all parties “the opportunity to discuss their thoughts and concerns.” A landlord added that their mediator “listened and asked good questions.”

Additionally, mediators were often commended for their patience and thoroughness, as exemplified by this comment: “The [mediator] was very respectful and kind and listened to all my doubts and questions. The [mediator] helped a lot and explained everything in very good detail.” Another participant described their experience this way: “The meeting was not at all stressful once the call was started. Each party received ample time to discuss any offer, explanations of situation, possible resolutions to ensure both parties benefited.”

These comments underscore the program’s success in creating an environment where both tenants and landlords feel they can speak openly, are listened to and are treated equitably.

Resources for Vulnerable Individuals

Several survey responses highlighted the program’s value for vulnerable individuals. For example, a number of comments mentioned the importance of the resources beyond mediation that the program can connect parties with. One tenant wrote that the program “was there to help me [with court paperwork] when they really didn’t have to.” Multiple commenters emphasized that they had applied to the court-based rental assistance program — which the mediation program directed many parties to, and which parties were able to access until funds ran out in June 2024. When asked if they would recommend eviction mediation to a friend, a tenant shared that they would (even though they didn’t settle their case in mediation) because “sometimes tenants are unaware of the resources available due to lack of communication or shame.”

Others expressed more general appreciation for being able to participate in the program as someone in a challenging situation: “Just every single mother that actually works and tries 100% for her and children deserves this opportunity.” Another shared: “I think it is an excellent program for families who are going through difficulties, personally I felt supported and confident in the ability of this incredible program to help.”

Perceptions of Mediator as Biased

Some survey respondents indicated that their mediators worked to minimize the usual power imbalance between tenants and landlords. For example, one commented: “My mediator was a rockstar ! The attorney tried to bully us and was interrupting them but they kept their calm and brought up my rights.”

However, some tenants expressed concerns about potential bias in the mediation process, with one tenant stating, “They are there to mostly help the landlord … It doesn’t help the tenant. At all … The mediator is on the landlord side to help them evicted you.”

“I felt because the lawyer and the mediator were familiar with one another they may have been more partial to the landlord.”

Similarly, a tenant shared, “Not only was I not informed what mediation was going to be like, but also the mediator didn’t take everything I said and just went with what the landlord [said].” And one tenant went so far as to say, “I feel like I was gaslit by the Lawyer, and [the lawyer], as well as the mediator didn’t want to hear my side.”

Some comments about pro-landlord bias reflected a tenant’s understanding that the mediator already knew or had worked with the landlord’s attorney in the past, with one tenant writing: “I felt because the lawyer and the mediator were familiar with one another they may have been more partial to the landlord.”

Another explained: “[I]t’s like the mediator and the landlord and the landlord Lawyer have some kind of friendship already[,] so that kind of singles you out when they’re cracking jokes and laughing with each other[. A]nd then [the mediator is] only hearing [the landlord’s] side and telling you [that] you have to take your side to court.”

Other criticisms related to feeling unduly pressured to reach an agreement. One tenant explained how they felt both unheard and pressured: “I feel as if, regardless what was disclosed, my landlord and his attorney pushed too hard at a hard no to mediation, not giving me a chance, and the mediator seemed to take what I had to say about the situation with a grain of salt.” Another tenant stated, “My concerns were not fully addressed, felt bullied into settling.” One expressed their frustration, saying: “The mediator didn’t listen to my needs and disregarded any of my opinions[. The mediator] listened to every word the lawyer said and also wrote anything they wanted word for word.”

How RSI Uses Feedback

RSI uses party responses to our post-mediation surveys both to evaluate and improve our program, as well as to tell the full story of the eviction mediation process. These firsthand accounts allow us to go beyond statistics to better understand how eviction mediation programs affect real people in our communities. Furthermore, these comments provide invaluable evidence of the program’s tangible benefits. By thoughtfully analyzing and presenting these comments, we strengthen our partnerships, justify program funding and, ultimately, improve our ability to effectively serve the Kane County community.

RSI takes party criticisms of our mediators and program very seriously. Every quarter, we evaluate the participant post-mediation survey responses to determine what is or is not working. Every month, we hold meetings with our program staff to review our findings. We also use our research to train mediators on how to cultivate better party experiences. When necessary, we provide further training and support to mediators with recurring issues. Occasionally, if feedback was provided but no improvement made, we remove underperforming mediators from our roster.

Negative party perceptions, even if not reflective of actual bias, indicate a need to ensure and communicate impartiality more effectively. While it can be demoralizing to read party comments complaining that a mediator was biased, these comments help us to understand the impact that such perceptions have on party experience. Mediation is meant to be a level playing field for all parties — landlords and tenants, unrepresented parties and those with attorneys.

How Can Community Mediation Centers Successfully Diversify Their Rosters?

Stephen Sullivan, August 7th, 2024

Community mediation centers have long recognized that parties feel seen and benefit from working with mediators who are from diverse backgrounds. Despite this, they have found it difficult to develop mediator rosters that reflect the communities they serve. The Center for Conflict Resolution (CCR) in Chicago is undertaking an innovative project to address this issue. The staff of CCR has engaged RSI to develop a guide to support other community mediation center staff in their efforts to increase equity in their mediator rosters, and to assess their implementation outcomes. Funding for this project has been generously provided by the American Arbitration Association-International Centre for Dispute Resolution Foundation (AAA-ICDR) Foundation. 

Ongoing DEI Assessment 

The Center for Conflict Resolution in Chicago has engaged RSI to create a guide to help community mediation centers diversify their mediator rosters.
RDNE Stock Project via Pexels

Over the past year, CCR has been working with an external diversity, equity and inclusion (DEI) partner to audit its mediator mentorship program (MMP). The MMP is an intensive, three-month training program that prepares participants to meet CCR’s performance-based evaluation standard. Participants then provide a minimum 18 months of mediation services through CCR’s programs. CCR’s primary goals are to increase the demographic diversity and inclusivity of its recruitment, selection, training and retention processes. Meeting these goals has required a comprehensive and holistic retooling of the program. For example, CCR staff are experimenting with new models for meeting program requirements, establishing new communication practices, and creating ways to accommodate different types of mediation skill sets. Measuring the success of these changes is critical; CCR is also developing new survey instruments and tools to determine impact.

RSI is assessing CCR’s progress in meeting its DEI goals, to document lessons learned and share what CCR staff would recommend to other organizations.  

Project Outcomes

RSI will communicate findings from our assessment of CCR’s efforts in two main ways: 1) a guide for community mediation centers, and 2) an evaluation report.

Through interviews with key CCR staff, review of audit documents, and analysis of demographic data, RSI will create a reference guide for community mediation centers to learn from CCR’s approach. The guide will include instructions and templates for engagement techniques, methods for measuring demographic data, and forms for screening and interviewing. Along with these materials, we will discuss how CCR staff members implemented changes and addressed challenges.

Our collaboration will also culminate in a final evaluation report, which will focus on insights from CCR’s new survey instruments. RSI will analyze survey data to determine which areas of the retooled program saw the most success and which areas require further refinement. Both the guide and the report will be disseminated widely, through CCR’s website, RSI’s website, the National Association for Community Mediation’s virtual library, conference presentations and social media. Ultimately, our goals are to understand existing exclusionary practices or biases within mediator programs and break down barriers to diversity, equity and inclusion in mediation practice.

We plan to share the guide for community mediation centers and our CCR evaluation in mid-2025. Follow RSI’s blog, newsletter and social media for the latest updates.

Most Give High Ratings for Mediator Fairness, Trust in Mediator in Recent Surveys of RSI’s Kane County Eviction Mediation Program

Jasmine Henry, January 10th, 2024

RSI administers an eviction mediation program in Kane County, Illinois. Every quarter, we provide a report to the court on the participants’ experience in mediation based on their responses to a post-mediation survey.

Between July 1, 2023, and September 30, 2023, 174 eviction mediations were held in the 16th Judicial Circuit of Illinois (Kane County). After every mediation, participants were invited via email or text to complete an online survey about their experience; not all of the participants completed surveys. In our latest survey report, we examined participant responses from those three months. Specifically, we focused on participant opinions regarding fairness, trust and satisfaction. In all, 21 tenants, one landlord and 11 attorneys responded. The participants responded to the questions according to a seven-point scale, which we consolidated into three categories: low (1–2), medium (3–5), and high (6–7). Participants were invited to add comments to some of their responses. Their responses are summarized below.

Trust in Mediator, Perceived Fairness

We asked respondents about their perception of the mediator. Specifically, we asked: “How fairly did the mediator treat you?” And, “How much did you trust the mediator?” Almost two-thirds of participants gave high ratings for mediator fairness and trust. However, respondents tended to rate mediator fairness higher than mediator trust. For example, fewer than 3% of respondents thought the mediator did not treat them fairly, while 15% of respondents had low trust in the mediator. There was a parallel, albeit smaller, difference observed in the positive ratings, with 63% of respondents rating the mediator as very fair, compared with 58% who had high trust in the mediator.

Turning more broadly to respondents’ perception of the mediation process as a whole, we asked: “Overall, how fair was the mediation process?” Most of the participants who responded felt that the mediation was fair overall, with 62% saying it was highly fair. Notably, this is very similar to the percentage of respondents who said the mediator was highly fair. Not all of the respondents were impressed with the process, and 10% of respondents rated the mediation a little fair or not at all fair.

Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.”

Comments of Tenants, Attorneys

We asked respondents to explain their overall fairness ratings. The landlord did not comment, but many tenants and some attorneys did. Tenants who rated overall fairness as high focused on the clarity mediators provided them, describing mediators as “helping” and “kind.” An attorney who rated overall fairness high also emphasized the mediator’s “sympathetic demeanor.” A quarter of the tenant comments mentioned court-based rental assistance, which tenants were often referred to by the program. Several tenants also saw the mediators as helping, saying, “They stood up for me … They didn’t let [the landlord] push me,” and “[We asked] for what we wanted and [the mediator] basically fought for us to get it.”

In contrast, tenants who gave medium and low ratings on overall fairness tended to focus their frustrated comments on the mediator’s relationship with the landlord. One tenant said the mediator “may have been more partial to the landlord” because they “were familiar with one another”; another tenant said plainly that “they are there to mostly help the landlord.” One tenant felt frustrated that the mediator did not seem to believe what the tenant said at mediation, saying, “The mediator seemed to take what I had to say about the situation with a grain of salt.” Attorneys who rated the overall fairness at a medium or low level focused on efficiency, with one saying, “I was disappointed that the mediator allowed the opposing side to spend valuable time on issues irrelevant to the case.”

Likelihood to Recommend Eviction Mediation

To further explore participant satisfaction, we asked participants: “If a friend or colleague had a dispute like yours, how likely are you to recommend eviction mediation?” Most of the participants who responded were likely to recommend mediation to a friend or colleague, with 67% saying they were highly likely to recommend it. One tenant commented, “I would recommend all mediation options; sometimes tenants are unaware of the resources available due to lack of communication or shame.” However, another tenant who was less satisfied with the process commented, “It doesn’t help the tenant. At all. It helps landlords.”

As was the case with the first question on participant satisfaction, the landlord did not comment on their responses to this question, but we did receive two attorney comments. One attorney who was highly satisfied with the mediation process commented, “We made the exact same settlement offer that was accepted at mediation to the landlord’s attorney months ago, and they never responded in any way despite multiple phone calls. I assume this was on their client’s instructions. Because of the mediation process, I believe they would have continued stonewalling us.” The attorney who was unlikely to recommend mediation to a colleague said: “The lengthy mediation process is not helpful in my view. Before this system was implemented, and still now (in other counties), I am often able to reach agreements with the tenants within 5–10 minutes in the hallway outside the Courtroom. There is no need for the mediator, in my opinion.”

Conclusion

In conclusion, the survey responses indicate that the program continues to provide a positive experience to most participants. Those who completed the survey generally had positive perceptions of the mediators and the program, with the majority giving high ratings on fairness, trust and satisfaction. However, some participants’ comments point to a perception among tenants that mediators are biased toward the other side and a perception among attorneys that the mediation process is not efficient.

Jennifer Shack Talks about Inspirations, Dream Projects and the Future of ADR

Just Court ADR, July 19th, 2023

RSI Director of Research Jennifer Shack often uses this space to tell us about a new research project or share findings from her latest ADR program evaluation. Today, we asked her to take a step back and answer a few questions about what drives her work, as well as share her thoughts on a few “big questions” in our field.

What drew you to studying alternative dispute resolution (ADR) as a career?

When I was a Peace Corps volunteer in Benin, West Africa, I observed how the village chiefs resolved conflicts through what I was to discover was mediation. I thought it would be great to have something similar here in the States – a way to resolve conflicts without court intervention and in a way in which both parties felt was fair. I was surprised to learn about mediation when I returned home, and excited when I saw an ad for a job opening that started with the words “Interested in mediation?” I applied, and 24 years later I’m still enjoying my work at RSI.

What is your favorite part of your work?

So much! I really enjoy designing evaluations and research projects. I love interviewing program participants and conducting focus groups because I get to learn on a much deeper level how mediation programs affect the participants – and because I get to meet so many interesting people. I also have a lot of fun digging into data to find out what story they tell about a program or an issue and then writing that story.

Do you have a long-term wish list in terms of aspects of court-based ADR that you’d like to study?

I have a lot of items on my wish list. I’ll just talk about my top three. As you know, Donna Shestowsky and I evaluated two text-based ODR programs. I have also evaluated programs that involved in-person and video mediation. I would love to delve further into how these three different processes affect participant experience, particularly in what and how they communicate with each other and the mediator, and whether agreement terms differ. The more we know about how these processes are experienced by parties, the better we can become at determining which method best fits with different case types and situations, and the more we can improve the participant experience.

I would also love to do longitudinal research on child protection mediation. Having conducted a couple of evaluations on child protection mediation programs and interviewed parents after they participated in mediation, I think this is one of the best uses of mediation. But I’d like to know more about its long-term impact on families.

My third item on my wish list is already starting to become true. For decades, I and so many others have wanted to look inside the black box of mediation and find out what works and what doesn’t. We’re starting to do this with the Mediator Trust Project, but that’s only the first step. There are many aspects that can be examined. For example, in family mediation we can examine mediation’s effect on co-parenting and family dynamics. Another possibility is researching whether there are certain things mediators do that increase the probability of impasse.

RSI’s research team has recently expanded to include two additional full-time employees. How has this affected your day-to-day work or RSI’s project work?

RSI’s Research and Evaluation team recently expanded to include Rachel Feinstein, left, and Jasmine Henry.

Having Rachel and Jasmine join us has been wonderful. It’s really helpful to be able to talk through ideas and issues with other research-minded colleagues. I also am happy to have Jas do research on an idea that I otherwise wouldn’t have time to explore. But most of all having Rachel take leadership on our OPEN Project has allowed me to focus on our Mediator Trust Project while Jasmine continues to monitor and report on the participant surveys from the eviction mediation program RSI administers.

What trends do you see in court-based ADR that you think are likely to persist?

I think remote dispute resolution is here to stay, whether it’s video mediation or text-based ODR. Video mediation will continue to be prevalent, and I’m seeing signs that text-based ODR is going to become much more common in the near future. Artificial intelligence (AI) will make inroads in dispute resolution, particularly in helping parties to negotiate and write agreements. AI may also one day mediate between parties as well.

Outside of technology, I believe courts will continue to implement ADR to address crises, as we have seen with foreclosure and eviction. My optimistic side leads me to think that more courts will treat such cases holistically, attempting to resolve not just the dispute but the problems that led to the dispute in the first place – for example, providing housing and financial counseling to parties at risk of homelessness.

What is your least favorite part of your work?

Probably not having the time or money to pursue all the projects I’d like to do.

What do you see as keys to making court-based ADR more accessible?

The main thing is to break down barriers to participation. This means making the ADR process easier to navigate and use. It also means communicating with parties using multiple methods and keeping in mind best practices for individuals with low literacy. Courts need to ensure that parties know about the existence of ADR options. Donna Shestowsky’s research on civil court ADR and our evaluations of court ODR programs have shown that too many parties don’t know that ADR programs exist. Courts should also educate parties about the benefits and risks of their options if they have them, so they can make informed decisions about those options.

Verified by ExactMetrics