Resources / Study / Innovation for Court ADR

Just Court ADR

The blog of Resolution Systems Institute

81% Rate Fairness of Mediator Highly in Survey of RSI’s Eviction Mediation Program

Jasmine Henry, August 21st, 2023

Participants in RSI’s Kane County, Illinois, Eviction Mediation Program continue to have an overall positive experience, recent surveys suggest, though tenants responding to the survey tended to rate the fairness of their mediator more highly than they rated the fairness of the process itself.

RSI implemented the mediation program in 2021 to mitigate the eviction surge precipitated by the COVID-19 pandemic. Thanks to ongoing grant funding from the Illinois Equal Justice Foundation, RSI continues to administer and monitor the program.

The Survey and Respondents

RSI’s most recent survey report examines responses from participants in the 264 eviction mediations held in Kane County from March 1–June 30, 2023. After every mediation, participants were invited via email or text to complete an online survey about their experience. In all, 62 participants responded to the survey — 31 tenants, four landlords and 27 attorneys. A full report on the survey findings is available here.

The survey sought participant opinions regarding fairness, trust and satisfaction. Participants were asked to respond to the questions using a seven-point scale, which we consolidated into three categories: low (1-2), medium (3-5), and high (6-7). In addition, participants were sometimes invited to add comments to their responses.

We asked participants: How fairly did the mediator treat you? Their responses were largely positive, with 81% of respondents rating the fairness of the mediator highly. To explore more of their perceptions about the mediator, we also asked: How much did you trust the mediator? Their responses to this question also were positive overall, with 79% of respondents rating their trust of the mediator highly.

Turning more broadly to participants’ experience of the mediation process as a whole, we asked: Overall, how fair was the mediation process? Most of the participants who responded felt that the mediation was fair overall, with 73% saying it was highly fair.

Tenants on Mediator, Process Fairness

Interestingly, however, tenants tended to give the mediator higher ratings for fairness than they gave the overall process. For example, only 60% of tenants found the overall process fair, while 78% of tenants felt the mediator treated their side fairly. There was a similar, albeit smaller, shift observed in attorneys’ ratings, with 85% rating the mediation process as fair, compared with 89% who felt the mediator treated them fairly. We did not observe a similar trend in landlords’ responses.

We asked tenants to explain their overall fairness ratings. Their comments may shed some light on why some tenants rated the fairness of the mediation process lower than they rated how fairly the mediator treated them. Tenants who rated overall fairness highly focused on the clarity mediators provided them, describing mediators as “helpful,” “nice,” and going “above and beyond in making sure that I understood [and] felt comfortable.” Several tenants also appreciated having a voice and feeling respected, with comments mentioning being able to “tell [their] story of how [they] got to where [they were]” and saying that mediators “made [them] feel like an actual person and not just some case number.” In contrast, tenants that gave medium and low ratings on overall fairness tended to focus their frustrated comments on landlords, rather than the mediator or the program; for example, some tenants who gave medium ratings on overall fairness still referred to the mediator as “gracious” and “fantastic” in their additional comments.

Landlord, Attorney Perspectives

All four landlords gave a high rating for process fairness. The two who explained their answers praised the rental assistance program that RSI regularly refers tenants to and acknowledged the benefit of an objective mediator’s “fresh perspective.”

Attorneys who rated the overall fairness highly described the benefits of having a facilitated dialogue so that both parties can speak their minds and consider their options. The few attorneys who rated the process as medium or low on fairness were frustrated with specific mediator actions; one comment stated that the mediator was “disrespectful to the plaintiff in many ways,” including because of their “lack of knowledge and deference to the defendant.”

Likelihood of Recommending Mediation

In order to further explore participant satisfaction, we asked participants: If a friend or colleague had a dispute like yours, how likely are you to recommend eviction mediation? Most of the participants who responded were likely to recommend mediation to a friend or colleague, with 70% saying they were highly likely to recommend it. One tenant commented that “It would be an absolute mistake no[t] going through mediation. It is the best decision when going through this type of process.” Only four tenants were unlikely to recommend mediation.

Attorneys who were highly satisfied with the mediation process commented on the benefit of, as one said, “a solution that both sides had some part in reaching.” Meanwhile, an attorney who was less satisfied with the mediation process cited “partial” mediators as a negative.

Conclusion

Overall, the survey responses indicate that the program continues to provide a positive experience to participants. Those who completed the survey generally had very positive perceptions of the mediators and the program, with most giving high ratings on fairness, trust and satisfaction. Some participants’ comments point to a possible topics for ongoing mediator education.

For more background on RSI’s 16th Judicial Circuit of Illinois Eviction Mediation Program, serving Kane County, read our evaluation of the first full year of the program, Addressing Eviction Holistically, published in late 2022, or RSI Director of Research Jennifer Shack’s blog summarizing its findings.

Jennifer Shack Talks about Inspirations, Dream Projects and the Future of ADR

Just Court ADR, July 19th, 2023

RSI Director of Research Jennifer Shack often uses this space to tell us about a new research project or share findings from her latest ADR program evaluation. Today, we asked her to take a step back and answer a few questions about what drives her work, as well as share her thoughts on a few “big questions” in our field.

What drew you to studying alternative dispute resolution (ADR) as a career?

When I was a Peace Corps volunteer in Benin, West Africa, I observed how the village chiefs resolved conflicts through what I was to discover was mediation. I thought it would be great to have something similar here in the States – a way to resolve conflicts without court intervention and in a way in which both parties felt was fair. I was surprised to learn about mediation when I returned home, and excited when I saw an ad for a job opening that started with the words “Interested in mediation?” I applied, and 24 years later I’m still enjoying my work at RSI.

What is your favorite part of your work?

So much! I really enjoy designing evaluations and research projects. I love interviewing program participants and conducting focus groups because I get to learn on a much deeper level how mediation programs affect the participants – and because I get to meet so many interesting people. I also have a lot of fun digging into data to find out what story they tell about a program or an issue and then writing that story.

Do you have a long-term wish list in terms of aspects of court-based ADR that you’d like to study?

I have a lot of items on my wish list. I’ll just talk about my top three. As you know, Donna Shestowsky and I evaluated two text-based ODR programs. I have also evaluated programs that involved in-person and video mediation. I would love to delve further into how these three different processes affect participant experience, particularly in what and how they communicate with each other and the mediator, and whether agreement terms differ. The more we know about how these processes are experienced by parties, the better we can become at determining which method best fits with different case types and situations, and the more we can improve the participant experience.

I would also love to do longitudinal research on child protection mediation. Having conducted a couple of evaluations on child protection mediation programs and interviewed parents after they participated in mediation, I think this is one of the best uses of mediation. But I’d like to know more about its long-term impact on families.

My third item on my wish list is already starting to become true. For decades, I and so many others have wanted to look inside the black box of mediation and find out what works and what doesn’t. We’re starting to do this with the Mediator Trust Project, but that’s only the first step. There are many aspects that can be examined. For example, in family mediation we can examine mediation’s effect on co-parenting and family dynamics. Another possibility is researching whether there are certain things mediators do that increase the probability of impasse.

RSI’s research team has recently expanded to include two additional full-time employees. How has this affected your day-to-day work or RSI’s project work?

RSI’s Research and Evaluation team recently expanded to include Rachel Feinstein, left, and Jasmine Henry.

Having Rachel and Jasmine join us has been wonderful. It’s really helpful to be able to talk through ideas and issues with other research-minded colleagues. I also am happy to have Jas do research on an idea that I otherwise wouldn’t have time to explore. But most of all having Rachel take leadership on our OPEN Project has allowed me to focus on our Mediator Trust Project while Jasmine continues to monitor and report on the participant surveys from the eviction mediation program RSI administers.

What trends do you see in court-based ADR that you think are likely to persist?

I think remote dispute resolution is here to stay, whether it’s video mediation or text-based ODR. Video mediation will continue to be prevalent, and I’m seeing signs that text-based ODR is going to become much more common in the near future. Artificial intelligence (AI) will make inroads in dispute resolution, particularly in helping parties to negotiate and write agreements. AI may also one day mediate between parties as well.

Outside of technology, I believe courts will continue to implement ADR to address crises, as we have seen with foreclosure and eviction. My optimistic side leads me to think that more courts will treat such cases holistically, attempting to resolve not just the dispute but the problems that led to the dispute in the first place – for example, providing housing and financial counseling to parties at risk of homelessness.

What is your least favorite part of your work?

Probably not having the time or money to pursue all the projects I’d like to do.

What do you see as keys to making court-based ADR more accessible?

The main thing is to break down barriers to participation. This means making the ADR process easier to navigate and use. It also means communicating with parties using multiple methods and keeping in mind best practices for individuals with low literacy. Courts need to ensure that parties know about the existence of ADR options. Donna Shestowsky’s research on civil court ADR and our evaluations of court ODR programs have shown that too many parties don’t know that ADR programs exist. Courts should also educate parties about the benefits and risks of their options if they have them, so they can make informed decisions about those options.

Courts Can Take Steps to Design Text-Based ODR Programs that Better Serve Parties

Jennifer Shack, June 22nd, 2023

While conducting two of the first independent evaluations of text-based online dispute resolution (ODR) programs in U.S. state courts, Donna Shestowsky and I found those programs promoted access to justice in some ways, but inhibited it in others. To help other courts, we wrote an article about how they might reduce potential barriers when developing and implementing their text-based ODR programs. The following is a summary of our advice from the article, “Access to Justice: Lessons for Designing Text-based Court-Connected ODR Programs,” which was recently published in Dispute Resolution Magazine, a publication of the American Bar Association.

Court adoption of text-based ODR allows parties to communicate asynchronously, at their convenience, from anywhere. This suggests that ODR has the potential to increase access to justice, particularly for self-represented litigants,[i] and could lead to increased efficiency and reduced costs for parties and courts alike.[ii] Conversely, however, for parties who lack digital literacy or access to technology, mandated ODR could instead benefit already advantaged parties and leave others behind. Furthermore, in some instances, mandating ODR could reduce access to justice by overriding consent and party self-determination.[iii]

The Texas and Michigan Programs

The programs we evaluated differed in the issues involved and the platforms used. In Collin County, Texas, we assessed a debt and small claims pilot program in a busy Justice of the Peace Court (JP3-1) that used the Modria platform. In Ottawa County, Michigan, we examined a program for post-judgment family matters brought to the Friend of the Court (FOC), an agency under the aegis of the Chief Judge of the 20th Circuit Court. The FOC used the Matterhorn platform. Both programs, however, were intended to be mandatory once the program was referred. And both required that the parties register and communicate via text on the ODR platforms.

Litigant survey responses suggested that many parties were unaware of the ODR program or did not understand its main features. When asked what would make them more likely to use ODR for a similar case in the future, half said more information.

Although the programs we evaluated used different ODR platform vendors, the platforms worked similarly and had comparable limitations. The platforms provided a chat space and permitted third-party facilitation or mediation. Neither was available to those with significant visual impairments or limited English proficiency. Both allowed only one individual per side to participate. This limitation meant that in Texas if a party had a lawyer, the lawyer participated alone. In Michigan, only parties could participate, and those who had lawyers were not referred to ODR.

Possible Reasons for Not Using ODR

Although ODR was ostensibly mandatory in both programs, the majority of parties in each court did not use ODR. In Texas, both parties to a case used the platform in only 81 of 341 cases (24%) referred to ODR. In Michigan, ODR use was twice as high: For the 102 matters in which caseworkers determined ODR was appropriate, 48% used ODR. In 26 of the 53 matters in which the parties in the Michigan program opted not to use ODR, at least one party did not register on the platform.

Survey and interview data suggest a few reasons parties did not use ODR. In both programs, staff indicated they did not send parties who lacked digital literacy to ODR, and litigant survey responses suggested that many parties were unaware of the ODR program or did not understand its main features. In the Texas program, of those who did not use ODR, only one survey respondent (out of ten) indicated having received information about the program. When asked what would make them more likely to use ODR for a similar case in the future, half said more information.

In survey responses for the Michigan program, parties appeared to lack a basic understanding of how ODR worked. Half of the 50 parties surveyed near the start of their matter did not know ODR was offered free of charge.

According to Texas court staff, litigants received information about the ODR program via the notice the court sent to them (or their lawyers) about their court date, and through an email or text from the platform when the court uploaded their case to it — if the court had their email address or cellphone number. Both the notice and the email lacked information about how ODR worked. Similarly, the Michigan program’s automated email and text, platform, and FOC website missed opportunities to educate the parties.

Implications for Courts

Despite their accessibility issues, both the Texas and Michigan programs had similar access to justice benefits. Our evaluations suggest that for those parties who use ODR, the process is convenient. We found that 72% of ODR use in Texas and 52% in Michigan occurred outside of court and office hours, i.e., at times not available to them in traditional dispute resolution methods. However, in both programs, many parties simply did not register to use ODR. In addition, 50% of ODR users who responded to our survey noted that they liked that ODR was easy to use. These findings indicate that ODR can increase convenience.

Nonetheless, our finding that some parties lacked information or had nontrivial misconceptions about ODR also suggests parties did not always make informed decisions about whether to participate. To enhance access to justice and self-determination, courts should incorporate a communications plan. The plan should:

  • Specify how parties can learn about the program and detail what information court personnel should relay about ODR
  • Indicate what information about ODR to include on the court’s websites and the ODR platform to educate parties about how to use ODR and its potential risks and benefits
  • Outline outreach efforts to urge social services or other agencies to inform their clients about the ODR program

Additionally, courts should present information about ODR in a way that is comprehensible to individuals with low literacy. They should also explain the privacy and confidentiality implications of using ODR, especially regarding whether and how communications shared on the platform might be used in subsequent legal proceedings.

Further, ODR offerings should be accessible to all eligible parties. Courts should urge ODR providers to facilitate use by parties with visual impairments and limited English proficiency. Additionally, courts should direct parties who do not have reliable internet access to computers in the courthouse or other community locations — though as a result of limited business hours and privacy concerns, this solution is far from ideal.

Courts should also ensure that text-based platforms are user-friendly for smartphone users. In the Michigan program, 71% of participants exclusively used a smartphone for ODR. (We did not have information on the devices Texas ODR participants used.) Yet our findings indicate that text-based ODR may be difficult for smartphone users. Courts should urge ODR providers to include in-app voice control to facilitate ODR use on smartphones generally, a change that might be especially important for individuals with disabilities that restrict their ability to type. Parties should also be able to participate in ODR with their attorneys.

Finally, courts should explore ways to maximize access to their platforms for those who lack digital literacy. Usability testing, similar to that conducted for Utah’s ODR pilot program,8 can help identify challenges and potential solutions for given platforms. Courts might also consider providing parties with links to web-based resources or trainings that could increase their comfort with technology.

Given ODR’s current technological limitations and the percentage of the population that continues to lack reliable internet access or digital literacy, ODR is not a panacea for the continued access to justice problem in the U.S. Additionally, our evaluations suggest that parties have different preferences for how to resolve their disputes. To enhance access to justice, and to advance party self-determination, ODR might best serve parties as part of a constellation of alternative dispute resolution (ADR) options rather than being the only form of court-connected ADR.


[i] Amy J. Schmitz, Measuring “Access to Justice” in the Rush to Digitize, 88 Fordham L. Rev. 2381 (2020).

[ii] Amy J. Schmitz, Measuring “Access to Justice” in the Rush to Digitize, 88 Fordham L. Rev. 2381 (2020).

[iii] Amy J. Schmitz & Leah Wing, Beneficial and Ethical ODR for Family Issues, 59 Fam. Ct. Rev. 250 (2021).

New, Free IPV Screening Tool Can Help Ensure that Mediation is Safe and Fair

Rachel Feinstein, June 14th, 2023

Screening for interpersonal violence and abuse (IPV/A) is an essential step for making family mediation safer and more fair for all parties. Unfortunately, in “Joint Session or Caucus? Factors Related to How the Initial Mediation Session Begins,” Roselle Wissler and Art Hinshaw found that only 11% of surveyed family mediators screened for IPV, or had access to the screening results, prior to mediation (see footnote 85). A new tool, the MASIC-S, might change that. The MASIC-S is available free at ODR.com for all mediators. It is an abbreviated version of the Mediator’s Assessment of Safety Issues and Concerns (MASIC), which was developed in 2010 and is widely referenced for use in family mediation.

Screenshot of the webpage for MASIC-S, a new tool to screen for interpersonal violence and abuse.
MASIC-S is an abbreviated version of the Mediator’s Assessment of Safety Issues and Concerns (MASIC). It is available for free at ODR.com (screenshot from website above) for all mediators.

RSI is particularly interested in this new tool in light of our 2018 project, supported by the Family and Interpersonal Resilience and Safety Transformation Fund, that studied IPV screening tools, surveyed experts in IPV dynamics (as well as lawyers, judges and mediators) on best and actual practices, and convened those experts to explore how to close that gap. The research led to an extensive report outlining RSI’s proposed solution and the steps to actualize it.

The questions in the MASIC-S focus on abuse in past or current relationships. Many of the questions have been validated, meaning that they accurately identify severe and concerning degrees of abuse, which the screening tool aims to assess. Mediators can screen parties during intake by privately administering the MASIC-S questionnaire in person or through videoconferencing. After each party has completed the questionnaire, the mediator will be prompted to consider specific questions regarding whether or not mediation is appropriate for the case, and if so, what accommodations might be necessary for a safe and voluntary process.

The Comprehensive Guidance provides detailed instructions for administering the MASIC-S, as well as directions to follow based on a party’s score. For example, if a party receives a score of 3 or higher, the guide recommends refraining from mediating jointly in-person; however, online mediation may still be a reasonable option. The Comprehensive Guidance even provides scripts to follow, such as what a mediator can say to safely terminate mediation without blaming either party or endangering a survivor.

The MASIC-S is also designed to protect confidentiality of the parties. For example, mediators do not record any identifying information that could connect responses to a particular person, and the results are not shared with the other party. Additionally, the responses are not stored online; mediators can instead download a PDF of the responses if they want a record of them. Using this abbreviated screening tool at intake can help to ensure mediation is appropriate for the parties and necessary accommodations are made for a safe and just process.

Verified by ExactMetrics