RSI Director of Research Jennifer Shack joined ODR.com CEO Colin Rule and Redek founder Nicolas Lozada this month for an ODR Cyberweek 2024 webinar on the topic of online dispute resolution to settle family law cases, focusing on a report Shack co-authored in 2021.
ODR Cyberweek is a free, virtual conference hosted annually by the National Center for Technology and Dispute Resolution (NCTDR).
Shack answered questions about the 2021 evaluation of a Michigan ODR program she conducted with University of California Davis law professor Donna Shestowsky, which was the first third-party evaluation of a family law ODR program in the United States.
The program, in Ottawa County, Michigan, was launched by the 20th Circuit of Michigan’s Friend of the Court in August 2020 with the goal of providing parties with post-judgment family law disputes a simpler, more convenient and cost-effective way to reach agreements related to child custody, parenting time and child support. It also aimed to increase efficiency in the disposition of these matters.
Among the topics that came up in the ODR Cyberweek discussion were the varying levels of interest by caseworkers and others in participating in ODR; the likelihood that cultural and demographic differences might impact openness to ODR, as well as ways of dealing with conflict in general; and the potential for and possible hurdles to using ODR to mediate cases where intimate partner violence is a factor.
Shack also offered insight on how to improve communications with parties about ODR as a means to improve understanding of the process and increase participation, based on findings from RSI’s ODR Party Engagement (OPEN) Project.
Lozada, who founded the Colombian ODR startup Redek, noted the challenges of advocating for ODR in Colombia when consistent internet access and use remains out of reach for much of the population. In addition to the access problems this presents, it can also mean that those who do have access remain wary of the legitimacy of online programs, Rule suggested.
NCTDR’s ODR Cyberweek serves an international audience, is open to the public, and includes panels in English, Chinese, Spanish and Portuguese. Additional topics this year included AI integration in courts, mediation and arbitration, recent innovations and research findings. The event also included tech demos and a student panel. Recordings of many of the ODR Cyberweek sessions from this and previous years can be found in the NCTDR’s ODR Cyberweek Archive.
RSI recently completed user testing research for the ODR Party Engagement (OPEN) Project! We are excited to share insights from our experience conducting the testing and what we learned from participants about our model court communication materials.
User Test Study
As part of the OPEN Project Phase 2, RSI developed prototype materials — a webpage, a notice document, an informational video and an interactive guide — to help courts communicate more effectively about online dispute resolution (ODR). We worked with an inclusive designer to create materials that exemplify best practices based on designer expertise, OPEN Phase 1 findings and previous research. To ensure these materials were accessible and user-friendly, we planned a series of user tests to obtain feedback from actual users whose backgrounds resemble those of self-represented litigants.
Over the past two months, the RSI research team conducted user tests across the country. We recruited a total of 15 demographically diverse participants from distinct areas: a rural town in New Hampshire; Baltimore, MD; and the Santa Fe-Española area of New Mexico. Participants were instructed to use the materials to complete relevant tasks: read about ODR, watch the informational video, prepare for ODR as a defendant, and sign up for an account. While they tested the materials, RSI researchers observed and took notes on their behavior, listened to their thought process, and asked follow-up questions. We also disseminated post-test surveys and conducted semi-structured interviews about their experience with the materials.
Read more background on the ODR Party Engagement (OPEN) Project, from concept, to focus group feedback, to creation of tools for courts, in RSI’s earlier blog posts.
Real World Feedback
Feedback from the user tests helped us to identify strengths and weaknesses in each of the materials. Users overwhelmingly praised our video for its clarity, engaging visuals, concise discussion about ODR and appropriate length; the video averaged a 4.73/5 rating for how easily it was understood.
The sessions highlighted the importance of accessible visual design. Participants missed some of the information on our webpages because they lacked sufficiently visible indicators for clickable or interactive content. Higher saturated colors, consistent blue hyperlinked text and plus signs are some of the fixes we will implement in new versions of the webpages.
Participants were excited about the prospect of ODR being available to them. The relative ease of their testing experience only amplified this excitement; participants shared frustration with previous instances trying to navigate difficult-to-use court materials. In contrast, they found our materials to be simple to follow and understand.
User testing also provides a beneficial opportunity for users to share their own creative ideas for improving the materials. During the sessions, these ideas encompassed: making the account login interface more accessible, identifying when repetitive information was useful or distracting, and enhancing the perceived trustworthiness of our notice document.
Participants were excited about the prospect of ODR being available to them. The relative ease of their testing experience only amplified this excitement; participants shared frustration with previous instances trying to navigate difficult-to-use court materials. In contrast, they found our materials to be simple to follow and understand, a sentiment shared even among those who were hesitant to use digital technologies generally. Describing the account creation pages, one participant in New Hampshire noted, “Yes, [I would feel confident helping a friend sign up for ODR using this system]. It’s really easy to use. I think my 10-year-old can do this.”
We will next return to our design partners to implement the feedback we received from user testers. RSI is grateful for the participants’ generosity and excitement for the project. We are planning to conduct one more round of user testing once the revised materials are completed to ensure any lingering issues are addressed before making the final models available for courts. RSI is also grateful to the American Arbitration Association-International Centre for Dispute Resolution (AAA-ICDR) Foundation for its funding of the OPEN Project.
RSI has completed the first phase of the ODR Party Engagement (OPEN) Project! We are excited to share what we have learned from our research and to roll out new downloadable tools designed to support courts. A summary of our research, our full report and the tools can be found in a new section of our website.
The Study
For this extensive study, RSI conducted six focus groups with a total of 41 participants from three areas of the country. Our aim was to uncover barriers to using court-referred text-based ODR (online dispute resolution) and ways courts can address these barriers. More information about the study, a summary of the findings and key takeaways are available on the OPEN Project section of our website, in addition to the full report.
Tools for Courts
Informed by our research findings, RSI developed tools to help courts communicate more effectively about ODR. The two downloadable documents, “Communicating Effectively about ODR: A Guide for Courts” and a “Document Preparation Worksheet and Checklist for ODR Programs,” are both accessible on our website.
The Guide for Courts provides recommendations for developing documents, websites and video tutorials for ODR that are easy to understand and use, particularly for those who have low literacy and are self-represented.
The Document Preparation Worksheet and Checklist walks readers through each step of the document prep process, from considering the characteristics of the audience to determining the key content to include and the best way to communicate this content. Once documents are completed, the interactive checklist provides an opportunity to review the material and ensure all key recommendations were incorporated.
Next Up in Phase 2: Model Materials
RSI is very proud of what we have accomplished in Phase 1, and there is more to come! We are now starting Phase 2, in which we will work with an inclusive designer to create model court documents, a brief informational ODR video and a mockup webpage, using our findings from Phase 1 and previous research on best practices.
We will user-test these models to learn what barriers persist for people when attempting to use the models to register for ODR. After all our user-testing is complete and barriers have been addressed, courts will be able to download the final model documents, video and webpage to use as examples when developing their communications. This will provide added support to courts with limited resources, who will be able to simply replicate these models and tailor them to their specific needs and audience using instructions we will provide.
RSI is grateful for the generous support of the American Arbitration Association-International Centre for Dispute Resolution Foundation for making the OPEN Project possible.
RSI’s research has shown that self-represented parties in small claims cases often don’t understand what online dispute resolution (ODR) is or how to use it, even when courts require their participation. To learn what self-represented parties need when a small claims case is filed against them, RSI’s OPEN Project is going to the source — conducting focus groups with people similar to these parties and asking what works for them.
Participants in an RSI focus group in Texas provide feedback on sample court documents in October 2023.
Director of Research Jennifer Shack and I led two focus groups in rural New Hampshire in August, followed by two groups in Texas in early October. We will finish our data collection for the ODR Party Engagement Project in Maryland this month. In the meantime, we want to share some of the initial insights we have gained.
Hearing from 26 participants so far, we have learned about many of the barriers people experience when faced with examples of court documents, a court website and instructional court videos. Groups also shared their recommendations for how the material could be improved and their preferences for receiving court notifications and instructions. The majority of participants have a maximum of high school education. Most, if not all, participants earn less than $50,000/year. These income and education characteristics parallel the backgrounds typical of self-represented litigants, making their insights regarding the comprehensibility and usability of court material invaluable as we aim to develop recommendations for accessible court resources.
Notification Preferences Vary Widely
We are excited to share some preliminary findings from our focus groups. First, we have learned that providing court resources in a variety of formats is essential to addressing the public’s needs and preferences. Focus group participants expressed minimal consensus about the ideal way to learn about their involvement in a lawsuit or how to proceed with online dispute resolution. For example, only half of the 26 participants said they would prefer to receive an initial notice about their lawsuit through the mail. Six people would prefer to receive notice about their case over the phone, while five would prefer text message, and only one person wants to learn of their case via email.
“There are times where the form of a video works wonders in comparison to throwing a chapter out of a book at me or something.”
— Focus group participant
Further reflecting this need for variety, participants in two of our groups were enthusiastic about using instructional videos to learn about registering for ODR. One participant in New Hampshire shared, “I think a video would be good. Where they could break it down and explain it a little bit more in depth.” Another person agreed, “Yeah, I think so. I mean, I’m a visual learner … If this was on YouTube … everything would be fine. It’d be perfect.”
A third participant added, “There are times where the form of a video works wonders in comparison to throwing a chapter out of a book at me or something.”
In contrast, most participants in the Texas focus groups did not express a need or interest in viewing videos to get this information. But several people did agree that, as one said, “options are good,” when attempting to meet the potential variety of needs, learning styles and preferences among self-represented litigants.
Participants Wary of Possible Scams
RSI focus group participants in Texas shared their recommendations for how court informational materials could be improved and their preferences for receiving court notifications and instructions.
One topic where focus group participants were largely in agreement was their concern about being scammed. During the focus groups, we asked all participants to look at one of two ODR websites on a laptop or tablet that we provided. The first step many participants took was to assess the credibility of the website. For instance, the first reactions routinely included comments about whether the site was legitimate or a scam. Some participants also expressed apprehension regarding receiving the mailed Notice to Defendants, wanting to contact the court to check that it was legitimately a lawsuit against them before following the instructions on the document.)
This initial step of assessing documents and websites for legitimacy may be crucial for courts to be aware of when developing their communications and other resources, since apprehension about whether the material is trustworthy could inhibit people from beginning the process.
Simple, Organized Info Is Desired
One of the most consistent themes among the participants so far has been the desire for court resources to be simple and quick to use. For instance, we heard from many individuals who want courts to use simple language, concise instructions and well-organized documents or videos. Some participants specifically requested more spacing around paragraphs, and people found sections with bullet points or short fill-in-the blank questions easy to understand.
We anticipate delving more deeply into strategies for making court resources simpler to use and comprehend. Additionally, participants have been identifying key information that is missing from the material and sharing their emotional responses to the court resources. We look forward to examining these and other themes in more detail after we conclude our focus groups later this month.
Check back soon for a summary of our findings and a guide for courts, which we will provide on a new RSI webpage this spring!
As always, RSI is grateful to the AAA-ICDR Foundation for supporting this important work.
