01
LEP (Limited English Proficiency) Healthcare Assitant
ROLE: RESEARCHER, DESIGNER
August 2020 - December 2020
Project completed in Cornell University with Xirui He, Danni Ji, Yuchang Zhou
Instructor: Professor Cristobal Cheyre Forestier
ABOUT THE PROJECT
A Digital Platform Which Improves the accessibility of medical resources for under-served communities
With "Making the World More Just and Equitable" as our high-level target, our team chose to tackle the problem of health care disparity with a creative technology solution to improve the accessibility of medical resources for the under-served population.
-
End-to-end treatment process all in patients' native language
-
High mobility with movable stand and height adjustment options
-
Assigning translation tasks based on complexity
.png)
.png)
01
Why is This Problem Important?
Our problem space focuses on under-served communities not getting equal health care accessibility as advantaged populations. Critical factors to healthcare disparities include race, medical insurance, education, transportation limitations, and language barrier, which would ultimately affect patients' chances to receive proper treatment and the quality of healthcare services.
Taking language barrier as an example: It's reported that patients with limited English proficiency widely suffered from long waiting times and high medical error rates caused by miscommunication which could even lead to their death.
Therefore, we hope to design a solution that could effectively improve the accessibility of healthcare services for less advantaged groups with technology that could be applied in realistic settings, which would essentially assist the under-served community to receive better treatment.
02
Narrowing down the problem scope
Not all differences in health status between groups are considered disparities, but rather only differences that systematically and negatively impact less advantaged groups are considered disparities. Much of the current research focuses on unequal health treatment because of racial and ethnic disparities in the community [3].
Other important factors to health disparities we found include food insecurity, limited access to transportation, housing instability, lack of insurance coverage, and cultural, linguistic barriers [4]. When caring for the under-served community, an important theme is the inadequate patient resources and provider resources. It might be difficult for patients to access educational materials, especially non-English ones [2]. Many local organizations target specific racial groups or issues to help the under-served communities get more information and better healthcare. However, a large portion of these materials is information-based and is inefficient to use.
After the initial research, we narrowed down the topic to improve the hospital service for patients with limited English proficiency. Currently, some issues around the language barrier for patients in hospitals are that the hospitals do not routinely monitor medical errors for patients with limited English proficiency (LEP). It is hard for hospitals to identify the language the patient speaks.
IN-PERSON INTERPRETER
The translation has higher accuracy, but there is usually a long waiting time, and there are not enough interpreters, especially for uncommon languages.
CROWDSOURCING
Crowdsourcing can play a pivotal role in future efforts to create parallel translation datasets. It provides access to languages that currently fall outside the scope of statistical machine translation research. By soliciting multiple translations and redundantly editing them than selecting the best of the bunch, it is possible to get high-quality translations without professional medical translation [6].
MACHINE TRANSLATION
Machine translation is an older technology and has many limitations. For example, machine translation can only be accurate when the machine has understood the text in an appropriate sense. Although the direct narration of the content can be exact, there are still limitations in the expressions of meaning [5].
03
User Needs Analysis
Other than the research paper, we conducted four interviews with researchers and experts from Cornell Center of Health Equity. With all the valuable notes we collected during our interview, we used an affinity diagram to analyze our findings further.
LANGUAGE DIFFERENCES
The hospital provides an interpreter, usually translated using the telephone.
The patient sometimes brings their children as interpreters.
Patients who go to the doctor speak a specific language.
Patients speak different languages, and they hardly receive health care.
The hospital usually pays for the interpreter service.
Health care products need to be multi-language because certain people do not speak English.
Hospital has interpreters by phone call and video translation with a small TV screen.
Some doctors use Google translate for people who speak non-English, not good for complicated medical terms.
RECEIVING HEALTH INSURANCE
Convince people they should get health insurance.
Patients do not have health insurance, and medical expenses are too expensive.
The government pays for community health centers, and people can get medical care even if they do not have health care.
Community health workers are people who have health training but are not a doctor.
For people after 65, the government will pay health insurance.
Convince insurance companies to pay since it could reduce the death rate in the long term.
Some companies do not offer health insurance, and some unemployed people.
Doctors are not willing to make an appointment for medical aid patients because of low salaries.
Government policy may help people get insurance.
People give health insurance to their employers.
USABILITY AND RESTRICTIONS
The phone call, SMS as alternative options.
Apply usability principle for health tech.
50% of low-income patients do not have internet access.
People might give up if they cannot understand the technology.
ACCURACY OF DIAGNOSIS
Community health workers need more information to help with diagnosis.
Doctors need to decide on specific diseases.
Community health workers mainly provide help for pregnant women, newborns.
DATA COLLECTION
Data collection tool to help government understand the situation and divide medical supply.
Essential data collection needs to understand their condition better.
Community health care workers can connect people better to the public healthcare system.
From our understanding of the affinity diagram results combined with our findings from the research paper, we found that language issues are of great concern to hospitals, doctors, and patients. The number of immigrants and non-English native speakers is enormous; however, the number of interpreters was limited. Most translation services are connected by phone, which lacks accuracy when the interpreter tries to understand the patient’s problems.
04
Storyboard & Goals
Reduce the waiting time caused by insufficient professional interpreters.
Decrease medical error rate related to language limitations.
Provide multi-language technology service while having alternative options, including text messages and phone calls for patients with limited internet access.
Protect privacy and build trust for patients and healthcare providers.
05
Initial Exploration & Feedback
In our first phase of the wire-framing (low fidelity) design process, we considered four main stakeholders: LEP patients, health workers, professional interpreters, and the volunteer staff.
We hope to solve the shortage of translators by recruiting volunteers and through appointment arrangements to increase translation staff's work efficiency. At the same time, we arrange a professional translator and a volunteer in each translation work to supervise the translation results and quality of each other to ensure the accuracy of the translation.
PRIMARY FUNCTIONS
-
Patients can make appointments online, by telephone, or by SMS.
-
Doctors can view the content of the appointment and request the participation of translators and volunteers. Translators and volunteers can agree to join the meeting.
-
A search function to facilitate interpreters and volunteers to confirm some professional medical terms further.
-
A function for volunteers to record relevant diagnostic results and prescriptions. This information will be sent to the patient after the meeting.
FEEDBACK FROM PROFESSIONAL HEALTHCARE WORKERS
You will not be able to keep the number of volunteers to participate in the meetings. In addition, having both a professional interpreter and a volunteer join in a meeting does not solve the problem of insufficient translators to a large extent but slows down the interaction.
06
Logic Behind Design
REALIZED PRODUCT MOBILITY BY ADDING A MOVABLE BASE
Allows the product to fit a more diverse situation and more convenient for doctors and patients to use.
COMBINED CROWDSOURCING WITH AI TRANSLATION
By including these two resources in our product, we mainly solve the problem of not having enough interpreters because it is possible to obtain high-quality translations from non-professional translators. The cost is an order of magnitude cheaper than professional translation.
TRANSLATION ABILITY TEST
The crowdsourcing staff needs to pass this test to serve as a real case. This test will give users some random medical-related sentences and ask them to finish translation in a short time, and then AI will check their accuracy level.
We minimized the possibility that individuals caused translation errors.
EXPECT FIVE CROWDSOURCING STAFF TO WORK TOGETHER WHEN DOING THE TEXT TRANSLATION AND HAVE AI AUTOMATICALLY CHECK THE TRANSLATION QUALITY
PRIVACY PROTECTION
Several experts mentioned that privacy and safety are significant concerns in designing the technology solution for patients and doctors in our interviews. Based on this requirement, we have strengthened the privacy protection methods of the original video call. In the initial video process, patients can only choose to turn their videos on or off, but this will also affect the accuracy of the translation at certain times. We took this into consideration and added a design that allows patients to mosaic their faces. This not only protects the privacy of patients but also allows translators to receive their physical expressions better.
INCREASED CONTACT EFFICIENCY WITH PROFESSIONAL TRANSLATORS
While connecting with the interpreters, the system will quickly calculate based on the interpreter's work schedule, availability status, and if the interpreter fits the patients' language needs. Based on the calculation result, there will be a list of interpreters the system is trying to connect with. For each interpreter, we will wait for 25 seconds for the interpreter to accept the translation request.
07
Finalized User Flow
PATIENT
DOCTOR
Choose Language
View Appointmnets
Take Translation Ability Test (First Time Only)
Input Basic Informations
Choose Crowdsourcing or Interpreter
Setup Work Schedule
Book Appointment
M
E
E
T
I
N
G
See Appointment Records
Change from Crowdsourcing to Interpreter
Submit Record
Review Record and Confirm Translation Results
CROWDSOURCING
Setup Work Schedule
INTERPRETER
Review Record and Confirm Translation Results
08
Hi-Fidelity Prototype
.png)
Multiple Users
Different Languages
The software will be available to patients, doctors, interpreters, and crowdsourcing staff. Each type of user will use different functions.
In the initial interface, users will choose their language to ensure that they will not be unable to use the software due to language problems.
Make an appointment
According to the patient's condition's prediction, he can choose the most suitable doctor by looking at different doctors' detailed information in various departments.
When making an appointment, he needs to fill in his detailed information, including his name, birthday, phone number, and language used. Besides, he also needs to describe his condition so that the doctor can predict. In the whole process, the patient does not need to use any English. All the interfaces he sees will be translated into the language of his choice, and the content he fills in will be automatically translated into English.
Review the Appointment
Doctors can view the appointment's content and request the participation of either a professional interpreter or the crowdsourcing translation service based on patients' descriptions of their situation. Ideally, if the patient's problem is not too severe, the doctor will choose to crowdsource and ai translation. If the doctor thinks the problem is severe, the doctor can choose a professional translator to join the consultation.
Normal CaseS
During the staff's work time, crowdsourcing staff will translate the patients' and doctors' words by typing them out. (The crowdsourcing staff needs to pass a translation ability test to serve for a real case.) The AI will check the translation quality. After comparing the crowdsourcing staff's translation results, the AI will send the translated text with audio.
SEVERE Cases
If doctors believe that the patient's symptoms are more severe than they thought before the meeting or doctors consider the case, they need high accuracy in understanding the patient's situation. The doctor can request a professional interpreter immediately. A skilled interpreter will be added to the conversation in the form of voice translation.
.png)
Doctors submit the records about diagnosis and prescriptions. The AI will do an auto-translation. Our professional Interpreter here (can be crowdsourcing staff if no professional interpreter is involved in the meeting) will confirm the translation results. After that, patients can view their records with translated results.
Diagnosis & Prescriptions
.png)
09
Evaluation
We used the Wizard of Oz as the testing technique.
In addition to the user, three people will be participating in the interview process: the doctor, the interpreter, and the crowdsourcing staff, and the other will be a notetaker. The person acting as the doctor will face the user, who will pretend to be a patient, while the remaining two will participate remotely.
We have prepared two tasks for users to try.
The first is a milder symptom, the patient has a rash, and crowdsourcing staff will translate for him throughout the process. The other is abdominal pain; the doctor underestimated the severity so crowdsourcing staff will be switched to professional interpreters during the diagnosis process.
We recruited two participants who are native Chinese speakers and do not speak or understand English well, one participant in the medical industry, and one participant who had experience translating for medical appointments.
How could we train the AI to recognize the best solution from different translations submitted by crowdsourcing contributors?
Would it be helpful if the patient could double-check or type out the sentence before translation so that crowdsourcing contributors would not have to listen?
Could the professional translator also complete crowdsourcing tasks if they are available?
QUESTIONS
The user flow is clear to her, and she appreciates the usage of AI as a double-checking tool for translation.
Users could easily understand the expected interactions on each screen. They did not hesitate to click the desired action item, which proves the high usability of our design.
It is considered to have privacy settings during video translation
Users completed the tasks without error. They showed no confusion for the translated results and expressed high satisfaction for the user experience.
WORKED
PROBLEMS
They were challenging for the crowdsourcing contributors to recognize clearly if the patient has some accent.
For cases in which crowdsourcing staff could not understand the sentence correctly, they should have a way to reject the task.
IDEAS
Professional interpreters need to have access to previous communication before switching to understand the context better.
Speed adjustment and other customization for AI readers of the translated sentence.
Patients may have various needs for AI reading, including play speed, font size, gender of the AI, etc. Iit is necessary for AI to accommodate to their needs to make them feel comfortable.
INSIGHT
-
The interpreter needs to understand the context to translate accurately quickly.
-
LEP patients need more assurance about the accuracy of translating content.
-
Crowdsourcing staff should have a way to reject the task.
-
Patients have various needs for AI reading, including play speed, font size, gender of the AI, etc.
10
IMPROVEMENT AFTER Evaluation
PROFESSIONAL INTERPRETERS NEED TO KNOW THE CONTEXT AND DISCUSSION BEFORE SWITCHING.
PATIENTS NEED CUSTOMIZATION TO UNDERSTAND AI READING.
HOW TO TRAIN THE AI?
We expect the AI to train by a series of translation scripts for medical cases and some databases/dictionaries containing medical terms that are frequently used.
HOW TO SOLVE PROBLEMS DURING CROWDSOURCING?
If the crowdsourcing contributor could not recognize the sentence or do not know the correct translation:
They should have the option to reject the task.
If the five submitted translations for one sentence are significantly different from one another and AI could not decide on the optimal solution:
Our platform would hire a certain number of experts (professional interpreters) as the final decision-makers. They would receive unsolved tasks from the system under the circumstance described above and give the final solution.
If the professional translator would like to work on crowdsourcing tasks in the available time:
They could turn on their availability for crowdsourcing, and they would be able to receive tasks.
11
REFERENCE
[1]
Anastasiou, D., & Gupta, R. (2011). Comparison of Crowdsourcing Translation With Machine Translation. Journal of Information Science, 37(6), 637–659. https://doi.org/10.1177/0165551511418760
[2]
Blumenthal, D. S. (2007). Barriers to the Provision of Smoking Cessation Services Reported by Clinicians in Underserved Communities. The Journal of the American Board of Family Medicine, 20(3), 272–279. https://doi.org/10.3122/jabfm.2007.03.060115
[3]
Dehlendorf, C., Bryant, A. S., Huddleston, H. G., Jacoby, V. L., & Fujimoto, V. Y. (2010). Health Disparities: Definitions and Measurements. American journal of obstetrics and gynecology, 202(3), 212–213. https://doi.org/10.1016/j.ajog.2009.12.003
[4]
Driving Innovative Health Care Solutions for Underserved Populations. AAMC. (2019, July 10). Retrieved from https://www.aamc.org/news-insights/driving-innovative-health-care-solutions-underserved-populations.
[5]
Wilks, Yorick (1979). Machine Translation and Artificial Intelligence. Translating and the Computer, B.M.Snell (ed.), North-Holland Company 27-43.
[6]
Zaidan, Omar F. And Callison-Burch, Chris. Crowdsourcing Translation: Professional Quality from Non-Professionals. Proceedings of the 49th Annual Meeting of the Association for Computational Linguistics, Pages 1220-1229. Portland, Oregon, June 2011.