A neural network defines health as…

One of Mike Lynch’s bot creation is the Glossatory, which generates definitions of words based a recurrent neural network. Mike illustrates some of the more outlandish ones at @glossatory@weirder.earth

a neural net defines AUTHORITY, created by Glossatory/Mike Lynch 2019. Shows a drone-like machine looking over a rough sketch of a village.
a neural net defines AUTHORITY, created by Glossatory/Mike Lynch 2019. Source

The Glossatory is based on a recurrent neural network, which is the type of artificial neural network that’s typically used for machine translation. Mike used a training set of 82,115 definitions that were taken from WordNet. The results are often amusing, frequently incomprehensible, but mostly they’re just odd.

I love the Glossatory, but it also highlights some meaningful issues about the use of machine translation. Recurrent neural networks have almost entirely replaced earlier machine translation approaches, such as statistical machine translation . This is because neural networks require markedly less memory and the translations tend to be more accurate because the neural model is trained end-to-end . Google translate switched from using statistical machine translation to neural machine translation a few years ago, as have Microsoft and Yando .

While neural network machine translations might be more accurate than other approaches, they’re far from perfect—as Glossatory shows. Because of this, a lot of research activity has been focused on making translations more accurate and reducing processor demands .

Some colleagues and I have been taking a different tack. We’ve been looking at how translation apps and websites that are based on recurrent neural networks are already being used in health care settings. This is for a couple of reasons:

  • We’re diverse: many users of Australia health services come from culturally and linguistically distinct communities, and may have distinct cultural and health beliefs .
  • It’s not just about literacy: levels of health literacy (the ability to gain and understand health information to make decisions and follow instructions) vary markedly within the Australian community .
  • There are meaningful risks: the consequences of mistranslation in health care settings can have significant consequences. It’s not simply a matter of being given bad directions. People in health care settings may be expected to make decisions without understanding what they’re agreeing to—or expected to understand information that is simply incomprehensible.

For example, this is what Google translate does when you translate information on how to prepare for a colonoscopy into Nepali:

The original text
The Nepali translation provided by Google Translate
And the final re-translation back into English

There may be some compounding errors due to this being translated back and forth from Nepali, but it illustrates the underlying issue pretty well. Parenthetically, this translation is much better than the one that Google translation provided nine months ago, which was “You need to drink some fluids before your colopyopause that will open your eyes and remove from stool so that we can see your face face.” Machine translation are getting better, particularly for non-Romance languages that have been markedly less accurate in the past .

In response to these concerns, my colleagues and I have undertaken research looking at the use of machine translation in health services. We presented on this at last week’s Australasian Association for Academic Primary Care conference in Adelaide.

The slides below outline the findings of the first phase of our research, based on a survey of more than 1,500 health employees. The key findings are that machine translation is already being quite widely used, and it’s often clinicians who are initiating use.

The use of translation apps and websites in health care settings: Results of a survey across five NSW Local Health Districts

We’re currently undertaking qualitative research to further understand the dynamics of how machine translation is used. Further research is also needed on how the use of translation apps and websites is perceived by consumers, carers, and family members. We also need to examine if the use of machine translation plays out differently in primary health care, social care, and residential aged care.

Machine translation may present significant opportunities but we should resist the urge to uncritically adopt its use in all settings and for all purposes. Tomáš Svoboda expresses this well:

“Super-linguists” will be needed to trim the linguistic engines and, eventually, override the recommendations of machines. For a long time still, complex semantics and the interpretation of text passages will remain the art of human involvement in processing the input and output. Technology is not set to become autonomous in the medium term. It can serve people very well if applied with boldness and creativity, as well as with responsibility.

In the meantime we can continue to learn from the Glossatory:

References

Armentano-Oller, C., & Forcada, M. (2006). Open-source machine translation between small languages: Catalan and Aranese Occitan (p. 127). Presented at the Strategies for developing machine translation for minority languages, Genoa: 5th SALTMIL Workshop on Minority Languages. Retrieved from http://lrec.elra.info/proceedings/lrec2006/workshops/W06/SALTMIL.pdf#page=57 Cite
Australian Bureau of Statistics. (2019, April 29). 4364.0.55.014 - National Health Survey: Health Literacy, 2018. Retrieved July 20, 2019, from https://www.abs.gov.au/ausstats/abs@.nsf/mf/4364.0.55.014 Cite
Bahdanau, D., Cho, K., & Bengio, Y. (2014). Neural Machine Translation by Jointly Learning to Align and Translate. ArXiv:1409.0473 [Cs, Stat]. Retrieved from http://arxiv.org/abs/1409.0473 Cite
Birch, A., Abend, O., Bojar, O., & Haddow, B. (2016). HUME: Human UCCA-Based Evaluation of Machine Translation. ArXiv:1607.00030 [Cs]. Retrieved from http://arxiv.org/abs/1607.00030 Cite
Bouamor, H., Alshikhabobakr, H., Mohit, B., & Oflazer, K. (2014). A Human Judgement Corpus and a Metric for Arabic MT Evaluation. In Proceedings of the 2014 Conference on Empirical Methods in Natural Language Processing (EMNLP) (pp. 207–213). Doha, Qatar: Association for Computational Linguistics. https://doi.org/10.3115/v1/D14-1026 Cite
Koehn, P., Hoang, H., Birch, A., Callison-Burch, C., Federico, M., Bertoldi, N., … Herbst, E. (2007). Moses: Open Source Toolkit for Statistical Machine Translation. In Proceedings of the 45th Annual Meeting of the Association for Computational Linguistics Companion Volume Proceedings of the Demo and Poster Sessions (pp. 177–180). Prague, Czech Republic: Association for Computational Linguistics. Retrieved from https://www.aclweb.org/anthology/P07-2045 Cite
Lewis-Kraus, G. (2016, December 14). The Great A.I. Awakening. The New York Times. Retrieved from https://www.nytimes.com/2016/12/14/magazine/the-great-ai-awakening.html Cite
Macnamara, J., & Camit, M. (2017). Effective CALD community health communication through research and collaboration: an exemplar case study. Communication Research and Practice, 3(1), 92–112. https://doi.org/10.1080/22041451.2016.1209277 Cite
Mariño, R., Minichiello, V., Macentee, M. I., Minichiello, V., & Macentee, M. I. (2016). Families and ageing: intergenerational relations in health and care negotiations. In E. Liu & H. Easthope (Eds.), Multigenerational Family Living (p. 7 (14 pages)). London: Routledge. https://doi.org/10.4324/9781315596266-14 Cite
Patil, S., & Davies, P. (2014). Use of Google Translate in medical communication: evaluation of accuracy. The BMJ, 349. https://doi.org/10.1136/bmj.g7392 Cite
Sutskever, I., Vinyals, O., & Le, Q. V. (2014). Sequence to Sequence Learning with Neural Networks. ArXiv:1409.3215 [Cs]. Retrieved from http://arxiv.org/abs/1409.3215 Cite
Svoboda, T. (2017). No linguistic borders ahead? Looking beyond the knocked-down language barrier. TranscUlturAl: A Journal of Translation and Cultural Studies, 9(2), 86–108. https://doi.org/10.21992/T93Q0F Cite