An overview of three recent papers on applying NLP to clinical settings.

So I attended EMNLP 2020, and in addition to one of the tutorials (blog post), I also attended the Clinical NLP Workshop which had many cool papers on the newest ways of applying NLP to clinical settings. In this post, I’ll be briefly discussing 3 papers that stood out to me.

Photo by Ilya Pavlov on Unsplash

1. Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures [Paper]

This paper focuses on text summarization in the context of medical dialogue. The idea is that when a patient has a conversation with a doctor, you want to be able to automatically summarize what transpired in the conversation. So for example, if the conversation went something like this:

Doctor


In the post-truth era, how can we use NLP to tackle fact-checking and fake news?

One of the tutorials at EMNLP 2020 is “Fact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era.” I attended this tutorial because I had done some previous research in this area and found it quite fascinating. Here are the primary points I learned that I think you’ll also find quite interesting.

Photo by AbsolutVision on Unsplash

Fake News is a Problem. In modern times, fake news has become a huge issue for many reasons. Not only has the public lost confidence in traditional media, but it also has low levels of critical thinking and news literacy which, combined with a shift…


A quick explanation of using curriculum learning for medical image analysis.

General Overview. In this study, I worked with a team of researchers to apply curriculum learning to improve the accuracy of a deep learning model for classifying colorectal cancer images. The full paper can be found here, and it is going to be published and presented at the 2021 Winter Conference on Applications of Computer Vision (WACV).

Proposed curriculum learning scheme for training a colorectal polyp classifier. The classifier first trains on easy images, and progressively-harder images are gradually added in subsequent stages.

The Motivation. Curriculum learning is an elegant idea inspired by human learning that proposes that deep learning models should be trained on examples in a specified order based on difficulty (typically easy examples and then hard examples), as opposed to random sampling. …


With all the recent advances in NLP, it’s refreshing to see new developments in real-world applications of the field.

A couple of years ago, I watched Google reveal their new Google Duplex system. If you’re not familiar with Duplex, it’s essentially an AI that helps complete tasks over the phone (e.g., scheduling an appointment, booking a reservation, etc.).

Photo by Mitchell Luo on Unsplash

The problem. Maybe it’s just my generation, but I’d rather text than have to make a phone call, as texting means I can send the text and quickly get it off my mind. This isn’t always possible, however, as many tasks tediously require making a phone call to a business. Here’s where Duplex comes in.

The idea. The overaching idea behind…


SemEval-2017’s Task 3 consists of five question-answering subtasks using a dataset of tens of thousands of questions and comments

One of the premier ways to evaluate NLP models, SemEval consists of several different tasks. Today we’ll be going over the 2017 version’s Task 3, including how the data for it was collected and also the five unique question-answering subtasks that it presents.

Overview of the dataset for SemEval-2017 Task 3. This year’s task includes data from previous year’s versions of this task, and these data are indicated in the second and third columns. Previous data was used as training data, with the newly-collected data being used as testing data (fourth column). Image credits to Nakov et al., the original authors of the SemEval-2017 Task 3 paper.

The Data. As SemEval is a pretty big competition, its data sources are well-gathered and thoroughly annotated. …


A quick guide to the Stanford Sentiment Treebank (SST), one of the most well-known datasets for sentiment analysis.

Published in 2013, “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank” presented the Stanford Sentiment Treebank (SST). SST is well-regarded as a crucial dataset because of its ability to test an NLP model’s abilities on sentiment analysis. Let’s go over this fascinating dataset.

Predicting levels of sentiment from very negative to very positive (- -, -, 0, +, ++) on the Stanford Sentiment Treebank. Image credits to Socher et al., the original authors of the paper.

The task. SST handles the crucial task of sentiment analysis in which models must analyze the sentiment of a text. For example, this could come in the form of determining whether restaurant reviews are positive or negative. Here are some made-up examples that display a range of positivity and negativity in their sentiment:

This was…


Embeddings from Language Models (ELMo) is a state-of-the-art language modeling idea. What makes it so successful?

Published in 2018, “Deep Contextualized Word Embeddings” presented the idea of Embeddings from Language Models (ELMo), which achieved state-of-the-art performance on many popular tasks including question-answering, sentiment analysis, and named-entity extraction. ELMo has been shown to yield performance improvements of up to almost 5%. But what makes this idea so revolutionary?

“Elmo Up Close” by creativedc is licensed under CC BY 2.0

What’s ELMo? Not only is he a Muppet, but ELMo is also a powerful computational model that converts words into numbers. This vital process allows machine learning models (which take in numbers, not words, as inputs) to be trained on textual data.

Why is ELMo so good? There are…


All the basic information you need to know about the Stanford Question Answering Dataset (SQuAD).

The Stanford Question Answering Dataset (SQuAD) is a set of question and answer pairs that present a strong challenge for NLP models. Whether you’re just interested in learning about a popular NLP dataset or planning to use it in one of your projects, here are all the basics you should know.

Photo by Emily Morter on Unsplash

What task does SQuAD present? As implied by its name, SQuAD focuses on the task of question answering. It tests a model’s ability to read a passage of text and then answer questions about it (flashback to reading comprehension on the SAT).


A fun application of state-of-the-art methods in natural language processing that demonstrates how far the field has come.

Natural language processing (NLP) as a field has seen unprecedented growth (especially in the past 2 years due to the publication of BERT). And while much research is focused on tasks with huge implications (e.g., question-answering, text summarization), we should always remember that there are some fun applications of natural language processing research as well. For example, generating puns.

Let’s delve into a recent paper by He et al. that proposes a pretty-strong model for pun generation. P.S. …


BERT is a language model that boasts high performance on many tasks. But what makes it so good?

BERT, which stands for Bidirectional Encoder Representations from Transformers, is a language model published in 2018 that achieved state-of-the-art performance on multiple tasks, including question-answering and language understanding. It not only beat previous state-of-the-art computational models, but also surpassed human performance in question-answering.

What’s BERT? BERT is a computational model that converts words into numbers. This process is crucial because machine learning models take in numbers (not words) as inputs, so an algorithm that converts words into numbers allows you to train machine learning models on your originally-textual data.

BERT is a computational model that converts words into numbers. Image from (Devlin et al., 2019).

What’s so great about BERT? For me, there are three main…

Jerry Wei

Interested in AI, specifically medical image analysis and natural language processing.

Get the Medium app

A button that says 'Download on the App Store', and if clicked it will lead you to the iOS App store
A button that says 'Get it on, Google Play', and if clicked it will lead you to the Google Play store