An overview of three recent papers on applying NLP to clinical settings.
So I attended EMNLP 2020, and in addition to one of the tutorials (blog post), I also attended the Clinical NLP Workshop which had many cool papers on the newest ways of applying NLP to clinical settings. In this post, I’ll be briefly discussing 3 papers that stood out to me.
1. Dr. Summarize: Global Summarization of Medical Dialogue by Exploiting Local Structures [Paper]
This paper focuses on text summarization in the context of medical dialogue. The idea is that when a patient has a conversation with a doctor, you want to be able to automatically summarize what transpired in the conversation. So for example, if the conversation went something like this:
Doctor…
In the post-truth era, how can we use NLP to tackle fact-checking and fake news?
One of the tutorials at EMNLP 2020 is “Fact-Checking, Fake News, Propaganda, and Media Bias: Truth Seeking in the Post-Truth Era.” I attended this tutorial because I had done some previous research in this area and found it quite fascinating. Here are the primary points I learned that I think you’ll also find quite interesting.
Fake News is a Problem. In modern times, fake news has become a huge issue for many reasons. Not only has the public lost confidence in traditional media, but it also has low levels of critical thinking and news literacy which, combined with a shift…
A quick explanation of using curriculum learning for medical image analysis.
General Overview. In this study, I worked with a team of researchers to apply curriculum learning to improve the accuracy of a deep learning model for classifying colorectal cancer images. The full paper can be found here, and it is going to be published and presented at the 2021 Winter Conference on Applications of Computer Vision (WACV).
The Motivation. Curriculum learning is an elegant idea inspired by human learning that proposes that deep learning models should be trained on examples in a specified order based on difficulty (typically easy examples and then hard examples), as opposed to random sampling. …
With all the recent advances in NLP, it’s refreshing to see new developments in real-world applications of the field.
A couple of years ago, I watched Google reveal their new Google Duplex system. If you’re not familiar with Duplex, it’s essentially an AI that helps complete tasks over the phone (e.g., scheduling an appointment, booking a reservation, etc.).
The problem. Maybe it’s just my generation, but I’d rather text than have to make a phone call, as texting means I can send the text and quickly get it off my mind. This isn’t always possible, however, as many tasks tediously require making a phone call to a business. Here’s where Duplex comes in.
The idea. The overaching idea behind…
SemEval-2017’s Task 3 consists of five question-answering subtasks using a dataset of tens of thousands of questions and comments
One of the premier ways to evaluate NLP models, SemEval consists of several different tasks. Today we’ll be going over the 2017 version’s Task 3, including how the data for it was collected and also the five unique question-answering subtasks that it presents.
The Data. As SemEval is a pretty big competition, its data sources are well-gathered and thoroughly annotated. …
A quick guide to the Stanford Sentiment Treebank (SST), one of the most well-known datasets for sentiment analysis.
Published in 2013, “Recursive Deep Models for Semantic Compositionality Over a Sentiment Treebank” presented the Stanford Sentiment Treebank (SST). SST is well-regarded as a crucial dataset because of its ability to test an NLP model’s abilities on sentiment analysis. Let’s go over this fascinating dataset.
The task. SST handles the crucial task of sentiment analysis in which models must analyze the sentiment of a text. For example, this could come in the form of determining whether restaurant reviews are positive or negative. Here are some made-up examples that display a range of positivity and negativity in their sentiment:
This was…
Embeddings from Language Models (ELMo) is a state-of-the-art language modeling idea. What makes it so successful?
Published in 2018, “Deep Contextualized Word Embeddings” presented the idea of Embeddings from Language Models (ELMo), which achieved state-of-the-art performance on many popular tasks including question-answering, sentiment analysis, and named-entity extraction. ELMo has been shown to yield performance improvements of up to almost 5%. But what makes this idea so revolutionary?
What’s ELMo? Not only is he a Muppet, but ELMo is also a powerful computational model that converts words into numbers. This vital process allows machine learning models (which take in numbers, not words, as inputs) to be trained on textual data.
Why is ELMo so good? There are…
All the basic information you need to know about the Stanford Question Answering Dataset (SQuAD).
The Stanford Question Answering Dataset (SQuAD) is a set of question and answer pairs that present a strong challenge for NLP models. Whether you’re just interested in learning about a popular NLP dataset or planning to use it in one of your projects, here are all the basics you should know.
What task does SQuAD present? As implied by its name, SQuAD focuses on the task of question answering. It tests a model’s ability to read a passage of text and then answer questions about it (flashback to reading comprehension on the SAT). …
A fun application of state-of-the-art methods in natural language processing that demonstrates how far the field has come.
Natural language processing (NLP) as a field has seen unprecedented growth (especially in the past 2 years due to the publication of BERT). And while much research is focused on tasks with huge implications (e.g., question-answering, text summarization), we should always remember that there are some fun applications of natural language processing research as well. For example, generating puns.
Let’s delve into a recent paper by He et al. that proposes a pretty-strong model for pun generation. P.S. …
BERT is a language model that boasts high performance on many tasks. But what makes it so good?
BERT, which stands for Bidirectional Encoder Representations from Transformers, is a language model published in 2018 that achieved state-of-the-art performance on multiple tasks, including question-answering and language understanding. It not only beat previous state-of-the-art computational models, but also surpassed human performance in question-answering.
What’s BERT? BERT is a computational model that converts words into numbers. This process is crucial because machine learning models take in numbers (not words) as inputs, so an algorithm that converts words into numbers allows you to train machine learning models on your originally-textual data.
What’s so great about BERT? For me, there are three main…
Jerry Wei
Interested in AI, specifically medical image analysis and natural language processing.
