On Simultaneous Meanings
To the extent that the Mitch Hedberg quote above is funny, the humor resides in the multiple meanings of the initial sentence “I used to do drugs”. The more obvious meaning of that sentence is a statement about the past where the phrasing suggests that the present is drug free. The bottom part of the meme shows a drug-filled present, and we reread the original and see the other meaning as equally grammatical.
The humor resides in understanding both meanings simultaneously. If a person sees only one meaning, no humor. But equally importantly, if they read the later part and come back and just replace the previous meaning (not seeing both meanings at the same time), it will still not occur to them as funny.
Thus, when I say “simultaneous meanings” I do not mean mere ambiguity. An ambiguous sentence such as “I love Jaguars” may be talking about the speaker liking the cars, the animals, the Jacksonville Jaguars, the guitar, the Aztec warriors, or other things that go by that name. Presumably, though, the speaker has one of these senses in mind, although the listener may not know which. By contrast, when Comedian Mitch Hedberg says during a stand-up routine the line above about doing drugs, the speaker most certainly has both senses in mind and hopes the audience will, too. By “multiple meanings” I only refer to cases where multiple senses are juxtaposed.
Implications for NLP
Long long ago, around 2014 or so, when Word2Vec’s star was ascendant, NLP had a habit of representing word meaning in a vector space. The word Jaguar was one point in this space, and that was what Jaguar meant, thank you. There was no need to deal with the multiple meanings of the term.
As NLP made giant strides, with a long succession of Sesame Street Characters (starting with ELMo in 2018), NLP got better at contextualizing meaning. That is, when Jaguar was seen with the word cubs or with the word dealer, the vector for Jaguar would be different: the car sense could now be separated from the animal sense.
Progress! However, although we have moved from ignoring ambiguity to dealing with ambiguity, we are still making the assumption of one sense per utterance: a particular sentence in a particular paragraph has one meaning. This “contextualized” vector is still incapable of dealing with multiple simultaneous meanings.
My frustration with NLP research today is that it oversimplifies. The problems NLP aims to tackle are complex and unruly, but the tools with which to tackle them and the formalisms of how to represent the problem rob it of all the essential character. I have complained previously of work on humor making assumptions that a sentence has at most one pun or that a thoughtful and nuanced 1000-word movie review can be boiled down to a thumbs up or thumbs down. All this simplification is done since it is much easier to fit the problems (where is the pun? What is the sentiment of this movie review?) as a nice mathematical function where we get to bring out and play with our fancy mathematical toys. It worked for Physics, surely it will work for NLP?
The assumption that a sentence in context has a single meaning fits this mathematization-at-the-cost-of-fidelity mold. If a sentence can have multiple meanings, the function mapping a sentence to a meaning ain’t a function no more. How then can we use our neural networks, which are really function approximators?
If multiple meanings are pervasive, we would have to tackle that problem before we claim victory over language understanding.
How prevalent are multiple meanings?
It is easy to brush off this challenge to NLP by classifying it as a rare phenomena. I think it is highly prevalent, and I will given three classes of examples, starting with rather esoteric and moving to the omnipresent.
The Esoteric: Multiple Meanings in Multiple “Languages”
I use the word “Languages” rather loosely here, as a way of saying “different mechanisms for extracting the meaning”. Here I will include things such as this Sanskrit Shloka (of recent antiquity, circa 1900 A.D., despite being claimed by the probable author as being Vedic)
गोपीभाग्यमधुव्रात शृङ्गिशोदधिसंधिग
खलजीवितखाताव गलहालारसन्धर
This has three well-known parallel meanings. There are two literal meanings, as a devotional hymn to either Krishna or as a hymn to the god Shankar. The third meaning originates by code: there is a way of encoding numbers as Devanagari letters (think of this as A through I being 1 through 9, J through R also 1 through 9, S through W being 1 to 5, and X, Y and Z being 0). Using this, we get a 32-digit number, that begins 31415926. In this (admittedly esoteric) sense, one of the meanings is the mathematical number “pi”.
Another example I am rather proud of is my simultaneous invention (aka discovery) of a thousand Knock-knock jokes:
Knock knock.
Who’s there?
Hungry.
Hungry who
तो कुछ खा लो
(For the non-Hindi readers: the word who sounds like the Hindi हूँ, which means “I am”. “Hungry who” means “I am hungry”. You can guess the remaining 999 jokes in this family).
The Intentional: Word Play
Many names of creative works often have names that depend on multiple meanings for their rich texture and allure. The NPR show “All things considered” has a name that means both that it considers all topics and it considers these from all angles. Many songs owe their popularity to the double entendre.
An example of an extended stretch of multiple meanings comes from the movie “My Cousin Vinny” when Stan is in jail and is visited by his lawyer, Vinny, who Stan has never met. Vinny is helping Stan without a fee, and assumes Stan would be grateful. Vinny, however, has been led to believe that new prisoners are subjected to sexual assault, and assumes that Vinny is the would-be assaulter. The conversation that ensues is of course full of sentences with multiple meanings. The movie’s transcript is here, and the relevant portion is pages 23 through 26. One quite typical example is when Vinny expresses sympathy about Stan being in jail and promises to get him free soon. But of course what Stan hears is something different. What Vinny says “I don’t blame you. If I was in your situation, I’d want to get through this whole thing over as quickly and with as little pain as possible. So let’s try our best to make this thing a simple in and out procedure.”
Word play is frequent in real text. But now we move to the final example, where parallel meanings are everywhere.
The ubiquitous: Metafunction in Systemic Functional Linguistics
Research papers about embeddings usually cite J. R. Firth and his quote “You shall know a word by the company it keeps”. Here I am interested in the work of his most famous student, Michael Halliday, who invented Systemic Functional Linguistics (SFL).
Although I find SFL exceedingly baroque, I find compelling the Systemic Functional Linguists’ emphasis that every conversation has many meanings happening in parallel. They identify three kinds of meanings: ideational, interpersonal and textual, and these kinds of meanings they call metafunctions.
In a nutshell, “ideational” meaning roughly corresponds to the “information content” of the utterance. Interpersonal meaning concerns the roles and relative stature of the speakers. Consider Sentence A uttered by John addressed to Mike: “Is your father home?” Now consider sentence B, again by John to Mike: “Is your daddy home?”. Ideationally, they have nearly the same information. By contrast, the interpersonal meaning differs: sentence B, but not sentence A, somewhat suggests that Mike is a child.
Meaning can be carried not just in the literal words but the conveyed mood and register. The same informational content can be said in a formal, informal or intimate register; can be said calmly or in anger or in derision. All these meanings are present at the same time.
NLP systems need to represent simultaneous meaning
There are systems that can consider and represent multiple parallel interpretations. These have been build mostly for micro domains, but could in theory be extended to wider language. One example of such a system is Melanie Mitchell’s CopyCat, which solves letter analogy puzzles. During processing, it can entertain conflicting interpretations of parts of its inputs using a Blackboard architecture.
An analogy used to explain how Blackboard architectures work involves a group of detectives coordinating solving a crime. As they find clues, they add it to a shared board. Conflicting facts can easily be added. The detectives occasionally stare at the board and notice additional connections that they can also add as a note to the board, and over time, enough could be accumulated to solve the crime.
We need something similar for language understanding.
