Emoji in eDiscovery: Technical and Interpretive Challenges
-min.avif)
Matthew Verga is an attorney, consultant, and eDiscovery expert proficient at leveraging his legal experience, his technical knowledge, and his communication skills to make complex eDiscovery topics accessible to diverse audiences. A fifteen-year industry veteran, Matthew has worked across every phase of the EDRM and at every level, from the project trenches to enterprise program design. As Director of Education for Consilio, he leverages this background to produce engaging educational content to empower practitioners at all levels with knowledge they can use to improve their projects, their careers, and their organizations.
As mobile sources, collaboration tools, and other messaging services have rapidly increased in number and importance, practitioners have begun to wrestle with emoji in eDiscovery
As smartphones and social media communication channels have become more frequent sources, so too have emoji shown up more frequently in cases. In 2019, Santa Clara University Professor of Law Eric Goldman published “Emojis and the Law” in the Washington Law Review, which revealed that “[b]etween 2004 and 2019, there was an exponential rise in emoji and emoticon references in US court opinions, with over 30 percent of all cases appearing in 2018” [emphasis added]. Examples range from landlord disputes to sex trafficking cases.
As they increase in frequency, emoji are creating special challenges for eDiscovery and litigation – both technical challenges and challenges of interpretation.
What Are Emoji and Why Do They Matter?
First came emoticons, which were representations of different facial expressions created using punctuation and other characters, like the classic smiley face 🙂 or winking face 😉. These representations could be used to add an indication of tone or emotion to a short text message, or as a reaction message by themselves.
As mobile devices and the software running on them became more sophisticated, emoticons were supplanted by emoji, which replaced the punctuation-based representations with tiny images of assorted facial expressions and, eventually, other things.
Adoption was rapid and widespread. In 2015, an emoji was selected was selected as Oxford Dictionaries’ word of the year (“the crying-from-laughter / crying-from-happiness emoji — officially known as the Face with Tears of Joy emoji”). By 2019, according to a survey report from Adobe, more than 90% were using emoji in personal communications, and more than 60% were using emoji in work communications. For example, according to a Microsoft spokeswoman, emoji use in 2019 was “basically universal” among the 13 million daily active users of Microsoft Teams. And, as noted above, the dramatic increase in their usage has led to a corresponding increase in their appearance in litigation.
What Technical Challenges Do They Cause?
Emoji carry with them a number of technical challenges for discovery and litigation. First, there are many of them, and the number grows every year. Today, there are more than 3,500 emoji recognized by the Unicode Consortium. Beyond those cross-platform emoji, many platforms also include platform-specific emoji or allow for the creation of custom emoji. In popular collaboration tool Slack, for example, “26 million custom emojis have been created since the feature was introduced.” This makes it a challenge for discovery tool developers attempting to support emoji to keep up with the increasing volume and growing diversity of emoji that may need to be collected, reviewed, and produced.
These myriad emoji also work in different ways. The cross-platform emoji recognized by the Unicode Consortium exist as alphanumeric codes that various software knows to replace by displaying a corresponding image. This is the same way that the display of various currency, language, and technical symbols has been handled since Unicode was first introduced in 1991. The platform-specific and user-created emoji, however, may be based on custom, platform-specific codes, or they may exist only as image files that function more like attachments. This further complicates the development challenges for makers of discovery tools.
These same challenges also affect legal practice in ways beyond just discovery tool development challenges. As Professor Goldman said:
Unfortunately, opinions still struggle with displaying Emoji. Opinions routinely omit the emoji altogether, or the judge imprecisely characterizes the emoji in evidence. (It doesn’t help to call an emoji a ‘smiley’ because there are a dozen different smiling emoji symbols). Furthermore, Westlaw and Lexis still usually do not display emoji, and neither makes it possible to search for emoji in court opinions.
What Interpretation Challenges Do They Cause?
Emoji create a few different interpretation challenges. First, communicating through an image or a series of images can be more ambiguous than communicating through words and sentences. It may not be clear to you later what a particular custodian in your case was attempting to communicate with a particular string of Emoji. Moreover, the message recipient’s interpretation may have differed from the message sender’s intention.
Second, there is a challenge associated with the contextual relationship between emoji and text when they are used together. As noted above, not all discovery tools are able to capture or display all the different emoji or all the different types of emoji, which can lead to material alterations to messages and their meaning. For example, a message might include an emoji indicating it was intended humorously or sarcastically. If that emoji is omitted during collection or not displayed during review, the message might appear misleadingly serious or literal.
On top of those issues, there is an additional challenge created by the way that the Unicode-based cross-platform emoji work. Although which emoji matches with which code is standardized, each developer gets to create its own version of the images that will actually be displayed in that software or on that device, so the same emoji will look different depending on where it is viewed. Apple, Google, Samsung, LG, and Microsoft, as well as Facebook, Twitter, Snapchat, and Instagram, all have their own versions of the Unicode emoji. For example:
These variations can be inconsequential, or they can exacerbate both of the interpretation challenges described above, depending on which emoji were used and how much those particular emoji vary in appearance across platforms. In some instances, the sender, recipient, and later reviewers might all be seeing different things that suggest different connotations.
In What Kinds of Cases Have Emoticons or Emoji Appeared?
Emoticons and emoji have shown up in mobile, social, and email communications in a wide variety of case types over the past decade, such as:
- In the case of Lenz v. Universal Music Corp., No. C 07-3783 JF (N.D. Cal. Feb. 25, 2010), the plaintiff won partial summary judgment, in part, because a winking smiley face emoticon was taken, in context, as an indication of the plaintiff’s humor or sarcasm about the “stilted language” of lawyers.
- In the case of Elonis v. United States, in a February 14, 2014, Petition for a Writ of Certiorari to the Supreme Court, the petitioner argued, inter alia, that an allegedly threatening Facebook status update was “meant in ‘jest’” as indicated by his having “ended the post with an ‘emoticon’ of a face sticking its tongue out.”
- In the disability discrimination and retaliation case of Apatoff v Munich Re Servs. Inc., No. 11-7570 (RBK/KMW) (D.N.J. August 1, 2014), the plaintiff survived summary judgment on certain claims, in part, because of emoticons used in email messages:
The Court believes that a reasonable jury could find that the “emoticons,” attached to the emails of two Munich Re managers late in the day on which Plaintiff was terminated, are evidence that the decision-makers at Munich Re were happy to be able to terminate Plaintiff.
- In the case of United States v. Ulbricht, No. 14-cr-68 (KBF) (S.D.N.Y. Jan. 13, 2015), a dispute arose over messages being read aloud to the jury without the inclusion of emoticons. The judge emphasized “[t]hat is part of the evidence of the document” and that it was important for the jury to get all of the relevant details for comprehension: “The jury should note the punctuation and emoticons” [emphasis added].
- In February 2015, a grand jury declined to indict a teenager for making a terroristic threat on Facebook by “posting an emoji of a police officer with three guns pointing at the cop.”
- In 2017, in a small-claims court in Tel Aviv, Israel, a judge found that emoji used in a text message to a prospective landlord indicated intent:
The…text message sent by Defendant…included a smiley, a bottle of champagne, dancing figures and more. These icons convey great optimism. Although this message did not constitute a binding contract between the parties, [it] naturally led to the Plaintiff’s great reliance on the Defendants’ desire to rent his apartment…These symbols, which convey to the other side that everything is in order, were misleading.”
- In the sexual harassment case of Harrison v. City of Tampa, No. 8:17-cv-01369-T-02CPT (M.D. Fla. Jun. 4, 2019), evidence included messages in which the plaintiff’s supervisor “sent her a number of emojis that can be read to indicate that [he] was romantically attracted to Plaintiff,” including “emojis that show a face kissing, a face with hearts for eyes, and what appears to be a smiling dog with hearts next to it.”
As mobile sources, collaboration tools, and other messaging services have rapidly increased in number and importance, practitioners have begun to wrestle with emoji in eDiscovery
As smartphones and social media communication channels have become more frequent sources, so too have emoji shown up more frequently in cases. In 2019, Santa Clara University Professor of Law Eric Goldman published “Emojis and the Law” in the Washington Law Review, which revealed that “[b]etween 2004 and 2019, there was an exponential rise in emoji and emoticon references in US court opinions, with over 30 percent of all cases appearing in 2018” [emphasis added]. Examples range from landlord disputes to sex trafficking cases.
As they increase in frequency, emoji are creating special challenges for eDiscovery and litigation – both technical challenges and challenges of interpretation.
What Are Emoji and Why Do They Matter?
First came emoticons, which were representations of different facial expressions created using punctuation and other characters, like the classic smiley face 🙂 or winking face 😉. These representations could be used to add an indication of tone or emotion to a short text message, or as a reaction message by themselves.
As mobile devices and the software running on them became more sophisticated, emoticons were supplanted by emoji, which replaced the punctuation-based representations with tiny images of assorted facial expressions and, eventually, other things.
Adoption was rapid and widespread. In 2015, an emoji was selected was selected as Oxford Dictionaries’ word of the year (“the crying-from-laughter / crying-from-happiness emoji — officially known as the Face with Tears of Joy emoji”). By 2019, according to a survey report from Adobe, more than 90% were using emoji in personal communications, and more than 60% were using emoji in work communications. For example, according to a Microsoft spokeswoman, emoji use in 2019 was “basically universal” among the 13 million daily active users of Microsoft Teams. And, as noted above, the dramatic increase in their usage has led to a corresponding increase in their appearance in litigation.
What Technical Challenges Do They Cause?
Emoji carry with them a number of technical challenges for discovery and litigation. First, there are many of them, and the number grows every year. Today, there are more than 3,500 emoji recognized by the Unicode Consortium. Beyond those cross-platform emoji, many platforms also include platform-specific emoji or allow for the creation of custom emoji. In popular collaboration tool Slack, for example, “26 million custom emojis have been created since the feature was introduced.” This makes it a challenge for discovery tool developers attempting to support emoji to keep up with the increasing volume and growing diversity of emoji that may need to be collected, reviewed, and produced.
These myriad emoji also work in different ways. The cross-platform emoji recognized by the Unicode Consortium exist as alphanumeric codes that various software knows to replace by displaying a corresponding image. This is the same way that the display of various currency, language, and technical symbols has been handled since Unicode was first introduced in 1991. The platform-specific and user-created emoji, however, may be based on custom, platform-specific codes, or they may exist only as image files that function more like attachments. This further complicates the development challenges for makers of discovery tools.
These same challenges also affect legal practice in ways beyond just discovery tool development challenges. As Professor Goldman said:
Unfortunately, opinions still struggle with displaying Emoji. Opinions routinely omit the emoji altogether, or the judge imprecisely characterizes the emoji in evidence. (It doesn’t help to call an emoji a ‘smiley’ because there are a dozen different smiling emoji symbols). Furthermore, Westlaw and Lexis still usually do not display emoji, and neither makes it possible to search for emoji in court opinions.
What Interpretation Challenges Do They Cause?
Emoji create a few different interpretation challenges. First, communicating through an image or a series of images can be more ambiguous than communicating through words and sentences. It may not be clear to you later what a particular custodian in your case was attempting to communicate with a particular string of Emoji. Moreover, the message recipient’s interpretation may have differed from the message sender’s intention.
Second, there is a challenge associated with the contextual relationship between emoji and text when they are used together. As noted above, not all discovery tools are able to capture or display all the different emoji or all the different types of emoji, which can lead to material alterations to messages and their meaning. For example, a message might include an emoji indicating it was intended humorously or sarcastically. If that emoji is omitted during collection or not displayed during review, the message might appear misleadingly serious or literal.
On top of those issues, there is an additional challenge created by the way that the Unicode-based cross-platform emoji work. Although which emoji matches with which code is standardized, each developer gets to create its own version of the images that will actually be displayed in that software or on that device, so the same emoji will look different depending on where it is viewed. Apple, Google, Samsung, LG, and Microsoft, as well as Facebook, Twitter, Snapchat, and Instagram, all have their own versions of the Unicode emoji. For example:
These variations can be inconsequential, or they can exacerbate both of the interpretation challenges described above, depending on which emoji were used and how much those particular emoji vary in appearance across platforms. In some instances, the sender, recipient, and later reviewers might all be seeing different things that suggest different connotations.
In What Kinds of Cases Have Emoticons or Emoji Appeared?
Emoticons and emoji have shown up in mobile, social, and email communications in a wide variety of case types over the past decade, such as:
- In the case of Lenz v. Universal Music Corp., No. C 07-3783 JF (N.D. Cal. Feb. 25, 2010), the plaintiff won partial summary judgment, in part, because a winking smiley face emoticon was taken, in context, as an indication of the plaintiff’s humor or sarcasm about the “stilted language” of lawyers.
- In the case of Elonis v. United States, in a February 14, 2014, Petition for a Writ of Certiorari to the Supreme Court, the petitioner argued, inter alia, that an allegedly threatening Facebook status update was “meant in ‘jest’” as indicated by his having “ended the post with an ‘emoticon’ of a face sticking its tongue out.”
- In the disability discrimination and retaliation case of Apatoff v Munich Re Servs. Inc., No. 11-7570 (RBK/KMW) (D.N.J. August 1, 2014), the plaintiff survived summary judgment on certain claims, in part, because of emoticons used in email messages:
The Court believes that a reasonable jury could find that the “emoticons,” attached to the emails of two Munich Re managers late in the day on which Plaintiff was terminated, are evidence that the decision-makers at Munich Re were happy to be able to terminate Plaintiff.
- In the case of United States v. Ulbricht, No. 14-cr-68 (KBF) (S.D.N.Y. Jan. 13, 2015), a dispute arose over messages being read aloud to the jury without the inclusion of emoticons. The judge emphasized “[t]hat is part of the evidence of the document” and that it was important for the jury to get all of the relevant details for comprehension: “The jury should note the punctuation and emoticons” [emphasis added].
- In February 2015, a grand jury declined to indict a teenager for making a terroristic threat on Facebook by “posting an emoji of a police officer with three guns pointing at the cop.”
- In 2017, in a small-claims court in Tel Aviv, Israel, a judge found that emoji used in a text message to a prospective landlord indicated intent:
The…text message sent by Defendant…included a smiley, a bottle of champagne, dancing figures and more. These icons convey great optimism. Although this message did not constitute a binding contract between the parties, [it] naturally led to the Plaintiff’s great reliance on the Defendants’ desire to rent his apartment…These symbols, which convey to the other side that everything is in order, were misleading.”
- In the sexual harassment case of Harrison v. City of Tampa, No. 8:17-cv-01369-T-02CPT (M.D. Fla. Jun. 4, 2019), evidence included messages in which the plaintiff’s supervisor “sent her a number of emojis that can be read to indicate that [he] was romantically attracted to Plaintiff,” including “emojis that show a face kissing, a face with hearts for eyes, and what appears to be a smiling dog with hearts next to it.”
