CVE-2023-5217 Explained: VP8 and Memory Buffer Overflow

By Jared McCann on December 8, 2023

Introduction

CVE-2023-5217 is a recently identified exploit of the VP8 codec within the libvpx library. This exploit relies on memory corruption and heap buffer overflow. By utilizing a crafted HTML page, an attacker can exploit the library to inject malware, gain remote elevated access privileges, and execute other exploits. While first detected by Google within its Chrome software [1], this vulnerability affects numerous different software packages that depend upon the libvpx library. This includes all Chromium-based browsers, Firefox, and hundreds of Ubuntu and Debian packages [2].

How the exploit works

Libvpx is a software video codec library that is free and provided by Google. [3] It provides several encoding formats, including the VP8 codec, which this exploit utilizes. In this exploit, an attacker uses a crafted HTML page to exploit heap corruption. In this case, the heap corruption involves out-of-bounds writing and memory buffer overflow.

Figure 1: C Code over allocating the buffer

In the above example, the set array is of size 3 with valid indices 0 to 2. Assigning to index 3 is an example of out-of-bounds and will return an error. [4]

Figure 2: C code allocating the memory buffer size

This example is of a code setting the memory buffer size manually. This can then be exploited since the size is set and a code injection simply needs to use up more space than the buffer. [5] These are just basic examples of out-of-bounds writing and memory buffer overflow, though they illustrate the principles of both.

In this exploit, it is likely an attacker utilizes a vulnerability within the VP8 codec allowing for a video stream to send more data than the memory buffer can keep up with. This then allows for out-of-bounds write and for the attacker to perform a code injection gaining more access. From here, an attacker would likely pivot through the system and deploy a remote access trojan (RAT) to gain persistent access. This exploit may rely on a return-to-libc attack after the memory buffer overflow.

Return-to-libc - A possible pathway

Because libvpx is a library written in C and assembly code, it relies on the C standard library (libc). This library is present in many systems, including Windows. The return-to-libc attack relies on a buffer overflow to escape the memory stack and inject a new function. [6] This is done by overflowing the memory buffer and changing the return address to a new location. Then, once we have a new address, we can insert a command argument. In the case of the VP8 exploit, this is likely shell code that allows for a port to be opened and a reverse shell or RAT to be injected into the system.

As NIST and Google have not released more details of this attack, a deeper analysis of libvpx’s source code is required to determine the exact vulnerability within VP8 and what attack type is being used.

Recommendations

Because this is a known exploited vulnerability, ensure all software using libvpx is up to date, and any packages and dependencies are using the latest version of libvpx. Follow basic online safety and do not follow suspicious links or visit suspicious web pages. Also, avoid or stop using any software that cannot be updated with the newest version of libvpx, where possible.

References

[1] National Vulnerability Database, “CVE-2023-5217 Detail,” Sept. 2023. https://nvd.nist.gov/vuln/detail/CVE-2023-5217

[2] Dan Gooding, “A new Chrome 0-day is sending the Internet into a new chapter of Groundhog Day,” Sept. 2023. https://arstechnica.com/security/2023/09/new-0-day-in-chrome-and-firefox-is-likely-to-plague-other-software/

[3] Google Source, “libvpx changelog,” May 2010. https://chromium.googlesource.com/webm/libvpx/+/master/CHANGELOG

[4] Mitre, “CWE-787: Out-of-bounds,” Oct. 2009. https://cwe.mitre.org/data/definitions/787.html

[5] Mitre, “CWE-121: Heap-based Buffer Overflow,” Jul. 2006. https://cwe.mitre.org/data/definitions/122.html

[6] c0ntex, “Bypassing non-executable-stack during exploitation using return-to-libc,” 2014. https://css.csail.mit.edu/6.858/2014/readings/return-to-libc.pdf