Build what's next on GitHub, the place for anyone from anywhere to build anything.
Join us October 28-29 in San Francisco or online for GitHub Universe, our flagship developer event uniting people, agents, and the world's code.
In this third and last part, I’ll share the results of my research on Apache HTTP server, and I’ll show some of the vulnerabilities that I’ve found.
In the first part of this series, I explained my fuzzing workflow and covered some of the custom mutators I’ve built for fuzzing Apache HTTP. In the second part, I explained how to build custom ASAN interceptors in order to catch memory bugs when custom memory pools are used.
In this third and last part, I’ll share the results of my research on Apache HTTP server, and I’ll show some of the vulnerabilities that I’ve found.
So, let’s get to it!
This bug can be triggered setting a cookie with a NULL key and value:
In the example above, you can see that in the first position of the cookies there is a session key and a choco value. In the second position, we can find the admin-user key and the number 2 as a value. However, in the third position there is an empty key and value pair.
What’s the problem here? Well, if you look at the following code snippet, you can see two calls to apr_strtok, to extract the first and the second string (key and value):
const char *psep = "="; char *key = apr_strtok(pair, psep, &plast); char *val = apr_strtok(NULL, psep, &plast);
const char *psep = "=";
char *key = apr_strtok(pair, psep, &plast);
char *val = apr_strtok(NULL, psep, &plast);
Let’s see now what happens in the apr_strtok function when the first argument is NULL:
APR_DECLARE(char *) apr_strtok(char *str, const char *sep, char **last)
{
char *token;
if (!str)
str = *last;
while (*str && strchr(sep, *str))
++str;
APR_DECLARE(char *) apr_strtok(char *str, const char *sep, char **last)
{
char *token;
if (!str)
str = *last;
while (*str && strchr(sep, *str))
++str;
You can see in this code snippet how the while loop tries to dereference the first function argument (str pointer). So, if this first argument is NULL, it will trigger a NULL dereference bug. Also,this is exactly what happens with the following statement char *val = apr_strtok(NULL, psep, &plast); when the previous key value is also null.
In order to exploit this bug, mod_session needs to be enabled. This vulnerability can lead to a denial of service at the child level, affecting the other threads in the same process.
In order to exploit this bug, the mod_auth_digest module should be enabled, and the application has to be using the DIGEST authentication.
For triggering this bug, we need to assign a specific set of values to the nonce field as follows:
As you can see, the nonce field is a BASE64 value. In order to decode this value, the check_nonce function does a call to:
where nonce_time.arr is a local array of size 8. Let’s see the code of the apr_base64_decode_binary function:
APR_DECLARE(int) apr_base64_decode_binary(unsigned char *bufplain, const char *bufcoded)
{
int nbytesdecoded;
register const unsigned char *bufin;
register unsigned char *bufout;
register apr_size_t nprbytes;
bufin = (const unsigned char *) bufcoded;
while (pr2six[*(bufin++)] <= 63);
nprbytes = (bufin - (const unsigned char *) bufcoded) - 1;
nbytesdecoded = (((int)nprbytes +3) / 4) * 3;
bufout = (unsigned char *) bufplain;
bufin = (const unsigned char *) bufcoded;
while (nprbytes > 4) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
bufin += 4;
nprbytes -= 4;
}
if (nprbytes > 1) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
}
if (nprbytes > 2) {
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
}
if (nprbytes > 3) {
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
}
APR_DECLARE(int) apr_base64_decode_binary(unsigned char *bufplain, const char *bufcoded)
{
int nbytesdecoded;
register const unsigned char *bufin;
register unsigned char *bufout;
register apr_size_t nprbytes;
bufin = (const unsigned char *) bufcoded;
while (pr2six[*(bufin++)] <= 63);
nprbytes = (bufin - (const unsigned char *) bufcoded) - 1;
nbytesdecoded = (((int)nprbytes +3) / 4) * 3;
bufout = (unsigned char *) bufplain;
bufin = (const unsigned char *) bufcoded;
while (nprbytes > 4) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
bufin += 4;
nprbytes -= 4;
}
if (nprbytes > 1) {
*(bufout++) =
(unsigned char) (pr2six[*bufin] << 2 | pr2six[bufin[1]] >> 4);
}
if (nprbytes > 2) {
*(bufout++) =
(unsigned char) (pr2six[bufin[1]] << 4 | pr2six[bufin[2]] >> 2);
}
if (nprbytes > 3) {
*(bufout++) =
(unsigned char) (pr2six[bufin[2]] << 6 | pr2six[bufin[3]]);
}
Under normal circumstances, the nprbytes variable will give 11 as a result, and the while loop will be executed two times, writing a total of 8 bytes into the bufplain array (6 + 2).
However, if the date format is wrong, the calculation of the nprbytes variable can give 12 as a result. So in these cases, the while loop will be executed three times and 9 bytes will be written into the bufplain array. Consequently, the program is writing 1 byte outside of the boundaries of the local array nonce_time.arr, overwriting 1 byte in the program stack (aka off-by-one).
Here we have a use-after-free (UAF) in the cleanup_tables function. Let’s take a look at the code:
static apr_status_t cleanup_tables(void *not_used)
{
ap_log_error(APLOG_MARK, APLOG_INFO, 0, NULL, APLOGNO(01756)
"cleaning up shared memory");
if (client_rmm) {
apr_rmm_destroy(client_rmm);
client_rmm = NULL;
}
if (client_shm) {
apr_shm_destroy(client_shm);
client_shm = NULL;
}
static apr_status_t cleanup_tables(void *not_used)
{
ap_log_error(APLOG_MARK, APLOG_INFO, 0, NULL, APLOGNO(01756)
"cleaning up shared memory");
if (client_rmm) {
apr_rmm_destroy(client_rmm);
client_rmm = NULL;
}
if (client_shm) {
apr_shm_destroy(client_shm);
client_shm = NULL;
}
You can see this function calls the apr_rmm_destroy function in order to free the client_rmm memory block. Yet the problem here is that, under certain circumstances, this memory block could have been previously freed by the apr_allocator_destroy function (not shown in this code snippet).
So, the program is trying to access an address that is no longer valid, leading to a use-after-free vulnerability. It’s important to mention that this vulnerability can only be triggered in the ONE_PROCESS mode.
In this case, we have a heap out-of-bounds write affecting the ap_escape_quotes function. This function escapes any quotes in the given input string. The origin of this bug is a calculation mismatch, between the length of the input string and the size of the “malloced” outstring buffer.
In the following code snippet, you can see the code that calculates the length of the input string:
while (*inchr != '\0'){
newlen++;
if (*inchr == '"') {
newlen++;
}
if ((*inchr == '\\') && (inchr[1] != '\0')) {
inchr++;
newlen++;
}
inchr++;
}
outstring = apr_palloc(p, newlen + 1);
while (*inchr != '\0'){
newlen++;
if (*inchr == '"') {
newlen++;
}
if ((*inchr == '\\') && (inchr[1] != '\0')) {
inchr++;
newlen++;
}
inchr++;
}
outstring = apr_palloc(p, newlen + 1);
In this second code snippet, you can see the code that calculates the size of outstring:
while (*inchr != '\0') {
if ((*inchr == '\\') && (inchr[1] != '\0')) {
*outchr++ = *inchr++;
*outchr++ = *inchr++;
}
if (*inchr == '"') {
*outchr++ = '\\';
}
if (*inchr != '\0') {
*outchr++ = *inchr++;
}
}
*outchr = '\0';
return outstring;
while (*inchr != '\0') {
if ((*inchr == '\\') && (inchr[1] != '\0')) {
*outchr++ = *inchr++;
*outchr++ = *inchr++;
}
if (*inchr == '"') {
*outchr++ = '\\';
}
if (*inchr != '\0') {
*outchr++ = *inchr++;
}
}
*outchr = '\0';
return outstring;
As you can see, it uses a different logic for this second size calculation. As a result, if we provide a malicious input to the ap_escape_quotes function, it is possible to write outside the bounds of the outchr array.
This bug was previously reported by Google OSS-Fuzz, just a few days before I found it.
Now, I’m going to explain something totally different. In this case, the bug is a race condition leading to use-after-free and affecting the Apache Core.
During my fuzzing work, I found multiple non-reproducible UAF crashes. After looking into it more deeply, I discovered a kind of race condition between calls to apr_allocator_destroy and allocator_alloc. All the signs suggested that these functions might not be thread safe in concurrent scenarios. This could lead to a corruption of some nodes of the memory pool and, occasionally, the program tries to release a block that is already present in the free pool. This bug shares some similarities with the bug I reported in ProFTPD (CVE-2020-9273), a year ago.
Here you can see an example ASAN trace:
This is not a new problem. Similar issues were reported by Hanno Böck (@hanno) in 2018. You can check Hanno’s previous reports here.
During my fuzzing session, I found some other minor bugs, and I would like to show you one of them: an integer overflow in the Session_Identity_Decode function. It’s not a dangerous bug, but I think that it can be interesting to show an example of how trivial it is to trigger this bug.
So, take a look at the following example in which we send a LOCK WebDav request to MOD_DAV with a large TimeOut value (Second-41000000004100000000):
In the following code snippet you can see the statement:
return now + expires;
return now + expires;
where there is an addition of two (32-bit) integer values, that will be stored in another integer variable. So if these values are big enough, we can overflow the returned value.
while ((val = ap_getword_white(r->pool, &timeout))
if (!strncmp(val, "Infinite", 8)) {
return DAV_TIMEOUT_INFINITE;
}
if (!strncmp(val, "Second-", 7)) {
val += 7;
expires = atol(val);
now = time(NULL);
return now + expires;
}
}
while ((val = ap_getword_white(r->pool, &timeout))
if (!strncmp(val, "Infinite", 8)) {
return DAV_TIMEOUT_INFINITE;
}
if (!strncmp(val, "Second-", 7)) {
val += 7;
expires = atol(val);
now = time(NULL);
return now + expires;
}
}
Since this bug is triggered with a LOCK request, the MOD_DAV module should be enabled in order to be triggered.
While Apache HTTP security has been extensively studied by researchers, based on recently disclosed vulnerabilities involving path traversals and file disclosures (CVE-2021-41773 and CVE-2021-42013), it is clear that there is still room for discovering new critical vulnerabilities.
With this research, I wanted to make my own contributions to improve the security of the Apache HTTP server, and show that it is possible to use fuzzing for finding vulnerabilities in one of the most used open source software out there. At the same time, I hope that I was able to share all the knowledge I learned with you.
With this third part of my “fuzzing Apache” research, I concluded the “Fuzzing sockets” series. You can find a summary of all my previous posts here:
In my next blog entry, I’ll start a new series focused on fuzzing Javascript engines. Stay tuned!
I’ve used the following resources in this blog post: