MySQL and SQL Column Truncation Vulnerabilities
August 18th, 2008 | by Stefan Esser |While SQL-Injection is one of the most discussed security problems in web applications other possible problems for SQL queries like overlong input are usually ignored although they can lead to all kinds of security problems.
This might be caused by the fact that security problems that are the result of overlong input are often buffer overflows and buffer overflows are something many web application security experts know nothing about and choose to ignore.
There are however several security problems for SQL queries that are caused by overlong input and no one talks about.
max_packet_size
In MySQL there exists a configuration option called max_packet_size which is set to one megabyte by default and controls the maximum size of a packet sent between the SQL client and server. When queries or result rows do not fit into a single packet a error is raised. This means an overlong SQL query is never sent to the server and therefore never executed.
This can lead to security problems when an attacker is able to supply long data elements that are then used in SQL queries. A good example are logging queries that combine information like the HTTP User-Agent, session ids and log messages into a large query that then does not fit into the packet anymore.
Another example from a real world application is a session table cleanup process that first selects all sessions matching certain parameters into a PHP array, then performs a multiple level cleanup and in the end all selected session ids are put into single delete query. It should be obvious that when there are many session identifiers in the table that need deletion the query gets too long. The result of this is that flooding the application with new sessions in a short time will result in no unused session being deleted later anymore.
Therefore web application developers should always ensure that they do not sent overlong data to the server. And it doesn’t matter if they use prepared statements or not.
SQL Column Truncation Vulnerabilities
When user input is not checked for its length SQL Column Truncation Vulnerabilities can arise. “SQL Column Truncation Vulnerability” is the name I use to describe security problems arising from overlong input that is truncated during insertion in the database. By default MySQL will truncate strings longer than the defined maximum column width and only emit a warning. Those warnings are usually not seen by web applications and therefore not handled at all. In MySQL the sql_mode STRICT_ALL_TABLES can be activated to turn these warnings into errors but applications will run most of the time on servers that run in the default mode and even if an application uses the stricter sql_mode it should not produce this error in the first place. Therefore a length check is required.
To understand why the truncation on insert can lead to security problems imagine the following application.
- The application is a forum where new users can register
- The administrator’s name is known e.g. ‘admin’
- MySQL is used in the default mode
- There is no application restriction on the length of new user names
- The database column username is limited to 16 characters
A potential attacker might now try to register the name ‘admin ‘, which will fail because the ‘isAlreadyRegistered’ check will result in the SQL query.
SELECT * FROM user WHERE username='admin '
Because MySQL does not compare strings in binary mode by default more relaxed comparison rules are used. One of these relaxations is that trailing space characters are ignored during the comparison. This means the string ‘admin ‘ is still equal to the string ‘admin’ in the database. And therefore the application will refuse to accept the new user.
If the attacker however tries the username ‘admin x’ the application will search for it in the database and will not find it, because it is impossible to find a username with a length of 17 in a database field that has a 16 character limit. The application will accept the new username and insert it into the database. However the username column is to short for the full name and therefore it is truncated and ‘admin ‘ is inserted into the database.
The result of this is that the user table now contains two users that due to trailing spaces both will be returned when the SELECT query above is executed. At this point a potential security problem arises because now it depends on how the username is treated throughout the application. The following pseudocode for example is vulnerable.
$userdata = null; if (isPasswordCorrect($username, $password)) { $userdata = getUserDataByLogin($username); ... }
When the previous piece of code uses the SQL query
SELECT username FROM users WHERE username = ? AND passhash = ?
to detect if the user password is correct and then does a lookup of the user data by name a security problem manifests.
SELECT * FROM users WHERE username = ?
Because the attacker created the newly created admin user he knows the correct password to pass this check. And because the real admin user is first in the table it will be returned first when the user data lookup by name is executed later.
Conclusion
Both problems described here are two new things web applications needs to be audited for because both can lead to real security problems. And because no one searches for these kind of vulnerabilities, now that it is public most probably the next weeks will bring several advisories about open source software suffering from these problems.





189 Responses to “MySQL and SQL Column Truncation Vulnerabilities”
By xaitax on Aug 18, 2008 | Reply
Nice discovery. Often thought about that flaw, but didn’t expect it will be go wild.
Thanx for the article.
By Gareth Heyes on Aug 18, 2008 | Reply
If the username column has a unique index would this still allow a username padded with spaces?
By Stefan Esser on Aug 18, 2008 | Reply
No this will trigger an error, because the trailing spaces are ignored during the comparison.
By Gareth Heyes on Aug 18, 2008 | Reply
Thanks I thought so, another good article
I use a combination of strlen checking, a separate SQL comparison for usernames and a unique column index
By Andreas on Aug 18, 2008 | Reply
But that’s only a problem because MySQL has a sloppy handling of invalid input data, right?
If MySQL would reject the input string in the first place because it’s too long, this problem would not exist. But the “make the user happy instead of raising an error” input data handling creates another security problem.
By Stefan Esser on Aug 18, 2008 | Reply
Like I stated the approach to throw an error on overlong packets when they exceed max_packet_size also leads to security problems.
By Andrew Bidochko on Aug 18, 2008 | Reply
Stefan, thanks for the article. As always, good proof of concept!
I’d like to note that having a UNIQUE KEY on a `username` column will overcome the reported issue.
But anyway, Truncation Vulnerabilities may lead to other potential flows in web applications.
By Stefan Esser on Aug 18, 2008 | Reply
setting a UNIQUE KEY on the username column will result in a database error on insert, therefore it is still up to application to catch this error case before actually triggering it.
By rvdh on Aug 18, 2008 | Reply
Conclusion
Both problems described here are two new things web applications needs to be audited for because both can lead to real security problems.
–
Actually it isn’t new. SQL server suffered from the same technique. SQL Server 2000 SP4 and SQL Server 2005 SP1 silently truncate the data if the variable does not have big enough buffers.
–
And because no one searches for these kind of vulnerabilities.
–
Oh? I guess I discused them many times before, regarding data buffers and checking data sizes before handling/passing data into a query.
–
By Stefan Esser on Aug 18, 2008 | Reply
mr. rdvh
first of all MySQL does not silently truncate anything. It is a documented feature that results in a warning. It is the applications fault to ignore those warnings.
If you discussed this before then great let me see the url where it is documented. It is simply not possible to read everything released somewhere in a corner of the web.
And I also believe that not many people think about these vulnerabilities, because otherwise there were advisories covering this type of vulnerability on bugtraq.
By rvdh on Aug 18, 2008 | Reply
first of all MySQL does not silently truncate anything. It is a documented feature that results in a warning. It is the applications fault to ignore those warnings.
–
Correct, I talked about T-SQL in perticular which led to a truncation attack also.
–
If you discussed this before then great let me see the url where it is documented. It is simply not possible to read everything released somewhere in a corner of the web.
–
I mentioned it here briefly, though it was T-SQL specific with using single quotes. Although the technique of truncation is mostly the same:
http://www.0×000000.com/index.php?i=396
–
And I also believe that not many people think about these vulnerabilities, because otherwise there were advisories covering this type of vulnerability on bugtraq.
–
Correct, I’m glad you point that out Stefan.
But that holds the same truth for many memory limits e.g. buffers triggered by PHP or MySQL. As you know, most “scripters” seem to lack the knowlegde that low-level (like C) programmers have, but it’s a fair assumption that one must treat all user data as tainted, including data-size. I advocated many times before the need for writing buffers or data limiters in PHP to limit or/and restrict supplied user-data, I am glad you found this in MySQL because it confirms my overal thoughts about MySQL and PHP interaction alltogether.
–
By Lukas on Aug 20, 2008 | Reply
I guess the chances of people really going through the effort of explicit length checks in all the right places (WHERE clauses seem to be the problem spot if you are running in strict mode), are pretty low. However this is again an area where an ORM can help. It would allow centralized handling of length checks. In theory an ORM could also try to circumvent max packet size restrictions (as in split the DELETE statement), but this would be quite hard to do reliably (it could do some serious business logic breackage).
By Arshan on Aug 20, 2008 | Reply
Good post.
> One of these relaxations is that trailing
> space characters are ignored during the
> comparison.
Here you seem to be hinting that there are other types of relaxations used when comparing non-binary strings. The best technical resource for this subject I found here:
http://dev.mysql.com/doc/refman/5.0/en/char.html
However, after looking that over and doing a bit more searching, I can’t find any other quirks in the way strings are compared. Can you shed any light on this?
By Stefan Esser on Aug 20, 2008 | Reply
@Arshan: I maybe was a bit vague…
Another relaxation depends for example on the collation. The default collations of mysql for different charsets all end with ‘_ci’. The _ci means that it is case insensitive. That means all strings are compared case insensitive by default.
By Andi on Aug 21, 2008 | Reply
Nice article!
I had trouble with “max_packet_size” one time I inserted _many_ rows at once - so I never had the idea of truncating any string before executing a _single_ row insertion.
I use a UNIQUE index for both nicknames and email address, so in this case truncating the string in PHP is not absolutely necessary.
By Kyo on Aug 24, 2008 | Reply
Good article!
I usually trim() the username and remove double spaces in the registering process (php)
By pstradomski on Sep 9, 2008 | Reply
Well, data validation should be performed in the database, if it’s possible. So UNIQUE index is a must. Application-space validation usually fails in some obscure way. Some other situations, where uniqueness constraints in the application often fail:
* integer truncation
* race conditions (even when using transactions, true SERIALIZABLE isolation level would be required, which is almost never enabled, and for MVCC databases still does not prevent simultanous insertion of identical values).
By Nil on Sep 9, 2008 | Reply
Thats why everybody should use the Strict-Mode in MySQL, then it will raise an error.
By Al O'Nerd on Sep 23, 2008 | Reply
Will the BINARY keyword used in selecting user from username and passhash avoid that type of exploit?
By aslifm on Nov 23, 2008 | Reply
Nice discovery. Often thought about that flaw, but didn’t expect it will be go wild.
Thanx for the article.