mt_srand and not so random numbers
August 17th, 2008 | by Stefan Esser |PHP comes with two random number generators named rand() and mt_rand(). The first is just a wrapper around the libc rand() function and the second one is an implementation of the Mersenne Twister pseudo random number generator. Both of these algorithms are seeded by a single 32 bit dword when they are first used in a process or one of the seeding functions srand() or mt_srand() is called.
Because of such a short seed it should be obvious to everyone that neither rand() nor mt_rand() are random enough for cryptographic usages. However web application programmers tend to use rand() or mt_rand() to create cryptographic secrets like passwords, activation keys, autologin cookies or session identifiers. In many situations this seems secure enough, because not only a 32 bit seed needs to be guessed but also the amount of previously generated random numbers. Therefore bruteforcing seems impractical.
There are however several situations and conditions that make bruteforcing feasible or not required at all.
The Implementation
When mt_rand() is seeded internally or by a call to mt_srand() PHP 4 and PHP 5 <= 5.2.0 force the lowest bit to 1. Therefore the strength of the seed is only 31 and not 32 bits. In PHP 5.2.1 and above the implementation of the Mersenne Twister was changed and the forced bit removed.
Implementation Bugs
In PHP 4 and PHP <= 5.2.5 the automatic seed of rand() and mt_srand() is buggy. Whenever the lowest 26 bits of the timestamp are zero the internal seed will become zero (or 1 due to the forced bit) on 32 bit systems because of an overflow of the 32 bit register. On 64 bit systems there is a precision loss when the seed is casted from a double to int that results in a seed about 24 bit strong.
Weak seeding
Although PHP 4.2.0 and above internally seeds the random number generators many PHP applications still call srand() and mt_srand() themself to seed the random number generators. Popular seedings look like one of the following examples.
mt_srand(time()); mt_srand((double) microtime() * 100000); mt_srand((double) microtime() * 1000000); mt_srand((double) microtime() * 10000000);
These seedings are all bad because
- time() is not random. It is known by the attacker(). Even a wrongly set clock on the server side does not protect because time leaks through the Date HTTP Header.
- The first factor is between 0 and 1 and the second factor is 100000. Therefore there are only 100000 possible seeds which is a strength of about 17 bits. On older PHP versions the forced seed bit results in only 16 bit strength. (Due to rounding issues there might be less)
- The first factor is between 0 and 1 and the second factor is 1000000. Therefore there are only 1000000 possible seeds which is a strength of about 20 bits. On older PHP versions the forced seed bit results in only 19 bit strength. (Due to rounding issues there are less)
- The first factor is between 0 and 1 and the second factor is 10000000. However because microtime() depends on the current microsecond() there are only 1 million possible results, which again results in a seed with a strength of about 20 bit. (Due to rounding issues there are less)
CGI seeds once per request
Because in a CGI environment every request is handled by a fresh process the random number generators are always freshly seeded. Therefore an attacker only needs to bruteforce 32/31 bits and does not need to guess how many random numbers where already generated. In case of information leaks generated during the same request using the same random number generator it is possible to use precalculated attack tables.
Crashing PHP for fresh random numbers
Whenever a process is reused for random numbers attacking becomes more difficult, because the internal state of the random number generator is unknown. Therefore any crash bug in PHP is welcome. In older PHP versions crashing PHP is just a matter of sending deeply nested arrays in GET, POST or COOKIE variables. Whenever a crashed PHP process is replaced by a fresh one, a new seed is generated and the internal state of the random number generator just depends on the 32 bit seed. Here precalculated tables can also be used in case of information leaks.
Keep-Alive is your friend
When some information is known about the internal state of the random number generator Keep-Alive HTTP request can make exploits very easy. Because follow request during a Keep-Alive HTTP connection are handled by the same process (same random number generator) the state of the random number generator stays the same and random numbers can be precalculated from the outside. While this is always true for mod_php, it is not true for CGI and only sometimes true for fastcgi setups.
Shared Ressource Problem
Both random number generators are a shared ressource, because the state is stored in process memory and future request in the same process will continue with the previous random number state. Because of this in shared hosting environments that do not use a CGI setup a malicious customer on one VHOST can set the random number generators in a known state by calling srand(0) or mt_srand(0). With the help of Keep-Alive requests it is then possible to predict the random numbers used on other VHOSTs. This of course allows the prediction of newly generated passwords, activation keys or session identifiers.
Information Leaks
Sometimes applications initialize the random number generator and at the same time they generate a random number that is then sent to the user. An example is the search function of phpBB2, where the search_id is generated with mt_rand() and then written to the HTML.
mt_srand ((double) microtime() * 1000000); $search_id = mt_rand();
The problem with this is that in the example above the search function does not only set the random number to a 20 bit wide state but it also leaks the state through the search_id output. In this example there are only 0,003% collisions. Therefore in 99,997% of the calls knowing the search_id means knowing the seed, too. An attacker can simply create a lookup table with one millon entries to determine the seed by search_id. In case one of the collisions is hit (which is quite unlikely) he can just search again.
For PHP versions that force the lowest bit to be 1 like PHP 4 and PHP 5 <= 5.2.0 the numbers above are incorrect. These versions use only 19 bits, which also causes fewer collisions.
The Art of Cross Application Attacks
It is not so obvious why the phpBB2 search_id information leak is a problem, because it is not used in a cryptographic manner. The previous paragraphs however contained enough information to explain why this is a problem after all.
- Initializing the random number generator affects not only the PHP script doing it, but also every other script executed by the same process
- When the state of the random number generator is leaked all future random numbers are predictable
- Several applications create passwords, session identifiers, activation links or other cryptographic tokens with mt_rand()
In addition to that you must stop to consider your application (phpBB 2) to be the only one. In the real world multiple applications that perform different tasks are installed on the same server, most probably even in the same VHOST. Imagine for example a company running a support forum (phpBB 2) and a blog (Wordpress).
In such a situation touching the random number seeding becomes a threat. Especially when the seed leaks. A real world attacks against the combination phpBB2 and Wordpress would look like.
- Use a Keep-Alive HTTP Request to search in the phpBB2 forum for the string ‘a’
- The search should return enough results so that multiple pages are returned which leaks the search_id
- A simple table lookup is performed by the attacker to determine the random number seed
- The attacker initializes his random number generator and throws away one random number. (the search_id)
- The attacker then uses the still active Keep-Alive HTTP request to send an admin password reset request to the Wordpress blog.
- The blog uses mt_rand() to generate the activation link and sends it to the admin by email.
- The attacker can calculate the activation link because his random number generator has the same state.
- The exploit triggers the activation link (over the still active Keep-Alive Request) which results in the new admin password beeing sent to the admin.
- Because the new password is generated by mt_rand() the attacker can also calculate it on his side.
- After that the attacker knows the admin password of the Wordpress blog an can take it over.
This shows that installing two independent applications that both use PHP’s random number generators can result in new cross application security vulnerablities that are as serious and dangerous as every vulnerability contained in a single application.
In order to fight this class of cross application vulnerabilities in the next days a generic protection will be added to the Suhosin extension that increases the strength of the mt_rand() seed from a single dword to a multi-dword width. In addition to that an option will be added to ignore the mt_srand() which will be activated by default. This closes the problem of the cross application attack without requiring any changes in the applications.
On the other hand it is strongly recommended for the PHP developers to add more secure random number functions to the PHP core and it is strongly recommended for PHP application developers to keep their fingers away from srand() or mt_srand() and to never ever use rand() or mt_rand() for cryptographic secrets.





128 Responses to “mt_srand and not so random numbers”
By Gareth Heyes on Aug 17, 2008 | Reply
Excellent article a really refreshing read.
By Luka Kladaric on Aug 17, 2008 | Reply
wow… that’s really thorough… and properly explained.. thanks for an interesting read!
By ascii on Aug 17, 2008 | Reply
thanks Stefan, you always entertain us with quality stuff
By Kyo on Aug 17, 2008 | Reply
Excellent article. Cool stuff
By Jonathan Street on Aug 17, 2008 | Reply
A very interesting read and slightly disturbing. I would be interested to read how you would implement a password reset facility.
By Stefan Esser on Aug 17, 2008 | Reply
Well when you need good random numbers on unix systems there is always the possibility to open and read /dev/(u)random
Another possibility is to mix lots of stuff into your random numbers that cannot be guessed or influenced from the outside. A PHP application developer for example created the following piece which is not that easily broken: http://phpfi.com/344478
When you look at other applications: they overcome the whole seeding problem by creating their own seed (during installation or whatever) as MD5 hash. They store this seed in the database (you can actually store it whereever you want).
When they want to create a random number they read the seed, modify it (append something and reMD5) and store the new seed in the database. And then they create a random number out of that seed…
Combined all these methods overcome the problem that the seed is only 32 bit and that it can be influenced from the outside.
By Jonathan Street on Aug 17, 2008 | Reply
Thanks for the swift reply.
Thinking about it I believe I have seen the cached seed approach used before.
Cheers
By Lars Gunther on Aug 17, 2008 | Reply
I thought rand was changed a while ago to use the Mersenne Twister as well. I suggested it on internals and David Coalier implemented it. It has been pushed to HEAD, so it probably will show up in 5.3
By Stefan Esser on Aug 17, 2008 | Reply
Changing rand() and srand() into aliases for mt_rand() and mt_srand() just makes the whole situation worse, because then applications using srand() suddenly also influence how mt_rand() works in another application.
I just grepped through PEAR. 11 times srand() dangerously used and 4 times mt_srand() dangerously used. So if srand() would become an alias of mt_srand() the situation is obviously worse than before.
BTW: Neither in PHP 5.3 nor in HEAD the aliases are introduced.
By AntonioCS on Aug 18, 2008 | Reply
How about using random generators like random.org? Most of the stuff php is used for is web related so we should have an internet connection.
I just made this simple function using curl to connect to random.org and return a random integer.
http://pastebin.com/f36266a23
By Stefan Esser on Aug 18, 2008 | Reply
@AntonioCS: I know that web developers always want to outsource things and use cool APIs on other servers.
But you really don’t want to do that.
If you question random.org for random numbers then an external entity (random.org) knows your random numbers. You don’t want anyone external to know your random numbers. Additionally you are requesting the random number over a plaintext connection. This means not only random.org but everyone on the route between the server and random.org knows your random number.
That is a very bad idea for security.
By AntonioCS on Aug 18, 2008 | Reply
But this is the only way to get a true random number. Everything generated in the pc will be pseudo-random
By Michael Gauthier on Aug 18, 2008 | Reply
@AntionioCS: /dev/random and /dev/urandom generate random numbers using environmental noise from device drivers. See http://en.wikipedia.org/wiki/Urandom
By Axel on Aug 19, 2008 | Reply
You forgot a PHP function: http://www.php.net/uniqid
Axel
By Stefan Esser on Aug 19, 2008 | Reply
@Axel: I did not forget uniqid() it simply does not fit into the article. The article only covers mt_srand/srand.
By Dmitry-Sh on Aug 21, 2008 | Reply
Great article!
To get more random results from mt_rand(), I’m using str_shuffle() for the list of characters before making a random string.
http://snippets.dzone.com/posts/show/5959
By Stefan Esser on Aug 21, 2008 | Reply
Well using str_shuffle() instead of just mt_rand() is only a little bit better because str_shuffle() uses internally the rand() function.
There are plenty of applications that e.g. use Adodb which itself initialises the rand() number generator. So attacks are still possible it just means an attacker needs to attack both random number generators() at the same time.
And of course on a shared hosting server the attacker owning another vhost can still put up a php script with the content srand(0);mt_srand(0);
By Thijs Kinkhorst on Aug 21, 2008 | Reply
Good story. I really believe these kind of things should be tackled within PHP itself: rather than every application needing to reinvent some secure seeding, the initial seeding done by PHP should be secure enough to be used.
By Raz0r on Aug 29, 2008 | Reply
Great article! I wrote an exploit and it works.
http://raz0r.name/wp-content/uploads/2008/08/wp1.html
By Jan Hingl on Sep 4, 2008 | Reply
why can you still predict the output of mt_rand() - in your wordpress example - when php 5.2.1 states:
“The Mersenne Twister implementation in PHP now uses a new seeding algorithm by Richard Wagner. Identical seeds no longer produce the same sequence of values they did in previous versions.”
?
By Stefan Esser on Sep 4, 2008 | Reply
The documentation is badly worded.
What they actually mean is that because the algorithm is now slightly different PHP >= 5.2.1 and PHP <= 5.2.0 will produce different sequences for the same seed.
However for the same seed and the same PHP version there will still be the same random numbers.
This means an attacker just needs to use the same algorithm as his victim uses. (but he will do that anyway, because prior to 5.2.1 only 31 bits of the seed were used so a faster attack is possible by default)
By Logician on Sep 5, 2008 | Reply
@Stefan: Thanks for the great article. It is really very well written and right on spot..
@Raz0r: Is it really a good idea to post your exploit publicly and let script kiddies attack many sites?
By Anonymous on Sep 9, 2008 | Reply
Then how will random numbers be generated?
By Neil Davis on Sep 9, 2008 | Reply
Now all we need to do is train php to not stomp the same bit when doing bitwise operations =D
Adding unsigned int support to the parser would be very constructive and solve all such bugs(and probably create a whole new crop of bugs in the process =D)
As it is you need to grab them in 8 or 16 bit chunks to do it accurately. (as I’m sure the person who fixed mt_rand() implementation found out
Fixing this would make me really really happy. I tend to operate as close to the metal (even when scripting) as possible and could use such a thing.
When doing things such as IP calculations you need all 32 bits.
Neil Davis
By jf on Sep 11, 2008 | Reply
you guys are my hero’s ;]
By dber on Sep 21, 2008 | Reply
Really nice guide but it isn’t clear enough how to solve this…
In the php.net page of ‘uniqid()’ there is an example with the following code:
$better_token = md5(uniqid(rand(), true));
What about using this as a seed?
And what about the MySQL random:
SELECT RAND();
?
By Mazzu on Sep 30, 2008 | Reply
Thanks Stefan for this great article (and the others).
Now, I’m looking for better ways to generate random numbers and I have few questions :
What about openssl_random_pseudo_bytes() ? Is it more unpredictable ? What’s the best way to use it ? So, will it be a good idea to use it ?
http://news.php.net/php.internals/40225
https://www.openssl.org/docs/crypto/RAND_bytes.html
By Stefan Esser on Sep 30, 2008 | Reply
The function in question is supposed to be added in PHP 5.3
When it is added and when ext/openssl is enabled on the server it is the recommended way to get random numbers.
By YellowSEO on Nov 20, 2008 | Reply
Cool I was looking for a way to get more random numbers then mt_rand().
By Me on Feb 16, 2009 | Reply
So with out installing the Suhosin patch, how can we securely implement random functions?