I came up against a really baffling problem the other day. I was tasked with adding a Wiki for VideoSift. Rather than just dumping MediaWiki onto the primary VideoSift web server, we wanted to keep it self-contained for security and load reasons, so we installed into one of our other back-end servers which was being used as a MySQL Slave and not hosting any web content. To start with I just did a quick, basic install of Apache2 and PHP5 (I would prefer and attempted to use Nginx with PHP5-FPM, but MediaWiki complains about implementation bugs in the packaged versions available to me and I don’t want to recompile from source).
I have never dealt with MediaWiki previously, so I just went and setup the wiki.videosift.com subdomain to load from the back-end server then downloaded the tarball into the document root and went through the install process. I spent an inordinate amount of time figuring out how it might be possible to synchronize existing VideoSift member accounts and logins seamlessly into the Wiki, (but the details of how I did that are for another story, which I may clog about soon). All of my software changes seemed to make sense and even after in-depth inspection should have been functioning properly, but if anyone went from VideoSift to the Wiki or vice versa, they were instantly automatically being logged out of both.
After spending many clueless hours debugging back and forth, I finally found that the sessions designed to be properly shared across the two servers was to blame. The session data on one server would be written to a centralized session depot and the other server should have read that session data, allowing the session to persist across both websites.
I use session_set_save_handler() to manipulate how sessions are managed, so I put in some debug code on both servers to dump the raw session data that was being read and written. To my surprise they were totally different. On the Wiki the session data was a normal session-serialized string of variable data like this:
but the VideoSift session contained a seemingly random string of characters:
After a lot more digging and googling, I finally came across this comment at php.net. Unbeknownst to me, as a security measure, the suhosin module for PHP was using its Transparent Encryption Options to automatically encrypt all session data as it was being written.
This is a good idea and I’m glad to leave encryption on, but why was it being encrypted differently on the two servers? Suhosin by default uses a couple of different pieces of information as its session encryption key. One of the enabled session encryption options (see the previous link) is “suhosin.session.cryptdocroot” which includes the path to the website’s document root in its encryption key. Since VideoSift and the Wiki are located in a different folder on each server, this was causing the shared session data written by one server to be unreadable (un-decryptable really) by the other server.
There are a couple of obvious solutions, and since I’d prefer to maintain whatever shreds of security are already in place, I opted to specify a shared session encryption key on both servers (using suhosin’s suhosin.session.cryptkey option) and disable the suhosin.session.cryptdocroot option. Et voila, she works.