Mysterious NoReverseMatch Error

A couple of days ago we have faced quite a creepy issue: a Django-based site with tens of thousands hits/day has been throwing NoReverseMatch errors with no particular pattern. Basically, any url on the site could be found in the error emails. These emails were flowing in at a very worrysome rate, and user complaints started to arrive as well.

I'm sharing our findings just so that if anyone gets into the same trouble he could find the info on google.

All the credit for discovering the roots of the problem goes to Vadim Shender whom you might know for his OCaml posts if you read this blog.

If you are using Django, by now you are probably thinking "swallowed ImportError". If only it was that simple! Without going into too much detail, here's what we eventually found: django/core/urlresolvers.py is not thread-safe and the server was using Apache+mod_wsgi config with multiple threads per process. Once you know this is a thread safety problem, it's trivial to find the relevant info, for instance: http://code.djangoproject.com/wiki/DjangoSpecifications/Core/Threading. The conclusion: Django is known to have thread safety issues so if you are using a multithreaded config be prepared to see some magic under the load and react to it.

Possible solutions?

1. Switch to a prefork config. This is safest, but you probably won't get as much performance.
2. Patch your copy of Django. We shall post a patch for this particular issue here.
3. Increase the maximum number of requests per process. Haven't tried that yet, but it should be helpful because the problem is in initialization which is only done once per each new process (see details below).
4. Other suggestions are very welcome.

For the record: the site I mentioned uses Django SVN revision 7153. Just looked at trunk head at it seems to have the same problem, but this needs testing. We are going to submit a ticket and probably a patch which we'll also attach to this post.

For curious, here's a more detailed description. The code that causes the problem:

    def _get_reverse_dict(self):
        if not self._reverse_dict and hasattr(self.urlconf_module, 'urlpatterns'):
            for pattern in reversed(self.urlconf_module.urlpatterns):
                if isinstance(pattern, RegexURLResolver):
                    for key, value in pattern.reverse_dict.iteritems():
                        self._reverse_dict[key] = (pattern,) + value
                else:
                    self._reverse_dict[pattern.callback] = (pattern,)
                    self._reverse_dict[pattern.name] = (pattern,)
        return self._reverse_dict