How a firewall can break your app

Picture this: after we had moved an old app from an old server to a new one, we kept getting “I/O Exception: Broken Pipe” now and then from our database connections. In the new environment, database lives on a different server than our app (as opposed to the old environment). Is there something wrong with the network?

My suspects turned out right: it’s the firewall between app and database. It drops “unused” tcp connections after an hour. For some security reasons, this cannot be changed.

As our apps is rather old, some parts of it still do not use a connection pool and so there are db connnections that “linger” around unused for an hour and are used thereafter – bang, I/O exception.

So watch out for that and use a connection pool. Possibly one that can shrink (i.e. throw away unused connections).