This is slightly odd and not something I have saw when using a different third-party firmware.
Asus WL500gPv2 - Tomato-RAF-K26USB-1.28.8703MIPSR1-8MB-VPN
Russian Dual Access Internet connection
For those who don't know about dual access, it's essentially local LAN services on a 172 subnet and internet connectivity via an L2TP tunnel.
When I start a download from the torrent server hosted by my ISP, my Internet connection is immediately terminated, but I still have local access and the download continues. Also, I'm unable to access the router via the browser interface. If I reduce the download rate, in my torrent client from 5MB to 4MB/sec, internet access is resumed. I believe the local download rate is 10MB/sec and the Internet download rate is 5MB/sec.
I don't use QOS as I'm the sole user of the Internet at this location, but is this simple a bandwidth management problem?