Thats the only way I am able to get it to work is by setting a rule.
In order for a QOS system to work, you have to set some rules. Now you've done that it works, is that right? Then we'll move on.
Next - you say your useage is very basic, but then go on to describe that you have at least 5 machines using a variety of applications - web/email/games/facebook, which themselves interface with video applications using several other protocols, perhaps IMAP, POP3, flash video for example, etc. - together with the ability to download things by HTTP such as updates, new applications etc. Your useage is not quite so "basic" as it seems and quite probably you WILL need some of those rules at certain times.
The default QOS rules are there to act as a base for you to amend as you think fit. They cover most things that are in common useage, but if they are not being used then the rule simply won't have any effect. If you change or delete many of those rules, you will often find they had a considerable effect on your system. You can use the pie charts and the connection details pages to see what connections the rules are affecting at any given time. Whether the rule actually did what you wanted is for you to decide based on what you and the other users see happening. The best way to proceed is to disable rules one at a time, note the effect they had, and learn how to adjust it or if you can leave it disabled. To actually change the setup of a rule you need to understand how QOS works and how to adjust it to get it to do what you want. This link will help you to do that.
For best games latency alone, you could try switching off the normal tomato QOS and instead use the bandwidth limiter QOS. Limit people to their own "chunk" of bandwidth and makes sure they add up to say 20% less than your available bandwidth - and that may preserve your latency. That may do what you want but you will slow down each person's applications, trading latency for bandwidth.
Good luck in your endeavors!