I stumbled upon this post by Martin Fowler via another blog. It's a bit dated (2004), but interesting. It reveals that JUnit creates a new instance of TestCase for each test method defined within it. The primary reason for doing this is so that tests are isolated from each other. That is tests don't share the state of objects. So they can be run in any order needed.
JUnit has two special methods setUp and tearDown (these I guess can be called anything in the newer version using Annotations) that are automatically run (if defined) before each test method. I wondered how they did this. Now it makes sense.