I regularly delete some of my unit tests, sometimes within minutes of writing them. Even as a TDD fanatic I’ve come to realize tests are just a means to an end. If I decide to write a constructor test when first designing a bit of code and delete it a few minutes later, nothing’s wrong. These sorts of initial tests can become redundant quickly since to you have to construct the object for all the later tests.
As you adjust to TDD there’s a tendency to see the tests as important code. Deleting them is the last thing you’d think of doing. It turns out just like with production code less tests are better easier to maintain and easier to refactor. And as an added benefit your test suite runs faster. Tests are path to good code and you get an added benefit of a regression suite. And besides everyone’s favorite checkins are ones where you remove more lines of code then you add.