If you plan on doing heavy amounts of tagging and branching, use
CVS. Otherwise, I recomend
Subversion.
In my expieriance, Subversion tools are much better than CVS ones (translation: TortoiseSVN is much more streamlined and strange than TortoiseCVS). It is also easier to setup a server on Windows and Linux. (In Subversion, use the included mini-server or add mod_svn to Apache. In CVS, use a Linux box. CVSNT is the only Windows CVS server and I have not heard good things about it.) Subversion also handles directories, removals, and renames infinately better than CVS. CVS cannot handle directories; removals are so-so; renames consist of either moving the file in the repo (so that the entire history is moved, and no reference to the old name exists), or delete the old and create a new (so that there is no connection between them.
As for the tag/branch comment, here's the thing: in SVN, you have to actually create a new repository for each tag and branch, in addition to the trunk (main line of code). CVS handles tags and branches natively.
For further details, I defer to google. (BTW, I've found that Subversion has much better "official" documentation than CVS.)