I would recommend downloading Visual C# Express as mentioned earlier, and going through the Help files, which are quite comprehensive once you learn how to navigate them.
To start with, you can go to Help->Contents, navigate the tree through Development Tools and Languages->Visual Studio->Visual C#->Creating your First C# Application.
The folders might be slightly different since I've got Visual Studio 2005 installed, but they should be pretty similar.
Another avenue is to start with something fun that will keep you coming back.... look up
Microsoft XNA, a free game development engine that can produce applications for PC or Xbox360. This engine is very easy to learn, and there are a large number of tutorials that can get you started in doing something you'll have a lot of fun with as well as learn a new language.