There's all the touristy stuff like the Statue of Liberty, Ellis Island, the Intrepid, Empire State Bldg, Rockefeller Center, United Nations, the Circle Line tour boat around Manhattan, more museums than you can shake a stick at.
Central Park Zoo - Southeast corner of the park
There are several nice playgrounds in the Park, one just north of the Metropolitan Museum of Art
Lots of puppet shows or the Paper Bag Players
Children's Museum on 83rd between Broadway and Amsterdam
Take a boat out in Central Park
I like the Alice in Wonderland statue in the park
I liked Beauty and the Beast for younger children.
If they go to Blue Man Group off Broadway the front 3 rows come with a poncho
Too many places to eat and depends on their preferences. There's the Bubba Gump Shrimp Co. in Times Square, Jekyll and Hyde's if you like themes.
P.S.
Pick up a map of Central Park at the Dairy just off the Park cut through at 65th St. behind the Central Park Zoo.
A note on the Statue of Liberty. It's an all day trip starting with a line for boat tickets down in Battery Park. Only the first boatload of people get to go in the statue itself, but the lines can be long for a 4-year old.