Firstly, let me just say this subject goes a bit over my head at this point and I doubt this is something you'll get a simple answer to; but, I've been told of some open source thing which functions like Maple/Mathematica. (It may have been
Maxima). So, you might want to look at how they do it.
Depending on what you're doing converting to
Reverse Polish Notation may be useful, too.