[Turing'sEgo]

The problem with doing RGB tracking of the boulder is that it is gray. In other words, you're trying to track an absence of color sort of speak. You could investigate dynamically thresholding techniques such as Otsu's method (which opencv has).