We've tried various strategies to get him to stop, and our latest attempt is to make him recite this poem every time we catch him with his thumb in his mouth.
"Oh the thumb-sucker's thumb
May look wrinkled and wet
And withered, and white as the snow,
But the taste of a thumb
Is the sweetest taste yet
(As only we thumb-sucker's know)." --- Shel Silverstein
The strategy isn't really working.