So, all I needed to do is turn off "Show on Profile Edit"? OK, done.
Yes, that will prevent captcha from rendering in profile edit. It's not really necessary there and seams Joomla's captcha doesn't work there. That would work fine if you used one of CB AntiSpams captcha modes, but again it's not really necessary in profile edit.