Spiritual adultery is far more than just worshipping other Gods, it is also borrowing the customs and symbolism of other gods in an attempt to worship YHVH. God makes plain that those who worship Him must not simply borrow, or re-label pagan worship practices in an attempt to worship the true God. God does not condemn everything that originated in pagan cultures, but He does condemn pagan worship customs that are used in a shameful attempt to honor him.