He is going to get more annoyed with you for ignoring
what he wants. Is he not treating you the same as he usually does?
The bottom line is that it doesn't matter
at this point who was or is at fault. Once
you begin, it will take small and significant st