I've had a problem like this - almost - with two different horses. The difference was that both gelding pinned their ears when I came at them with the saddle. The solution was a new saddle that fit. The second gelding came to me that way and he was really bad. Tried to bite. After a few saddlings with my saddle instead of his previous owners', he stopped snapping and pinning his ears. He never does it now.
Right now I'm riding a new mare and my saddle does not fit her so a new one is on order. In the meantime she is starting to pin her ears when she sees me coming with the old Circle Y. In a few more days the new saddle will be here and then she can be happy again.
All three of these horses have to be cinched slowly. The first one, a quarter horse, could blow up about 12 inches. But over the course of getting ready to ride we could slowly cinch him up.
Something I learned about saddle trees that surprised me is that the Arab tree is the widest tree. Arabs have flat, wider backs than quarter horses.