my problem

Hey, I'm a new IIDX player but I've played non-keysounded rhythm games for ages, starting from Guitar Hero and going to Stepmania and Osu! and now trying out IIDX.

My playstyle is very audial and wrist-heavy, and my accuracy in all games I've played so far has been very good, because of my strong sense of rhythm from being a drummer for 10 years. However, when I play like this in IIDX, it feels really off. Let me demonstrate this.

https://www.youtube.com/watch?v=ObKRkNTYE-M&feature=youtu.be

Here's a soundclip of me playing Mermaid Girl on normal. I want to focus on one specific part as an example, the chorus starting at 1:15. The main melody here mimics the vocals perfectly, so I can just ignore the keysounds and tap to the vocals and the backbeat, right? That's exactly what I do here, yet what the game awards me is all Goods instead of any greats or PGreats. As an experienced rhythm gamer, this really baffles me.

The whole song ends up 54 PGreats, 166 Greats and 272 Goods. I have noted that by playing more visually and focusing on the melody I play instead of the backbeat, of course I get much better accuracy. I have AAd this chart by doing that. The thing is however, I'd like to play to the rhythm, and the game just feels off sync in that way.

Even funnier is no matter what I set my delay at, +5 to -5, the end result is always the same...? I get all SLOWs and almost no FASTs.

Is there literally anything I can do except suck it up and give up my super audial playstyle?