It could easily be a firmware/acceleration thing on how your board handles tiny changes in direction. Marlin has look ahead, not all boards do and need the software to do it for them, I don’t know what yours needs.
A 4 flute bit is not a good choice for that high of rpm.
How long is your zaxis? more than 3″? that could be your problem. How large is your machine? too large could also be the problem.
At this point I can’t really help you it could be many thing. I would say list as much detail as you can, including answering the questions in the troubleshooting sticky so maybe someone else using similar hardware can help you.
Pictures go a long way. You don’t really give much information to go on so full pictures of your machine, pictures of your bit, screen shots of your settings. You need to post it all. Most people here are using the recommended hardware, making it very easy for me to diagnose things with little information. You need to flood the info here so someone might spot the issue.