Your blade is going to want to drift. You need to compensate for this by feeding your wood at an angle. Go slow and watch ahead of your blade slightly and cut toward a point in front of the blade rather than following the line. Once you try what I'm trying to describe you will understand.
Mainly it is practice, and prayer that the blade won't catch the grain of the wood and go sideways. If it does then it is finesse' sanding...LOL