Cats are very associative, meaning that if they have a bad experience in a place a time or two, they tend to avoid that place. You can use this to your advantage.
Cats have very sensitive hearing, so an effective but humane way to give a cat a bad experience is to use a shake can. Throw a handful of change in an empty coffee can, snap the lid back on, and keep that by the door. Every time you see the cat in your yard, grab the can, open the door, and shake it as hard as you can. It makes a loud obnoxious noise that absolutely jangles a cat's nerves, but doesn't inflict any harm. The second you start to shake that can, I guarantee the cat will bolt as fast as he can. You can even chase after him, shaking the can and yelling, to reinforce the experience (if you don't mind the neighbors looking at you funny). If you give him the shake can treatment a couple of times, he'll probably avoid your yard after that because he will associate going there with a loud nasty experience. You may have to reinforce it now and then if he gets enough courage to sneak back, but it should definitely make your yard a lot less attractive to him.
Obviously be careful when you do this so the cat doesn't run into danger. For instance, if you live on a busy street, you'll want to position yourself between the cat and the street when you shake the can so that the panicked cat runs away from the street, not towards it.
Also, depending on what county you live in, it may be illegal for the owners to allow the pet to roam freely, and to damage, soil, or defecate on other people's property. In Montgomery County those prohibitions apply to all animals, including cats. You can make anonymous complaints to the animal control agency, but unfortunately most owners tend to have a pretty good idea who made the complaint, so it can get awkward. Also, there's a natural reluctance to call in the authorities.
