I haven't found any bound spells yet.
The Jarl's Wizard in Dragonsreach (Farengar, I think?) has bound axe, but it's the two-handed variety (which works out well for a Nord). I think my bound blade tome was a loot drop, but I'm pretty sure I've seen it for sale either in Dragonsreach or Riverwood, or possibly the general store in Whiterun. Of course, I've noticed that the quality of items for sale seems to be level-related to some extent, so that might have something to do with it. I'm currently level 15, but I do remember Farengar already having bound axe for sale when I first met him, which was much earlier on.
My guy is a Nord, and generally boring the heck out of me, so I'm going to join the Stormcloaks tonight and see what this civil war is like. I wonder if Imperials will know I'm a Stormcloak and hate me, etc. http://uesp.net/wiki/Skyrim:Races
According to that, your intial buffs are
+5 Block, Light Armor, One-handed, Smithing, Speech
Not the best start for a caster, but it's not like the difference is insurmountable, esp. with trainers. I have yet to find any magical trainers, but I would guess there are probably at least a few in Winterhold. Really the impact of race selection seems more minimal than ever in Skyrim. Every race gets +10 to one thing, and +5 to a smattering of other stuff. The powers seem like the main differentiator. Nord battlecry sounds kind of bleh... Orc power sounds awesome. I'm sad that they nerfed the Redguard power - it used to be fantastic for picking up extra stuff.
One thing I'm really loving is the ability to choose your sign with guardian stones and change it any time you want (providing you find the stones of course).