If you want to support your own character, i think i would pick alchemy for your poisons.
Spell upgrades are not so hard to find, especially if you have the splitpaw saga adventure pack, where adpets drop at a maniacal rate. Maniacal is weak. You still can find adepts on the broker for a decent price most of the time.
As far as weapons go, just like the poster before me said, it will require you a lot of time to be able to craft good weapons that are in line with your adventuring level. Not to mention that theres a lot of good weapons that drop out there in the field or that you can quest for...in fact as a swashy if you have the patience enough to do heritage quests, you will prolly end up wearing the tomahawk and the serrated dirk for a long while, which may make your choice of weaponsmith not being so rewardful in the long run.
Hope that helps, note that its only my opinions though. You could even go provisioner and buy everything you need off the money you save for not having to pay high prices for food and drinks...