I think you've done a lot of good thinking and have got an interesting idea. An especially good start is that you seem to kind of know where you're wanting to go with the story.
Have you been keeping notes? That would be a good idea. So would a sort of general outline, since you do know where you want to go, and the path you want to take to get there. Probably the next thing to do is to sort of fill in blanks, if you haven't. The incidents that take your hero from being a blacksmith to being a pirate. What happens to him while he is a pirate and leading up to his being captured. How he gets from being captured to being a farmer...and so on. Tim Powers (who is a good writer, if you haven't read him) does his plotting with index cards, and he makes timelines or calendars. You could try writing incidents down on cards as you think of them, that way you can rearrange them until you have the plot points of your story in the order you want them in.
Alternatively, you could just start writing. Other writers prefer that method. You need to do what works best for you.
But you definitely need to start doing something. Just thinking doesn't make you a writer. Writing does. And I think you could do something with this idea.
Have fun with it. And keep us posted.
Oh, and by the way, a lot of us around here are nerds. It's nothing to be ashamed of, and in fact some of us are very proud of it.
