It sounds very interesting! I think (as previous persons) that the most important in your case is to visiting others blogs, writing comments, writing to authors. Probably, they will use your link, quotes etc. one day and the traffic will be growing day by day.
Social network is an important part in blogging but since you are talking about audience, yes you can build it through organic medium too. Just add a subscription form on your blog so you can be in touch with your audience.
I'll be honest with you. In this day and age, most people have social media- and spend a lot of time on it. I have never found a blog I enjoyed outside of social media. When I post links to my new blog posts on social media, my website analytics show that about 99% of traffic comes from Facebook, Twitter, and Instagram. Start by building your social media platforms and go from there. In order to do so, interact with other bloggers by "liking", "retweeting", "sharing", etc. Good luck!