are the only ones that matter to submit are the ones in the first two pages in google for the search results for "article directory"?
I think you should try to submit to other directories as well, look for ones that look active. The biggest of them all (ezinearticles) does not even show up for that google search.
When submitting articles, you should also check for niche article directories to submit. For example if you have an article about "flowers" search for "flower articles" and check if there is any site that you can submit your articles. From my experience those sites will send the most traffic, while the article directories will give you better search placements and link popularity.