Just video, no content? Now how in the world is google going to match ads to a video? They are pictures, not words.... Keep that in mind.
Well, what if you have a little bit of introductory words, you know, explaining the website on each page? On my test site (with video) I get great keyword text ads. But your answer is not why you can't have video and adsense ads on the same page. From the Adsense Policy Guidelines, Here's why: "Copyrighted Material In order to avoid associations with copyright claims, website publishers may not display Google ads on web pages with MP3, Video, News Groups, and Image Results." That is too restrictive for my time and money. I need to find another affiliate.
What if you use videos available from Google Video? Google encourages you to put these videos on your site and even offer you the code to do it. That's where I get all of my videos. Do you think that's protection enough?