If it takes you more than 5 minutes to read through what it does, it's probably too long. If you have a bunch of Deny rules, you may want to consider putting it in it's own custom .conf file included in the VirtualHost block. That way it will be in memory, and not have to load every time to parse rules.
.htaccess decreases the performance just because the web server looks for that file and opens that file for each and every request. So for best performance, you'd indeed have to put everything in the Apache configuration and configure "AllowOverride None". However, the performance impact might not be noticeable if your site isn't under heavy load.