Do you know if you have a proxy or nginx in front of your web server? Of so you can try switching to X-Forwarded-For which may give you better results. But first, do me a favor and check your live traffic. Are you seeing a lot of different IP's or are they all the same IP?
htaccess is actually just an apache configuration file that applies to a particular directory and subdirs. So to cover all directives in that file would require me to give you an exhaustive guide to apache configuration. However the specific directives you're interested in are related to mod_rewrite which you can find here:
http://httpd.apache.org/docs/current/mod/mod_rewrite.html
Regards,
Mark
PS: If you found this helpful, please rate Wordfence 5 stars.
http://wordpress.org/plugins/wordfence/