Can I disallow crawling of my CSS and JavaScript files?

orlistat 180 mg Discover More Here

On February 26, 2009, Google software engineer Matt Cutts collected questions on Google Moderator and answered many of them on video.

SEOmofo from Simi Valley asked:

If I externalize all CSS style definitions and JavaScript scripts and disallow all user agents from accessing these external files (via robots.txt), would this cause problems for Googlebot? Does Googlebot need access to these files?


6 thoughts on “Can I disallow crawling of my CSS and JavaScript files?”

  1. Allowing bots to crawl one's own CSS and JavaScript sounds the least susceptible to google reconstructing the said files. Definitely, I'd rather have google check on my stuff than not in order to keep things safe and clean.

  2. I do not agree to allow access to javascript and css. The reason for this is that url's will be accessed that is not meant to be indexed by a bot, for example ajax calls etc. I have seen that some other bot tries to access ajax urls decoded from javascript. In my case they will be blocked anyway so there is no reason to crawl javascript, not worth the extra traffic it cause. With CSS the only reason can be to index your images, a thing most people don't want. Best way: Don't serve when is bot.

  3. Basically, We don't touch your CSS Files unless we've received a spam report etc – which happens often in the web world, design company to design company.

    I've tested not allowing and allowing CSS and the Ranking doesn't change one bit, not even when using -3200px left margin headers for Logo / Site Title Text

  4. Does Google index javascript via the document.write() method? I have some sites that can not use includes and are on different servers so I created a navbar in javascript from one publish point using that method.

  5. interesting answer – at end u say that not blocking the css etc would allow fetching it in rare case eg of manual spam review needed. however ROBOTS.txt only blocks indexing bots, not others from seeing/using those files. anyone can find non-indexed files by reading robots.txt so u could do so for review… confused.

Leave a Reply

Your email address will not be published. Required fields are marked *