Knowledge Base Hub

Browse through our helpful how-to guides to get the fastest solutions to your technical issues.

Home  >  cPanel  >  How to Block Bots with robot.txt File via cPanel?
Top Scroll

How to Block Bots with robot.txt File via cPanel?

 2 min

Bot traffic is non-user traffic that can hurt your web performance significantly. It is essential to safeguard your website from abusive bot traffic and, there are a few ways to do it. In this article, you will learn to block bots using the robot.txt file with your cPanel account.

Located in the root directory of your website, robot.txt is a text file that enables the search engine bots to crawl and index your website. Apparently, it also decides what files should be accessible to the bots making its configuration essential. It is also why you can restrict these bots from indexing your website in the search engines using the robot.txt file.

To Block Bots Using Robot.txt File

1. Log in to your cPanel account.

2. Go to the FILES section and click on File Manager.

The File Manager interface will open. You will have to create the robot.txt file.

3. Click the +File button on the top left corner of the interface.

4. Enter the file name as robot.txt and click the Create new file button.

5. Locate and right-click on the robot.txt file. Select the Edit option.

The file will open for you to enter/edit its information.

6. Copy this code in the file.

RewriteEngine On
RewriteCond %{HTTP_USER_AGENT} Baiduspider [NC]
RewriteRule .* – [F,L]

This code will block the bot from the Baidu search engine.

7. Click the Save Changes button and Close the file.

With this robot.txt file, the Baidu search engine cannot use bots on your website.


You can also add the bot crawler of any other search engine to the code preventing it from indexing your website.

In short, you can even block bots from other search engines with the robot.txt file by editing it via cPanel File Manager.

For our Knowledge Base visitors only
Get 10% OFF on Hosting
Special Offer!
Claim the discount before it’s too late. Use the coupon code:
Note: Copy the coupon code and apply it on checkout.