Log in or Sign up
Digital Point
Home
Forums
>
Development
>
Site & Server Administration
>
robots.txt
Find help with setting up your robots.txt file on your web site.
Page 1 of 19
1
←
2
3
4
5
6
→
19
Next >
Sort by:
Title
Start Date
Replies
Views
Last Message ↓
Robots
Johnsmi2323
,
Aug 23, 2023
Replies:
1
Views:
1,498
seomanualsubmission
Aug 24, 2023
How to check robots.txt on site?
seoworld1234
,
Apr 21, 2008
...
2
Replies:
33
Views:
83,409
imranbashir.seo
Mar 7, 2023
Robots.txt is blocking my labels
Mrinal_Roy
,
Feb 5, 2015
Replies:
5
Views:
14,502
Gideon Duah
Jul 21, 2021
Is it good to block BaiduSpider from accessing/crawling the site?
MayurKawale
,
Jan 7, 2021
Replies:
1
Views:
2,919
qwikad.com
Jan 10, 2021
importance of robots.txt
kabali
,
Sep 27, 2016
Replies:
5
Views:
6,384
MayurKawale
Nov 24, 2020
Disallow all Bots
dabidovic123
,
Oct 13, 2019
Replies:
4
Views:
4,670
boshiant
Jun 6, 2020
Ban subdomain from robots, stop Google from indexing it
Mr.Dog
,
Dec 1, 2018
Replies:
6
Views:
4,712
Borislav Arapchev
May 13, 2020
How to Block All Robots to Crawl/Index All html Pages within My Website
vema123
,
Jul 9, 2019
Replies:
2
Views:
3,712
mmerlinn
Jul 9, 2019
Robots is good
gree124
,
May 2, 2019
Replies:
0
Views:
3,372
gree124
May 2, 2019
Is there a robots.txt interpreter for those of us who are newbies?
Melissa2007
,
Apr 29, 2019
Replies:
2
Views:
3,351
Melissa2007
Apr 30, 2019
Why do I need a robots.txt , and what to put in it?
Melissa2007
,
Feb 23, 2019
Replies:
8
Views:
4,368
mmerlinn
Feb 26, 2019
Want to exclude TEST folder from Google indexing
Melissa2007
,
Feb 3, 2019
Replies:
8
Views:
3,957
Melissa2007
Feb 4, 2019
Not Indexing Well. Is it robots.txt?
rosn007
,
Dec 14, 2018
Replies:
3
Views:
4,516
Joshuathompson
Jan 3, 2019
Issue Regarding Robots.txt
Kamaldeep Singh SEO
,
Oct 24, 2018
Replies:
4
Views:
5,340
sigen
Oct 25, 2018
Locked
asking you for a information
ronkoo
,
Mar 13, 2018
Replies:
4
Views:
8,941
hostechsupport
Aug 17, 2018
Help with Robots.txt
infopages
,
Sep 29, 2017
Replies:
4
Views:
8,212
Elevate
Aug 3, 2018
Google or robots.txt problem
macau2009
,
Feb 19, 2018
Replies:
6
Views:
4,906
qwikad.com
Feb 21, 2018
Help with robots.txt file
Divvy
,
Jun 14, 2017
Replies:
4
Views:
5,021
Arun Kumar Soni
Sep 24, 2017
Vbulletin 4 robots txt?
rants1990
,
Sep 19, 2017
Replies:
3
Views:
4,338
sarahk
Sep 19, 2017
Robots.txt for SMF Forum
pradip200
,
Mar 23, 2013
Replies:
4
Views:
18,604
Svoboda
Jun 30, 2017
Robot.txt for Google (error)
wartes
,
Apr 10, 2017
Replies:
6
Views:
13,425
Md.Saddam Hossain
May 7, 2017
Robot.txt or remove the page
Mubashir Ahmad
,
Mar 21, 2017
Replies:
6
Views:
6,352
Tutorials Feed
Apr 19, 2017
Robots.txt Question
PaRaDiGm13
,
Mar 16, 2016
Replies:
4
Views:
6,717
Cado Magenge
Feb 28, 2017
Noindex and robots disallow
smokey99
,
Dec 22, 2016
Replies:
2
Views:
5,315
PoPSiCLe
Feb 24, 2017
List of Bad Bots to disallow?
hmansfield
,
May 20, 2007
Replies:
14
Views:
24,664
Sergio Minozzi
Feb 22, 2017
Locked
robot.txt
Abdullah adil
,
Feb 11, 2017
Replies:
6
Views:
13,914
sarahk
Feb 16, 2017
Can someone tell me what is wrong with my robots.txt file?
Boxwell
,
Nov 24, 2016
Replies:
2
Views:
5,571
GameO.Over
Nov 24, 2016
I have default robots.txt and asking how to customize it
dinrock
,
Oct 13, 2016
Replies:
0
Views:
5,320
dinrock
Oct 13, 2016
Problem with file
shoprat
,
Sep 23, 2016
Replies:
2
Views:
4,916
JeffMichaels
Sep 23, 2016
Disallow google to index specific tekst with Robot.txt on webpage?
Noobnl
,
Apr 10, 2016
Replies:
4
Views:
5,916
Furquan Ahmed
Aug 25, 2016
What is the correct way to write my “robots.txt” file?
Official Gates
,
Aug 16, 2016
Replies:
0
Views:
4,568
Official Gates
Aug 16, 2016
Addon Domains Showing With Main Domain In URL
sensor88
,
Aug 9, 2016
Replies:
0
Views:
5,107
sensor88
Aug 9, 2016
Robots.txt Issues
ketanseo23
,
Apr 14, 2016
Replies:
1
Views:
6,072
MayurKawale
Aug 4, 2016
Problem mobile friendly test
irfansyah
,
Jun 23, 2016
Replies:
8
Views:
5,839
Host Little
Jul 14, 2016
Block a folder except some URL's
soseooooo
,
Mar 9, 2016
Replies:
1
Views:
6,073
MayurKawale
Jun 29, 2016
Robots.txt
Angelabailey
,
Nov 15, 2013
Replies:
15
Views:
19,432
Panda123
Mar 27, 2016
Robots.txt has 'Disallow: /forums/-/' line, what is the meaning
Xfactormaster
,
Mar 1, 2016
Replies:
2
Views:
6,941
stilliard
Mar 7, 2016
Robots.txt Help to Block a Particular File Name
Kushal Biswas
,
Feb 3, 2016
Replies:
1
Views:
16,399
Anubhav-soin
Feb 25, 2016
Do you block Majestic bots?
Jeffr2014
,
Apr 9, 2015
Replies:
4
Views:
16,370
neteater
Feb 20, 2016
Robots.txt error
Harris96
,
Jan 24, 2016
Replies:
1
Views:
7,395
Nigel Lew
Jan 24, 2016
How to add robots.txt file in Magento
anna30
,
Oct 16, 2015
Replies:
1
Views:
8,948
paco78
Oct 16, 2015
robots.txt please check
gree124
,
Nov 27, 2014
Replies:
7
Views:
11,418
rajendra1682
Oct 6, 2015
Why cant Google access my robots.txt ?????
MoneyPlanters
,
Sep 27, 2015
Replies:
1
Views:
17,937
Mit Thakkar
Sep 27, 2015
i can't stop Baidu, pls help!!!
web_master
,
Dec 27, 2011
Replies:
7
Views:
14,087
RoundShots
Sep 11, 2015
Fix to ban google search from Cloaking and/or sneaky redirects
MarcL
,
Jul 30, 2015
Replies:
4
Views:
18,589
MarcL
Aug 3, 2015
Robot text and partial fetch issues - need suggestions
full_time_writer
,
Jul 28, 2015
Replies:
0
Views:
9,339
full_time_writer
Jul 28, 2015
Very Strange Problem
Marker1982
,
Jul 21, 2015
Replies:
3
Views:
18,697
Jacob.k
Jul 25, 2015
How to create a robots file for PHP webpage to crawl
Prasseo
,
Jan 12, 2015
Replies:
3
Views:
11,354
abinjoy
Jul 14, 2015
how to block all other BOT except Google, MSN, Yahoo
danieloffice
,
Jan 9, 2008
...
2
Replies:
23
Views:
68,108
moneyroad
Jul 10, 2015
Robot.txt file not found on my blog
myfastrupee
,
Jun 21, 2013
Replies:
4
Views:
13,773
meethi jain
Jun 3, 2015
Showing threads 1 to 50 of 911
Thread Display Options
Sort threads by:
Last message time
Thread creation time
Title (alphabetical)
Number of replies
Number of views
First message likes
Order threads in:
Descending order
Ascending order
Loading...
(You must log in or sign up to post here.)
Show Ignored Content
Page 1 of 19
1
←
2
3
4
5
6
→
19
Next >
Log in with Facebook
Your name or email address:
Do you already have an account?
No, create an account now.
Yes, my password is:
Forgot your password?
Stay logged in
Digital Point
Home
Forums
>
Development
>
Site & Server Administration
>
Log in to view your Analytics, AdSense & PayPal stats
Home
Forums
Search Forums
Recent Posts
Spy
Search Engines
Google
AdSense
Guidelines / Compliance
Placement / Reviews / Examples
Reporting & Stats
Payments
AdWords
Analytics
Google+
Sitemaps
Google API
Shopping
Yahoo
Publisher Network
Search Marketing
Yahoo API
Bing
Bing Ads
All Other Search Engines
Directories
Solicitations & Announcements
ODP / DMOZ
Business & Marketing
General Business
General Marketing
Search Engine Optimization
Keywords
Microdata
Payment Processing
PayPal
Social Networks
Facebook
Facebook API
Twitter
Google+
Link Development
Legal Issues
Domain Names
Appraisals
Copywriting
Article Marketplace
eCommerce
Pay Per Click Advertising
Digital Point Ads
Google AdWords
Yahoo Search Marketing
Bing Ads
Affiliate Programs
Affiliate Program Management
Commission Junction
Google
eBay
Amazon
ClickBank
Chitika
Design
HTML & Website Design
CSS
Graphics & Multimedia
Photoshop
Reviews
Content Management
Blogging
WordPress
Drupal
Joomla
XML & RSS
Scripts
Development
Programming
PHP
JavaScript
jQuery
C#
Ruby
Site & Server Administration
Apache
Nginx
Security
Traffic Analysis
Google Analytics
robots.txt
Forum Management
XenForo
vBulletin
phpBB
Cluster Computing
Google Sitemaps
Web Hosting
Outages
Advertising
Databases
MySQL
Buy, Sell or Trade
Sites
Filter Sites For Sale
Domains
Filter Domains For Sale
Design Contests
Advertising
Secondhand Licenses
Services
Content Creation
Design
Programming
Traffic
Web Hosting
Plug-ins/Add-ons
Templates/Styles
Scripts
Designs
eBooks
Articles
The Digital Point
Introductions
General Chat
Movies, Music & TV
Games
Sports
Hardware
Politics & Religion
Products & Tools
Keyword Tracker
Cookie Search
Better Analytics
Legacy
Co-op Advertising Network
Optigold ISP
Reviews
Websites
Design
SEO
Support & Feedback
Advertising
Keyword Tracker
Cookie Search
The Underground
Area 51
Sphinx Search Support
XenForo Search Support
Tools
Most used tools...
Rank Checker
Cookie Search
Site Search Engine
Marketplace
Sites
Domains
Plugins/Add-ons
Scripts
Templates/Styles
Graphic Designs
eBooks
Articles
Advertising
Buy Ad Space
Sell Ad Space On Your Sites
Advertise on digitalpoint.com
Members
Notable Members
Current Visitors
Recent Activity
User Map
Directory
New Profile Posts
Menu
Search
Search titles only
Posted by Member:
Separate names with a comma.
Newer Than:
Search this forum only
Display results as threads
Useful Searches
Recent Posts
More...