Log in or Sign up
Digital Point
Home
Forums
>
Development
>
Site & Server Administration
>
robots.txt
Find help with setting up your robots.txt file on your web site.
Page 2 of 19
< Prev
1
←
2
3
4
5
6
→
19
Next >
Sort by:
Title
Start Date
Replies
Views
Last Message ↓
How do I unblock my robots.txt
scott2342
,
Nov 24, 2014
Replies:
3
Views:
11,522
meethi jain
Jun 3, 2015
Blocked Url's by robots.txt in google webmaster tool how to fix it?
Minesh Parikh
,
Apr 6, 2015
Replies:
0
Views:
17,929
Minesh Parikh
Apr 6, 2015
How to control when a bot(yahoo,google) can crawl a website
enikram
,
Mar 18, 2015
Replies:
4
Views:
18,754
Nigel Lew
Mar 18, 2015
Robots.txt code to disallow by default but allow domain name
Steviebone
,
Mar 4, 2015
Replies:
0
Views:
8,213
Steviebone
Mar 4, 2015
Will robots file help to cache the content in the web page
Prasseo
,
Jan 12, 2015
Replies:
1
Views:
8,241
Sycrid
Jan 18, 2015
How to create robots.txt for bookmarking website. Is it necessary?
wayne_hoton
,
Jan 11, 2015
Replies:
1
Views:
8,275
wayne_hoton
Jan 14, 2015
Webmaster Blocked URL's Understanding Issue
Jaden Alvin
,
Jul 9, 2014
Replies:
19
Views:
16,907
Jaden Alvin
Nov 24, 2014
can give an example of a good robots.txt to wordpress
Olvia
,
Nov 13, 2014
Replies:
0
Views:
8,013
Olvia
Nov 13, 2014
Is my robots.txt working fine?
aleale
,
Sep 10, 2014
Replies:
2
Views:
19,474
maddenitrous
Oct 3, 2014
How to block a specific folder in robots.txt file?
gvinaykumar
,
Jul 17, 2014
Replies:
0
Views:
9,472
gvinaykumar
Jul 17, 2014
Error - URLs restricted by robots.txt errors How Fix IT
linkladdy
,
Jul 2, 2014
Replies:
1
Views:
9,979
evildon
Jul 2, 2014
Can I add specific urls to robots.txt to stop google indexing them?
David Lowe
,
Jun 25, 2013
Replies:
2
Views:
9,850
BabyDoll123
Jun 19, 2014
How to clear these errors in robots.txt?
cathandrew
,
Feb 25, 2014
Replies:
1
Views:
9,008
BabyDoll123
Jun 19, 2014
how to dissalow an image?
mjfnd
,
Apr 29, 2014
Replies:
1
Views:
8,721
BabyDoll123
Jun 19, 2014
Magento Robots.txt file
BabyDoll123
,
Jun 12, 2014
Replies:
3
Views:
9,842
Nigel Lew
Jun 14, 2014
Blocking auto generated pages with robots.txt
imaginexhacked
,
Jun 9, 2014
Replies:
1
Views:
8,598
O-D-T
Jun 9, 2014
How can i find those website in which robots.txt disallow google spider?
jiangnancun
,
May 13, 2014
Replies:
2
Views:
9,218
neroux
May 13, 2014
What is robot.txt file?
James101
,
Apr 1, 2014
Replies:
4
Views:
9,504
John Dave
Apr 2, 2014
I have at-last found site that creates xml file of https sites
garry420
,
Mar 8, 2014
Replies:
0
Views:
8,168
garry420
Mar 8, 2014
how to disallow a folder from being indexed
bright776
,
Feb 1, 2014
Replies:
3
Views:
9,078
Ehteshamshaikh
Mar 7, 2014
Is it necessary to specify sitemap.xml in robots.txt file
cathandrew
,
Feb 25, 2014
Replies:
1
Views:
8,816
Ehteshamshaikh
Feb 27, 2014
Things you should be doing to block bad BOTS.
manish_khanna
,
Feb 13, 2014
Replies:
0
Views:
8,591
manish_khanna
Feb 13, 2014
Dissalow folder with the same name as allowed folder
SebastienD
,
Jan 15, 2014
Replies:
2
Views:
9,336
Nette
Feb 2, 2014
Robots.txt File Help
vatu
,
Jan 15, 2014
Replies:
8
Views:
8,638
alsgroup
Jan 23, 2014
Robots.txt code
Sunil Keshari
,
Jan 9, 2014
Replies:
3
Views:
9,950
ashokdhan
Jan 14, 2014
I need help on how to Noindex categories, tags, author, archive
Vel
,
Jul 25, 2013
Replies:
6
Views:
8,809
tymsh
Dec 9, 2013
Please review my robots.txt code
chadhaajay
,
Oct 10, 2013
Replies:
2
Views:
8,213
makeit easy
Nov 17, 2013
Would a robot.txt fix this SEO issue?
rob587
,
May 10, 2013
Replies:
19
Views:
10,976
msjaved
Nov 8, 2013
How to create Robots.txt Files?
Ricky Martin 3124
,
Oct 31, 2013
Replies:
1
Views:
5,757
Sayeri
Oct 31, 2013
Googlebot can't access my site (Robot Txt)
yourarnav
,
Sep 25, 2013
Replies:
4
Views:
6,662
shakiba.proba
Oct 9, 2013
Google Webmaster, URLs restricted by robots.txt
suganux
,
Aug 13, 2013
Replies:
4
Views:
6,289
suganux
Aug 23, 2013
Need Help with Robots.txt
PalakBhatt83
,
Jul 24, 2013
Replies:
4
Views:
5,693
blogdum
Aug 12, 2013
Robots.txt file can't be seen but SEOQuake says it exists?
Goran Nikolov
,
Jul 11, 2013
Replies:
7
Views:
5,975
MokricaIrina
Jul 17, 2013
How to Remove 404 Error?
prabhjot.singh
,
Aug 26, 2012
Replies:
18
Views:
11,662
Rakeshraj
Jul 15, 2013
Development site, don't want search engines...how?
HiSoC8Y
,
Jul 12, 2013
Replies:
4
Views:
2,866
ryan_uk
Jul 13, 2013
Blocked URL checking
ntrdtr
,
Jun 11, 2013
Replies:
9
Views:
3,696
Kyle Stester
Jul 11, 2013
Locked
What is Robot.txt file?
w3seotraining
,
Jul 3, 2013
Replies:
2
Views:
2,887
Vincentas
Jul 8, 2013
Want to block other domain from crawling
tanmay.kamath1992
,
Jun 22, 2013
Replies:
5
Views:
2,990
Stephen Moyers
Jul 4, 2013
Robots.txt - What is the proper format for a Crawl Delay for multiple user agents?
sam223
,
Jun 29, 2013
Replies:
0
Views:
4,346
sam223
Jun 29, 2013
When I search my blog on SE-I saw this-User-agent:Mediapartners-Google: * Disallow:
beafriend
,
May 13, 2013
Replies:
3
Views:
4,818
BobbyMcNiel
Jun 3, 2013
Robots.txt security protection setup
TiffanyJ.SSS
,
May 7, 2013
Replies:
2
Views:
2,791
ryan_uk
May 15, 2013
Help regarding Robot.txt
sam2220
,
Mar 19, 2013
Replies:
3
Views:
1,853
topcoder
May 14, 2013
Stop Bing from indexing my website
Tony Brar
,
Apr 22, 2013
Replies:
10
Views:
5,529
Revelations-Decoder
May 5, 2013
How to prevent robots.txt from being indexed?
Devtard
,
Apr 17, 2013
Replies:
5
Views:
1,578
ryan_uk
May 3, 2013
IS robots.txt is safe for site?
Sanjeev SIngh
,
Feb 21, 2013
Replies:
8
Views:
2,231
blogdum
Apr 25, 2013
How to allow crawling a URL with a certain parameter
Devtard
,
Apr 16, 2013
Replies:
1
Views:
1,172
Devtard
Apr 17, 2013
How Do I Ban Google From Seeing A Folder?
Mr.Dog
,
Jan 16, 2013
Replies:
8
Views:
2,250
MayurKawale
Mar 29, 2013
Block a URL using Robots.txt
viveknathr
,
Mar 24, 2013
Replies:
3
Views:
1,902
indyonline
Mar 24, 2013
right and default format of robots.txt
zeel_shah
,
Mar 13, 2013
Replies:
2
Views:
1,930
ryan_uk
Mar 14, 2013
disallow:/?
msaeedkk
,
Mar 7, 2013
Replies:
6
Views:
2,316
ryan_uk
Mar 14, 2013
Showing threads 51 to 100 of 911
Thread Display Options
Sort threads by:
Last message time
Thread creation time
Title (alphabetical)
Number of replies
Number of views
First message likes
Order threads in:
Descending order
Ascending order
Loading...
(You must log in or sign up to post here.)
Show Ignored Content
Page 2 of 19
< Prev
1
←
2
3
4
5
6
→
19
Next >
Log in with Facebook
Your name or email address:
Do you already have an account?
No, create an account now.
Yes, my password is:
Forgot your password?
Stay logged in
Digital Point
Home
Forums
>
Development
>
Site & Server Administration
>
Log in to view your Analytics, AdSense & PayPal stats
Home
Forums
Search Forums
Recent Posts
Spy
Search Engines
Google
AdSense
Guidelines / Compliance
Placement / Reviews / Examples
Reporting & Stats
Payments
AdWords
Analytics
Google+
Sitemaps
Google API
Shopping
Yahoo
Publisher Network
Search Marketing
Yahoo API
Bing
Bing Ads
All Other Search Engines
Directories
Solicitations & Announcements
ODP / DMOZ
Business & Marketing
General Business
General Marketing
Search Engine Optimization
Keywords
Microdata
Payment Processing
PayPal
Social Networks
Facebook
Facebook API
Twitter
Google+
Link Development
Legal Issues
Domain Names
Appraisals
Copywriting
Article Marketplace
eCommerce
Pay Per Click Advertising
Digital Point Ads
Google AdWords
Yahoo Search Marketing
Bing Ads
Affiliate Programs
Affiliate Program Management
Commission Junction
Google
eBay
Amazon
ClickBank
Chitika
Design
HTML & Website Design
CSS
Graphics & Multimedia
Photoshop
Reviews
Content Management
Blogging
WordPress
Drupal
Joomla
XML & RSS
Scripts
Development
Programming
PHP
JavaScript
jQuery
C#
Ruby
Site & Server Administration
Apache
Nginx
Security
Traffic Analysis
Google Analytics
robots.txt
Forum Management
XenForo
vBulletin
phpBB
Cluster Computing
Google Sitemaps
Web Hosting
Outages
Advertising
Databases
MySQL
Buy, Sell or Trade
Sites
Filter Sites For Sale
Domains
Filter Domains For Sale
Design Contests
Advertising
Secondhand Licenses
Services
Content Creation
Design
Programming
Traffic
Web Hosting
Plug-ins/Add-ons
Templates/Styles
Scripts
Designs
eBooks
Articles
The Digital Point
Introductions
General Chat
Movies, Music & TV
Games
Sports
Hardware
Politics & Religion
Products & Tools
Keyword Tracker
Cookie Search
Better Analytics
Legacy
Co-op Advertising Network
Optigold ISP
Reviews
Websites
Design
SEO
Support & Feedback
Advertising
Keyword Tracker
Cookie Search
The Underground
Area 51
Sphinx Search Support
XenForo Search Support
Tools
Most used tools...
Rank Checker
Cookie Search
Site Search Engine
Marketplace
Sites
Domains
Plugins/Add-ons
Scripts
Templates/Styles
Graphic Designs
eBooks
Articles
Advertising
Buy Ad Space
Sell Ad Space On Your Sites
Advertise on digitalpoint.com
Members
Notable Members
Current Visitors
Recent Activity
User Map
Directory
New Profile Posts
Menu
Search
Search titles only
Posted by Member:
Separate names with a comma.
Newer Than:
Search this forum only
Display results as threads
Useful Searches
Recent Posts
More...