Fetcher
BazQux Fetcher is how BazQux Reader grabs RSS/Atom feeds and comments when users choose to subscribe to your blog in BazQux Reader. Fetcher collects and periodically refreshes these user-initiated feeds. Find answers below to some of the most commonly asked questions about how this user-controlled feeds and comments grabber works.
How do I request that BazQux not retrieve some or all of my site’s feeds?
When users subscribe to your feed, BazQux fetcher attempts to obtain the content of the feed in order to display it. Since fetcher requests come from explicit action by human users, and not from automated crawlers, fetcher does not follow robots.txt guidelines.
If your feed is publicly available, BazQux can’t restrict users from accessing it. One solution is to configure your site to serve a 404, 410, or other error status message to user-agent BazQux
If your feed is provided by a blog or site hosting service, please, work directly with that service to restrict access to your feed.
How often will Fetcher retrieve my feeds?
Fetcher shouldn’t retrieve feeds from most sites more than once every hour on average. Some frequently updated sites may be refreshed more often. Note, however, that due to network delays, it’s possible that Fetcher may briefly appear to retrieve your feeds more frequently.
Why is Fetcher trying to download incorrect links from my server, or from a server that doesn’t exist?
Fetcher retrieves feeds at the request of users who have subscribed to them in BazQux Reader. It is possible that a user has requested a feed URL location that does not exist.
Why is Fetcher downloading information from our “secret” web server?
Fetcher retrieves feeds at the request of users who have added them to their BazQux. It is possible that the request came from a user who knows about your “secret” server or typed it in by mistake.
Why isn’t Fetcher obeying my robots.txt file?
Fetcher retrieves feeds only after users have explicitly subscribed to them in BazQux Reader. Fetcher behaves as a direct agent of the human user, not as a robot, so it ignores robots.txt entries. Fetcher does have one special advantage, though: because it’s acting as the agent of multiple users, it conserves bandwidth by making requests for common feeds only once for all users.
Why are there hits from multiple machines at bazqux.com, all with user-agent BazQux?
Fetcher was designed to be distributed on several machines to improve performance and scale as the web grows.
Can you tell me the IP addresses from which Fetcher makes requests so that I can filter my logs?
The IP addresses used by Fetcher change from time to time. The best way to identify accesses by Fetcher is to use its identifiable user-agent: BazQux.
Why is Fetcher downloading the same page on my site multiple times?
In general, Fetcher should only download one copy of each file from your site during a given feed retrieval. Very occasionally, the machines are stopped and restarted, which may cause it to again retrieve pages that it’s recently visited.
Do you support push technology?
Yes. BazQux Reader support push hubs. If your feeds advertise a push hub, Fetcher will subscribe for updates and reduce the number of polls to three times a day.
My Fetcher question isn’t answered here. Where can I get more help?
If you’re still having trouble, try posting your question to support at bazqux.com.
Home