Crawler information
SchwanerCoBot
Schwaner & Co. maintains a public record of distribution partnerships between companies in fintech, lending and insurance. SchwanerCoBot is the crawler that reads public pages to build it.
If this crawler has visited a site you run and you would rather it did not, write to the address below and it will be blocked. No reason is needed and none will be asked for.
Identifying it in your logs
Every request carries this user agent, and a From header with the same address.
SchwanerCoBot/1.0 (+https://schwanerco.com/bot)What it reads
A small number of pages that are public to any visitor:
- A newsroom or press release index.
- A partner, integrations or marketplace page.
- A public job board.
What it does not do
- It does not sign in. It reads nothing that requires an account.
- It does not submit forms or trigger any action on your site.
- It does not read LinkedIn. That domain is refused before a request is made.
- It does not collect names, email addresses, phone numbers, or anything else about individual people.
- It does not crawl your site broadly. It reads the pages above and stops.
How often it visits
- At most a handful of pages per company, and not more than once a month in normal operation.
- At least fifteen seconds between requests to the same site.
- If your robots.txt sets a Crawl-delay slower than that, yours is used instead.
- A 429 response stops it for at least a minute, and longer each time. A Retry-After header is honoured.
robots.txt
Your robots.txt is read before any page on your site, cached for a day, and obeyed. Three cases are worth stating plainly:
- No robots.txt means no restriction, and the pages above may be read.
- A robots.txt that returns an error, or cannot be reached, is treated as a refusal. Not knowing is treated as no.
- A group naming SchwanerCoBot is used in preference to the wildcard group.
Blocking it
Add this to your robots.txt. It takes effect within a day.
User-agent: SchwanerCoBot
Disallow: /If you would rather not wait, or would rather not edit robots.txt, write to the address below and the domain is added to a block list that the crawler cannot bypass.
Corrections
If something recorded about your company is wrong, write to the same address. A correction from someone at the company concerned is treated as the strongest evidence available.
Contact
Page last updated 25 August 2026.