CrawlWP works on WordPress Multisite, but its configuration is handled separately for each site in the network. This means each site has its own SEO settings, indexing data, sitemaps, redirects, and API connections.
How CrawlWP is set up on a network
CrawlWP does not provide network-wide settings. Open an individual site’s dashboard to manage that site’s CrawlWP configuration and SEO data.
You can make CrawlWP available across the network in either of two ways:
- Use Network Admin > Plugins and choose Network Activate to activate CrawlWP for every site.
- Activate the plugin separately from the Plugins screen of individual sites.
When CrawlWP is network-activated, it sets itself up on up to 100 sites immediately. Other sites are set up when someone first opens their dashboard.
What is managed separately for each site?
| Feature | Multisite behavior |
|---|---|
| SEO titles, descriptions, schema and social settings | Managed separately for each site |
| Sitemaps | Each site has its own sitemap at its own address, such as example.com/site2/wp-sitemap.xml |
| llms.txt | Managed separately for each site |
| Redirects and 404 monitor | Managed separately for each site |
| Indexing and API connections | Managed separately for each site. Google, Bing, and Yandex must be connected for each site where they are used |
| robots.txt | Depends on whether the network uses subdomains or subfolders; see below |
| Header & Footer Code | Configured per site, but only network administrators can change the script boxes |
robots.txt behaves differently on subdomains and subfolders
Search engines read robots.txt from the root of a domain. That makes the type of Multisite network important when you are using CrawlWP’s robots.txt settings.
- With a subdomain network, such as
site2.example.com, each site’s robots.txt can work independently. - With a subfolder network, such as
example.com/site2/, only the main site’s robots.txt is used. The Robots.txt settings on the other sites do not affect what search engines receive.
For more information about changing the file, see Editing robots.txt.
Header & Footer Code permissions
The script fields in Header & Footer Code require permission to save unfiltered HTML. On a Multisite network, only network administrators have this permission. Site administrators can see the fields, but their changes are not saved.
The dedicated GA4 and Google Tag Manager ID fields are different: site administrators can use those fields.
Using CrawlWP Premium across the network
Premium licensing is also handled on a per-site basis. Each site has its own License tab, and Premium must be activated on every site where you want to use the Premium features.
Activation is tied to each site’s address, so check that your license includes enough sites for the network.
What happens when CrawlWP is uninstalled?
The Settings > Advanced > Remove Data on Uninstall option determines what happens to CrawlWP’s data when the plugin is deleted, but Multisite has some additional behavior.
- If Remove Data on Uninstall is enabled on the main site, CrawlWP deletes its data from every site when the plugin is deleted, regardless of the setting on the individual sites.
- If the option is disabled on the main site, each site follows its own Remove Data on Uninstall setting.
- On networks with more than 10,000 sites, CrawlWP does not remove data from any site.
When an individual site is deleted from the network, CrawlWP removes that site’s indexing log. Its redirects and 404 log tables remain in the database.
Deleting CrawlWP Premium always removes the license details and SEO Indexing data from every site, regardless of the Remove Data on Uninstall setting. The exception is a network with more than 10,000 sites.
Back up the database before deleting CrawlWP from a Multisite network.
Troubleshoot common Multisite issues
/llms.txt or another CrawlWP address shows “page not found”. Open that site’s dashboard first. Then go to Settings > Permalinks and click Save Changes to refresh WordPress’s address rules.
A site’s robots.txt changes have no effect. Check whether your network uses subfolders. On a subfolder network, search engines use only the main site’s robots.txt, so the robots.txt settings of the other sites are not used.
A site administrator cannot save scripts. This is expected on Multisite. Only network administrators can save the script fields in Header & Footer Code. Site administrators can still use the GA4 and Google Tag Manager ID fields.
Premium features are missing on one site. Open that site’s License tab and make sure CrawlWP Premium is activated there. Premium activation is separate for each site.
Keep site-level configuration in mind
The easiest way to manage CrawlWP on Multisite is to treat every site as a separate CrawlWP installation. Configure its SEO settings, API connections, indexing rules, sitemaps, redirects, and Premium license from that site’s dashboard.
The main exception to this site-by-site model is functionality that depends on the network or domain structure, such as robots.txt and permissions for unfiltered HTML. Check those areas before assuming a setting on one site applies to the rest of the network.