WPeMatico includes duplicate controls that are enabled by default. They avoid creating the same post twice and also save time and resources: when a campaign finds a duplicate, it stops reading the feed, because it assumes that all the following items, which are older, were already fetched.
You can adjust these controls for all the campaigns in WPeMatico → Settings, in the Advanced Fetching box, or for a single campaign in its Duplicate controls box by checking Don’t use general Settings.
WPeMatico compares the title of each feed item with the posts that already exist on your site, of the same post type that the campaign creates. If there is a post with the same title, the item is a duplicate. It works with all the posts, whether they were created by hand or by a campaign.
If your feed has different items with the same title, check Deactivate duplicate controls and then Allow duplicates titles. The control by hash keeps working, so the same item won’t be fetched twice.
For each feed, the campaign saves a code (hash) of the address of the last item it fetched. When it finds that item again, it stops reading the feed. This control helps when the title control fails, for example if the source site changes the titles.
Since the hash only remembers the last item, if you allow duplicated titles and duplicated hashes, the campaign will fetch the same items again and again.
Note: The hash is saved in the campaign. If you delete a post to fetch it again, delete also the hash. You can do it in two ways: edit the campaign and, in the Campaign Control Panel, click the Delete hash code for duplicates button; or go to WPeMatico → Settings → Advanced, check Enable Del Hash in the Enable Features box, and use the Del Hash link of the campaign in the campaigns list.
Add an extra duplicate filter by source permalink in meta field value checks, for each item, if any post was already fetched from the same source address. It’s useful with feeds that are not standard, but it reads the database for each item, so it’s not recommended for large sites.
The items of a feed are ordered from newest to oldest. When the campaign finds the first duplicate, all the following ones were fetched before, so it stops. With this option, the campaign jumps over the duplicates and reads the whole feed every time, looking for new items. It remembers the last 20 items of each feed to skip them.
This is bad for performance, so use it only with feeds that publish new items in the middle of the list. See more in Jump duplicate posts.
- The feed is not standard. Check it with the W3C Feed Validator. Feeds without a fixed address for each item (the
<link>or<guid>tags) usually break the duplicate controls. - Custom Titles. If you use the Custom Title feature of the Professional add-on, the posts are saved with a different title, so the control by title can’t find them.
- Cache. Some cache systems show an old version of the feed or of your site. See Sync problems or duplicated contents.
- Firewalls or services like Cloudflare shouldn’t affect the duplicate controls, but some configurations might.
If you still have problems, open a case in our support ticket system and we’ll be happy to help you.
