Set up a new index
  • 09 Sep 2021
  • 3 Minutes to read
  • Contributors
  • Dark
    Light
  • PDF

Set up a new index

  • Dark
    Light
  • PDF

Article Summary

Before you can use the Product Matching feature to map products you need to set up your product index. A product index is a contained project with correlating robots, configurations, and fields.

To create a new index:

  1. Click Products.
  2. Click Create new product index.
  3. Name your new index.
  4. Open your newly created index from the list by clicking the name.

There are three additional parts to setting up a new index:

  • Set up property groups
  • Add sources to your index
  • Establish index settings

Set up property groups

After you create a new index, you will need to complete additional setup in Properties for the index. This includes creating property groups.

Property groups are identifiers that you want to be part of your product data. These identifiers can be data such as “InStock”, “Color”, “Size” and more. For each property group you need to decide if the property should be part of what makes a product unique and if the property should be required. Marking a property group as unique is part of the deduplication process.

  1. Open your new index.
  2. Click the Properties tab.
  3. Click Add property group.

If you add a new property group to an existing index the newly added property group will display in the Review Products tab when new data is generated.

  1. Name the property group.
  2. Select a Type from the drop-down.
  • Dictionary types are mappable. You can use these for properties like color and size.
  • Value types are not mappable. You can use these for tracking and sorting.
  1. Select if the property is required.
Beware required properties!

If the property is required, products that are missing this property will automatically be flagged as invalid.

  1. Select if the property group is unique. If the property is unique, it will be included as part of the key that comprises a unique product.
Note

The default key is created from the product URL only. If you are extracting multiple variants from the same page (e.g., multiple colors or sizes) you should mark those properties as unique. That means the key will be created from “URL + colour + size” for instance.

Add sources to an index

To generate data for a product index you need to add sources. A source consists of two or three robots that serve a specific purpose.

After you have created a product index:

  1. Click the Sources tab.
  2. Click Add source.
  3. Name your source.
  4. Select a country from the drop-down. This allows you to organize the sources in your systems by their geographic location.
  5. Next you’ll need to select two or three robots that will gather the data for the index:
    • Product URLs: This robot should gather the product URL for the source — and only the source. Ideally the robot will only fetch the URLs of products that you want, so be as specific as possible.
    • Product Info: This robot takes the URL as input and should be set up to gather product details that will be added to the index. The product info robot should gather at least one row of product data, but if a URL contains several alternatives (colours, size, etc.), the product info robot should gather all of them and output them as a row per alternative.
    • Robot Check: This robot is optional and can be used to auto-review everything that was generated by the Product URL robot. The purpose is to gather the same URLs as the Product URLs robot but in a secondary way. Essentially to “check your work.” This is often implemented using a Crawler robot since that will be less specific - and also less likely to miss a product url. The challenge is to keep its result size low and the robot fast - and still be sure it finds everything.
  6. If applicable, enter text in the Ignore Words field. If the title of a product contains a word in this list the product index will automatically ignore that entry.
  7. Click Save. The source will automatically be executed when you create it. Once it’s done running you can begin the product matching process.

Was this article helpful?