Finding equivalent products between catalogs
  • 22 Oct 2020
  • 7 Minutes To Read
  • Print
  • Share
  • Dark
    Light

Finding equivalent products between catalogs

  • Print
  • Share
  • Dark
    Light

In this tutorial, you will learn how to match products between different sources by using Aito.

Before you begin

Getting an Aito instance

If you want to follow along with this tutorial, you'll have to go and get your own Aito instance from the Aito Console.

  1. Sign in or create an account, if you haven't got one already.
  2. In the Aito Console go to the instances page and click the "Create an instance" button.
  3. Select the instance type you want to create and fill in needed fields, Sandbox is the free instance for testing and small projects. Visit our pricing page, to learn more about the Aito instance types.
  4. Click "Create instance" and wait for a moment while your instance is created, you will receive an email once your instance is ready.

If you're totally new to Aito the get started guides might be useful before reading this tutorial.

Accessing instance details

After the instance creation is ready, you can access the URL and API keys of the instance by

  1. Log in to Aito Console
  2. Click on the instance your created
  3. Go to the overview page. You can copy the API keys after clicking the eye icon.

api_info

Get the Aito Command Line Interface tool

  1. Get the Aito CLI tool if you don't have it yet (you'll need Python3.6 or higher installed)
pip install aitoai==0.4.0
  1. Configure the CLI using the instance URL and read/write API key.
aito configure

Use case: Finding equivalent products between catalogs

In this example case, there are two companies A and B which both build kitchens. The offerings of the two companies are complementary, meaning that you can buy frames from company A and the doors that fit the frames from company B. So the products of the two companies can be mixed and matched. If the product catalogs of the companies would stay stagnant, the matches could be just recorded to a relational database and fetched from there when needed. But both of the catalogs change contiunously so a more flexible solution is needed.

Aito fits the job as it can infer which products are complementary based on previous matches between other products. In Aito, things don't have to be explicitly stated in order to have an answer. For example, if company A has a black 60x80cm door that has been previously matched to company B's white 60x80cm door, Aito can make a match between company A's black 60x80cm door and company B's gray 60x80cm door without the match being explicitly stated in the DB. Aito also allows adding new products to the DB without the need of retraining the ML model which is needed in the traditional ML approaches.

Data

To be able to match products from company A to company B, Aito needs to have training data of existing matches between the products of the two companies. The company A products table consists of company A products that have an existing match to company B products. The company B products table consists of company B products that have an existing match to company A products. Match table has each company A product matched to company B products by their product IDs.

Company A products table (example)

itemID door_name type description height width door_height door_width color handle_placement
844 Paris door <p>Paris pearl 20x10 door<p> 10 20 97 197 pearl right
1501 Paris door <p>Paris oak 20x10 door<p> 10 20 97 197 oak right
1699 Paris door <p>Paris ash 20x10 door<p> 10 20 97 197 ash right

Company B products table (example)

partNumber name type metric
607 Cattern door Width: 19.7 cm<br/>Width when built: 20 cm<br/>Height when built: 10 cm<br/>Height: 9.7 cm<br/>Thickness: 1.4 cm<br/><br/>
906 Pompan door Width: 19.7 cm<br/>Width when built: 20 cm<br/>Height when built: 20 cm<br/>Height: 19.7 cm<br/>Thickness: 1.4 cm<br/><br/>
1838 Pompan door Width: 19.7 cm<br/>Width when built: 20 cm<br/>Height when built: 30 cm<br/>Height: 29.7 cm<br/>Thickness: 1.4 cm<br/><br/>

Match table (example)

companyA_product companyB_product
607 844
607 1501
607 1699
Note

The data is mock data so it may not fully behave as a real data set would.

Overview

The workflow will consist of two steps:

  1. Upload data into Aito.
  2. Match company A products to company B products.
  3. Adding new products

#1 Upload data

  1. Download the example dataset. There are three files you need to upload as three linked tables are created to Aito.
  1. Copy the example schemas into the corresponding files, notice how the linking of the tables has been done in the schema of the match table.

companyA_product_schema.json

{
	"type": "table",
	"columns": {
		"metric": {
			"type": "Text",
			"nullable": false,
			"analyzer": "english"
		},
		"name": {
			"type": "Text",
			"nullable": false,
			"analyzer": "whitespace"
		},
		"partNumber": {
			"type": "String",
			"nullable": false
		},
		"type": {
			"type": "String",
			"nullable": false
		}
	}
}

companyB_product_schema.json

{
	"type": "table",
	"columns": {
		"color": {
			"type": "String",
			"nullable": true
		},
		"description": {
			"type": "Text",
			"nullable": false,
			"analyzer": "english"
		},
		"door_height": {
			"type": "Int",
			"nullable": true
		},
		"door_name": {
			"type": "String",
			"nullable": true
		},
		"door_width": {
			"type": "Int",
			"nullable": true
		},
		"handle_placement": {
			"type": "String",
			"nullable": true
		},
		"height": {
			"type": "Int",
			"nullable": true
		},
		"itemID": {
			"type": "String",
			"nullable": false
		},
		"type": {
			"type": "String",
			"nullable": true
		},
		"width": {
			"type": "Int",
			"nullable": true
		}
	}
}

match_schema.json

 {
 	"type": "table",
 	"columns": {
 		"companyA_product": {
 			"type": "String",
 			"nullable": false,
 			"link": "companyA_product.partNumber"
 		},
 		"companyB_product": {
 			"type": "String",
 			"nullable": false,
 			"link": "companyB_product.itemID"
 		}
 	}
 }
  1. Upload the schemas
aito create-table companyA_product companyA_product_schema.json
aito create-table companyB_product companyB_product_schema.json
aito create-table match match_schema.json
  1. Upload data using the upload-file feature in the Aito CLI
aito upload-file -f csv companyA_product companyA_products.csv
aito upload-file -f csv companyB_product companyB_products.csv
aito upload-file -f csv match matches_companyA_companyB.csv

#2 Match company A products to company B products

Query

Use the match endpoint to match products between the companies.

aito match '{
	"from": "match",
	"where": {
		"companyA_product": {
			"name": "Åkeby",
			"type": "door",
			"metric": "Width: 39.7 cm<br/>Width when built: 40 cm<br/>Height when built: 10 cm<br/>Height: 9.7 cm<br/>Thickness: 1.4 cm<br/><br/>"
		},
		"companyB_product": {
			"door_name": "Helsinki",
			"color": "pearl"
		}
	},
	"match": "companyB_product"
}'

The from clause defines the table we're using for the prediction. Resembles FROM in SQL.

The where clause defines the prior information we have on the company A product by using propositions, e.g. "name": "Åkeby". The product does not have to exist in Aito in order to find a match for it. If the matching of products follows a certain pattern Aito is able to deduce the correct result from the products features. In the where clause you can also define that you're only interested in having in the results company B products that have a certain type and color, this is called hard filtering. When using a hard filter using the target data (companyB_product) the result space is restricted to only include rows with the defined values.

In the match clause we define the column from which we are trying to a match, in this case it's the companyB_product column which is linked to the companyB_product table. Match is able to read data behind links so that the information in the linked tables is used in the inference.

Result

{
  "offset": 0,
  "total": 58,
  "hits": [
    {
      "$p": 0.9306256960539888,
      "color": "pearl",
      "description": "<p>Helsinki pearl 40x10 door<p>",
      "door_height": 97,
      "door_name": "Helsinki",
      "door_width": 397,
      "handle_placement": "right",
      "height": 10,
      "itemID": "1156",
      "type": "door",
      "width": 40
    },
    {
      "$p": 0.04755437762421308,
      "color": "pearl",
      "description": "<p>Helsinki pearl 60x10 door<p>",
      "door_height": 97,
      "door_name": "Helsinki",
      "door_width": 597,
      "handle_placement": "right",
      "height": 10,
      "itemID": "1641",
      "type": "door",
      "width": 60
    },
    ...
   ]
},
...

In this example, you get that Helsinki pearl 40x10 door would be the best fit for the defined door with the probability of 93%, which would make sense that the door would match with a door of the same dimensions.

#3 Adding new products

The product catalogs of companies are ever-changing. In this chapter, we go through how to add products with new colors to Aito.

Uploading new products

If the schema of the new data is the same as for the old data, new data can just be appended to the existing table.

  1. Download the new products of company B from here
  2. Upload the data to your Aito instance
aito upload-file -f csv companyB_product companyB_new_products.csv 

Matching new products

Aito is able to use the new data in real-time, no retraining needed. So for the match query you used before, you can now try it with the new color feather grey.

aito match '{ 
	"from": "match", 
	"where": { 
		"companyA_product": { 
			"name": "Åkeby", 
			"type": "door", 
			"metric": "Width: 39.7 cm<br/>Width when built: 40 cm<br/>Height when built: 10 cm<br/>Height: 9.7 cm<br/>Thickness: 1.4 cm<br/><br/>" 
		}, \
		"companyB_product": { 
			"door_name": "Helsinki", 
			"color": "feather grey" 
		} 
	}, 
	"match": "companyB_product" 
}'

Aito then returns the correct sized Helsinki feather grey door:

{
  "offset": 0,
  "total": 35,
  "hits": [
    {
      "$p": 0.8216990289032818,
      "color": "feather grey",
      "description": "<p>Helsinki feather grey 40x10 door<p>",
      "door_height": 97,
      "door_name": "Helsinki",
      "door_width": 397,
      "handle_placement": "right",
      "height": 10,
      "itemID": "1247",
      "type": "door",
      "width": 40
    },
    ...
    ]
}

Creating your own data matching pipelines

If you're interested in making your own data matching pipelines using Aito, contact us at hello@aito.ai and tell us about your use case and we can see how we can help to bring your ideas to life!

Was This Article Helpful?