I always shop on Amazon, but for big purchases I wonder if I should use a shopping meta-search engine like pricegrabber.com or shopping.com. Theoretically they should always be at least as good as Amazon, but how are they in reality? And does it make sense to look on more than one?
It’s very tricky to scrape shopping sites to do this comparison. On Amazon you often have to put an item in to your shopping cart to see the price, or on pricegrabber.com you might need to check that the seller is a trusted merchant. Further, it’s difficult to know if you’re really finding the exact same item.
So I decided to try using Mechanical Turk to answer the question. I took 41 items from various Amazon best-seller feeds (specifically the Apparel, Baby Products, Photography and Wireless categories) and asked people to check the price on Amazon, EBay, PriceGrabber, and Shopping.com. To ensure accuracy I used multiple workers to check the work against each other.
It turns out that you can find cheaper deals on the shopping search engines, and in fact the different search engines find different better deals. Using PriceGrabber.com, you would find the item 90 percent of the time, and find a cheaper item from a reputable seller twenty percent of the time. Overall you would save an average of 6.3 percent of the cost on Amazon by checking PriceGrabber.com (note that this methodology doesn’t account for shipping which can obscure prices, especially on EBay).
| Website | Item Found? | Better Deal than Amazon? |
Average Savings |
Average Pct. Savings |
|---|---|---|---|---|
| Shopping.com | 90% | 28% | $0.37 | 2.90% |
| PriceGrabber.com | 90% | 20% | $1.15 | 6.30% |
| EBay.com | 68% | 38% | $1.54 | 7.20% |
| Combined | 98% | 58% | $2.29 | 11.40% |
Checking EBay, PriceGrabber and Shopping.com will find you a cheaper price than Amazon 58% of the time and on average you will save more than 11 percent of the cost. Seems like it pays to shop around a little, and it does make sense to try a few different websites. You can check out our raw data here.
brendano
12/20/08
This seems to indicate that
A meta-meta engine is better than a meta-engine.
This raises questions.
Is a meta-meta-meta engine better than a meta-meta engine?
Is a meta^k engine better than a meta^k-1 engine?
Does a meta^infinity engine converge on average savings?
(maybe yes: a thousand turkers would do little better than a hundred. i would guess.)
Need to know before Christmas.
-brendan
Ted Howard
12/20/08
Could you add Live Search Cashback (http://search.live.com/cashback) to that test? I, and I’m sure many others, would be curious to see the results.
I think Brendano’s logic is off. I say that a meta^infinity engine would likely converge on the arbitrage-free ratio under the assumption of open access to information. Of course, that’s excluding slippage, transaction feeds, taxes, … [efficient market derivatives humor]
Mike
12/20/08
Nice investigation. By the way, I noticed that both PriceGrabber and Shopping.com appear to take into account tax and shipping in their final prices.
Panos Ipeirotis
12/21/08
Also remember that when you are buying from any online merchant, you are not paying only for the product but also for the quality of the transaction (aka merchant reputation). Knowing that you are going to receive the item, knowing that the item will arrive on time, that it is easy to return, and so on, are all priced (typically) in the price offered by each merchant.
Patrick
12/22/08
This is interesting! Thanks for making the effort to provide data. It looks like a lot is missing from the file you made public, though. Why are there only 40 rows? Also, why are some items repeated twice? Do you have the raw prices found by each turker? How many workers did you assign to each item?
Deeper question: What does it say about America that “Harley Parking Only Violators Will Be Crushed” is on a list of popular items?
Sam
12/22/08
On a tangent, what was the error rate that you observed from Mechanical Turk?
Sam
Bob Carpenter
01/02/09
Did you run into problems with prices changing? I know Amazon twiddles their prices all the time (and I suspect they might charge me more as a Prime member).
To really do this solidly, you’d have to order the items to verify they were in stock and actually available at the prices quoted online.
johnrocks
05/04/09
OxyShopping is a shopping search site that enables shoppers to quickly and easily find products from various Indian online stores.
If you leave the cluttered UI, OxyShopping is actually a good meta engine implementation that collects data from various sites.
Products are classified under relevant categories like Mobile, Clothes, Computer etc and buying process happens at the original site.
Though OxyShopping has a long way to go in terms of building a usable UI (for e.g. ability to sort results based on price, de-cluttering etc), OxyShopping is a good stuff and does fill a gap.
johnrocks
Cheap Deals