Academy›Special courses: eight courses for day-to-day selling›AI for analysis and customer service: reviews, returns, reports
Special courses: eight courses for day-to-day sellingAI in everyday selling

AI for analysis and customer service: reviews, returns, reports

Course lesson 3/3 ⏱ ~14 Min. 🏅 up to 205 points By Enes Kurt Updated November 2026 Check your listing free →
What you'll take away

A language model reads 400 reviews in the time it takes you to fetch a coffee, and writes a polite reply to an angry message before you've had time to get annoyed yourself. That makes AI a strong tool for analysis and customer service, but one with three typical weaknesses: it summarizes convincingly even when it has miscounted. It calculates unreliably. And everything you paste into it leaves your business. This lesson shows how to use AI for reviews, return reasons, customer inquiries and ad reports without walking into these traps.

1

Where AI helps and where you stay in charge

TaskAI is good forStays with you
Reviews and returnsSpotting topics, sorting comments, finding fitting quotesCounting, finding the cause, deciding on action
Customer serviceDraft in the customer's language, tone, translationOrder facts, goodwill, approval before sending
Search term and PPC reportsGrouping terms by intent, describing patternsCalculating metrics, setting bids and negative keywords
Market and competitor monitoringSummarizing long texts, drawing out differencesChecking sources, drawing conclusions

The pattern is always the same: the AI sorts, describes and drafts. Calculating, deciding and approving stay with you. Keep to this division of labor and you save most of the reading time without basing decisions on a summary nobody has checked.

2

Analyzing reviews and return reasons

How to read competitors' reviews for product development is covered in niche validation, how your own reviews become an early-warning system in review management. AI makes both faster if you give it a fixed scheme. These sources are suitable:

  • Your own reviews: from the product page or from the review overview that brand owners see in Seller Central.
  • Return reasons and customer comments: the returns report in Seller Central gives a reason per return and often a free-text comment from the customer. The Voice of the Customer dashboard bundles feedback per offer.
  • Competitors' reviews: selected deliberately and copied by hand. Amazon's conditions of use prohibit data mining tools and robots, so no bulk retrieval by script.
  • Customer questions and messages: from the Q&A section and your inbox, without names and order numbers (see the data protection chapter).

The workflow in four steps:

  1. Fix a topic scheme: 8 to 12 topics you define in advance, for example burr, lid and seal, battery and electrics, size, cleaning, contents, look, packaging, operation, other. A fixed scheme makes months comparable.
  2. Have each item assigned: the AI assigns every comment to one or two topics and returns a table, one row per comment, with a short quote as evidence. No summary in running text.
  3. Count in the spreadsheet: you calculate frequencies yourself from the table, not the AI in its text.
  4. Cross-check: check 30 to 40 random rows by hand. If fewer than nine in ten assignments are right, sharpen the topic descriptions and reassign (expert chapter).
Example

Returns analysis for the AURELO set over one quarter (numbers as an assumption): 1,150 orders, 62 returns with a customer comment. The AI assigns, the spreadsheet counts: 14 comments on “lid and seal” (23%), 11 on “battery and electrics” (18%), 9 on “size”. The cross-check on 40 rows yields 36 matches, so 90%. At roughly €4.65 to €9.15 per return (€24.99 selling price, €4.50 purchase cost), the 14 lid returns cost €65.10 to €128.10 per quarter. A better silicone seal at €0.08 extra per unit costs €92 for 1,150 units. Whether that pays off depends on how many lid returns it prevents, and on the 1-star reviews that never get written.

Most common mistake

Believing the summary. “Many customers complain about a loose lid” sounds precise but says nothing about the number. Language models emphasize what stands out in the text, not what is frequent, and occasionally they invent quotes that don't exist. The same goes for Amazon's own AI summary “Customers say” on the product page: it shows what buyers read, but it is not a count. You make decisions about suppliers, batches or product changes based on counted rows and checked original comments, never on a paragraph.

3

Drafting customer service replies

Messages via Buyer-Seller Messaging are governed by Amazon's communication guidelines, and an AI doesn't know them reliably. So the key points belong in every prompt:

  • Permitted are messages needed to complete an order or answer a customer service inquiry. Proactive messages only for set reasons, such as a problem with the order, a question about a return, an invoice or one review request per order, and within 30 days of order completion, with the 17-digit order ID and in the buyer's preferred language.
  • Prohibited are, among other things, marketing and coupons, wording that pushes for positive reviews or offers something for them, asking for a review to be changed or removed, external links (except secure links needed for the order and links to Amazon), email addresses and phone numbers not legally required, emojis and glaring spelling errors.
  • Speed: Amazon expects replies to buyer messages within 24 hours, weekends included. For breaches of the guidelines, Amazon can restrict you to templates or temporarily remove your selling privileges.

Turn this into a fixed reply prompt: role (customer service of a brand owner), the rules above, product facts from the fact sheet (AI for listing copy), your goodwill rules, tone (friendly, concrete, no filler) and goal (solve the problem, name the next step). The AI writes the draft, you check facts, goodwill and rules, and you send it yourself.

Example

A buyer writes that her AURELO grinder stopped grinding after three weeks and she is disappointed. You give the AI only the message text without the name, plus the fact sheet and your goodwill rule (assumption: replacement for defects within the first 12 months). The draft: regret in one sentence, the two most common causes from the fact sheet (batteries inserted wrongly, grind set to the finest level with damp salt), a request for a short reply and the offer of a free replacement if the defect persists. No sentence about reviews, no coupon, no phone number. You check the goodwill commitment, the inbox links the reply to the order, and you send it. The math behind it: a replacement set costs you €4.50 purchase cost plus shipping, a return with goods that can no longer be sold €9.15, and what a 1-star review costs a €24.99 product can't even be cleanly expressed in euros.

Important

If a system answers customer inquiries fully automatically, without a person approving, Article 50(1) of the EU AI Act has required since 2 August 2026 that customers can tell from the start that they are communicating with an AI, unless that is obvious. Amazon's rules still apply to every single message regardless. For complaints, refunds and anything with legal effect, human approval is the better choice anyway.

4

Summarizing search term reports and PPC data

Ad reports quickly run to 2,000 rows. How to read the search term report weekly is shown in PPC optimization, the Brand Analytics reports in reading search query data. AI helps at one specific point here: interpreting words, not calculating numbers.

  • Calculate in the spreadsheet: totals, click-through rate, conversion rate and ACOS per search term go into formulas. Language models miscalculate long number series, skip rows or round silently, and the result still looks plausible.
  • Filter in the spreadsheet: apply your rules first, then give the AI only the filtered list, for example all terms with at least 10 clicks and no sale, or all with ACOS above your break-even.
  • Interpret with the AI: group terms by buying intent (fitting, wrong context, other brand, accessory search), describe patterns, propose negative candidates with reasons, derive new terms from customer language.
  • Decide yourself: you set bids, negative keywords and budgets, looking at the numbers and not at the AI's reasoning.
Example

The AURELO set's search term report for 30 days has 1,240 rows (numbers as an assumption). Break-even ACOS from the unit economics: €7.03 profit before ads ÷ €24.99 ≈ 28%. The spreadsheet filters 86 terms with at least 10 clicks and no sale (1,120 clicks together, €504 at €0.45 per click) and 41 terms with sales but ACOS above 28%. Only these 127 rows go to the AI, with the request to group them by intent. Result: 23 terms look for a wooden grinder, 17 for a manual one, 12 name another brand, the rest fit the product. The first two groups are wrong context and therefore candidates for negative keywords. You still check each term individually, because even a wrong context occasionally brings buyers.

Tip

Have the AI name two or three example terms from the list for each group and check whether they really appear there. Invented example terms are the clearest sign that the grouping isn't based on your data.

5

Data protection: customer data and AI

Names, addresses, order numbers and the content of customer messages are personal data. As soon as you put them into an AI tool, you are processing them under the GDPR. Amazon's rules apply on top: you may use customer data from orders only to fulfill the order and for the related customer service.

DataInto an AI tool?Condition
Review textsyesremove names and profile details
Return reasons and commentsyeswithout order number, name and address
Buyer message for a draft replylimitedmessage text only, personal details removed, tool with a data processing agreement
Names, addresses, phone numbersnonot needed for analysis or drafting
Health informationnospecially protected under Art. 9 GDPR
Ad and search term reportsyesusually contain no customer data
  • Data minimization (Art. 5 GDPR): put in only what the task needs. Nobody needs the customer's name for a topic analysis.
  • Processing on your behalf (Art. 28 GDPR): if a provider processes personal data for you, you need a data processing agreement with them. Business plans often offer one, free personal accounts usually don't.
  • Training and storage: check whether inputs are used for training and how long they are stored, and choose the setting that rules this out.
  • Third countries: if processing happens outside the EU, the transfer needs a legal basis, such as the EU-US Data Privacy Framework or standard contractual clauses.
  • Documentation: add the tool to your record of processing activities and update your privacy notice.
In plain terms

An AI tool is like an outside typing pool. You wouldn't hand over the card index with all your customers' addresses just so someone can sort three complaint letters. You hand over the letters, black out names and addresses, and have a contract that governs what the pool may do with the documents.

Germany's joint conference of data protection authorities (DSK) published guidance on “artificial intelligence and data protection” in May 2024 that helps with choosing and using such tools. For your specific case, your data protection officer or a lawyer specializing in IT law is the right contact when in doubt.

6

Limits and checks

Five weaknesses of language models hit analysis and service especially hard:

  • Invented quotes: a summary can contain customer voices that don't exist in that form. Always have quotes output with a row number and check them in the original.
  • Weighting by salience: one long, angry comment shapes the summary more than ten short, factual ones. Counting beats reading.
  • Silent omission: with very long inputs, models don't always process everything. Feed large volumes in batches and check that as many rows come back as went in.
  • Outdated rule knowledge: what the model knows about Amazon policies may be years old. You put the rules into the prompt, you don't ask for them.
  • Confident tone: a model writes “clearly” and “unmistakably” even when the data is thin. How certain a statement is lies in the number behind it, not in the wording.
Checks before every decision
  • Number of rows in equals number of rows out.
  • Frequencies counted in the spreadsheet, not taken from the AI's text.
  • Sample of 30 to 40 assignments checked by hand, hit rate noted.
  • Every quoted customer word found in the original.
  • No names, addresses, order numbers or health details ended up in the tool.
  • Draft replies checked against the communication guidelines and sent by a person.
  • Everyone on the team using AI has been briefed: the EU AI Act requires measures that support staff's AI literacy (Article 4).
7

Expert insight: sample, cross-check and margin of error

An AI analysis is a measurement, and every measurement has two sources of error: the assignment (did the AI sort correctly?) and the volume (are there enough cases to say anything?). You can put a number on both with simple means.

Assignment quality: draw 40 random rows from the finished table (sort by a random number) and assign them yourself without looking at the AI column. The agreement is your hit rate. If it is below 90%, the topic descriptions are almost always fuzzy: “quality” is not a topic, “burr goes dull” is. Sharpen, reassign, check again. Note the rate each month, and you will notice when a new model or a changed prompt sorts worse.

Margin of error: a share based on few cases is uncertain. The rule-of-thumb formula for the 95% interval of a share p from n cases is: p plus/minus 1.96 × √(p × (1 - p) ÷ n).

Example

In the AURELO quarter, 14 of 62 return comments concern the lid, so 23%. The math: √(0.23 × 0.77 ÷ 62) ≈ 0.053, times 1.96 gives about 0.10. The true share therefore lies somewhere between about 12% and 33%. That does not support the statement “the lid is our biggest return issue”: the second topic, at 11 cases (18%), sits in the same range, so both could be equally large. A second look is what decides it: if the 14 cases cluster after a new batch arrived, it's a batch problem, and you settle that with the supplier, not with statistics.

The right base: 23% of return comments is not 23% of customers. At 1,150 orders, 14 lid returns are 1.2% of all orders. Both numbers are correct, they answer different questions: the first, where to start; the second, how big the problem is for your business.

Comparability: compare months only with the same scheme and the same prompt. If the scheme or prompt changes at the same time as the numbers, you can't tell whether the product changed or just the measurement.

Continue reading for free

The first lessons of every track are open to everyone. From here on you just need a free account: no subscription, no costs.

Still ahead in this lesson:
  • 2Analyzing reviews and return reasons
  • 3Drafting customer service replies
  • 4Summarizing search term reports and PPC data
  • 5Data protection: customer data and AI
  • 6Limits and checks
  • 7Expert insight: sample, cross-check and margin of error
  • ✓Quiz: 6 questions with instant feedback
✎ Practice

Your turn: 2 exercises

Solve them to unlock the quiz. Up to 30 points each when you get it right first time without a hint.

1
Put in order

Put the AI returns analysis in the right order

You want to analyze last quarter's return comments for the AURELO spice grinder set with a language model. Put the steps in the right order.

  1. Count frequencies in the spreadsheet and only then decide on an action
  2. Assign 40 random rows yourself and compare your hit rate with the AI
  3. Have the AI assign every comment to one or two topics, as a table with a short quote
  4. Pull the returns report and remove names, order numbers and addresses
  5. Fix a scheme of 8 to 12 topics
  6. Check that as many rows come back as went in
2
Pick all that apply

How do you spot an unreliable AI analysis?

Your colleague shows you the AI analysis of 300 return comments. Which findings show that you cannot trust the result yet? Select all that apply.

Check yourself

6 quick questions, one at a time, with instant feedback. With a free account your progress is saved.

You want to analyze 300 comments from the returns report with AI. What do you do first?
Data minimization: the comment texts are enough for a topic analysis. Personal details don't belong in the tool, and Amazon permits customer data only for fulfillment and service anyway.
In a PPC analysis, which task is better done by the spreadsheet than the language model?
Language models miscalculate long number series and round silently. You calculate metrics with formulas, then the AI interprets the filtered list.
What must a reply via Buyer-Seller Messaging NOT contain?
Incentives for reviews and coupons are prohibited. The order ID, product instructions as an attachment and links to Amazon are allowed under the communication guidelines.
An AI summary reports: “Many customers complain about a loose lid.” What is your next step?
“Many” is not a number. Language models weight by salience and occasionally invent quotes. Only counted rows and checked originals are a basis for decisions.
A buyer writes that he needs the electric grinder because of his arthritis. What applies to this detail?
Under Art. 9 GDPR, health data is a special category. You don't need it for a draft reply, so you remove it beforehand.
Your tool is to answer buyer messages without human approval from now on. What must you consider?
Since 2 August 2026, Article 50(1) of the EU AI Act requires that people can tell when they interact directly with an AI system, unless that is obvious. Amazon's communication rules apply in addition.

Frequently asked

Can I enter customer data into an AI tool?

Only as much as necessary and with safeguards: remove names, addresses and order numbers first, have a data processing agreement with the provider for personal data, and keep health information out entirely. In addition, Amazon permits customer data to be used only for order fulfillment and customer service.

Can AI analyze Amazon reviews reliably?

For sorting by topic, yes, if you set a fixed scheme and check the result against a hand-checked sample. For counting and weighting, no: you calculate frequencies in the spreadsheet and check every quote in the original.

Can I answer Amazon customer messages automatically with AI?

Technically yes, via interfaces and tools. Every message must follow Amazon's communication guidelines, and if a system replies without human approval, customers must be able to tell an AI is writing under Article 50 of the EU AI Act. For complaints, refunds and goodwill, human approval remains the safe choice.

← Previous lessonAI images and video: product accuracy, labelling and rights Next lesson →Amazon Business explained: who buys there and how sellers take part
Enes Kurt
Founder of Listimo

Everything in this academy comes from more than ten years of hands-on work with Amazon listings, the same playbook behind Listimo: the tool that finds what holds an Amazon listing back and fixes it.