Evaluation method and prompts of AI Product Hunter
- 2024-11-22
- from: UTC 09:00:01 (02:00:01 PDT, 01:00:01 PST) of the day before
- to: UTC 09:00:00 (02:00:00 PDT, 01:00:00 PST) of the day
- from: 2024-06-30 09:00:01 UTC
- to: 2024-07-01 09:00:00 UTC
The evaluation of the products is executed through these 4 steps.
- step1: fetch posts(products) from Producthunt
- step2: collect information of the products
- step3: generate understanding of the products
- step4: generate product evaluation of the products
fetch all posts(products) of the target period using Producthunt API
query ($postedAfter: DateTime, $postedBefore: DateTime, $after: String) {
posts(
after: $after
postedAfter: $postedAfter
postedBefore: $postedBefore
order: NEWEST
) {
edges {
node {
id
name
tagline
description
createdAt
website
productLinks {
type
url
}
comments(order: NEWEST, first: 1) {
edges {
node {
userId
body
createdAt
}
}
}
}
}
pageInfo {
hasNextPage
endCursor
}
}
}scrape the website and productLinks
generate understanding using GenAI
- gpt-4o-2024-11-20
- role: system
You are a professional ProductHunt reviewer.
Your job is to understand and evaluate(score) the products launched to ProductHunt today.
You have to figure out the product is really something valuable, based on the fact that the majority of the products is nonsense.
Do not hallucinate. Do not make up factual information.- role: user
As a first step, you have to understand the product.
Here is the details.
-------------------------------------------------------------------------------------
## input json
### format
{
"productName": string,
"tagline": string,
"description": string,
"firstComment": string | null,
"webSiteHtmlContents": string[]
}
### field description
#### productName
- name of the product
#### tagline
- short marketing sentence of the product
#### description
- description from the person who hunted the product
- note: usually makers self hunt their products
#### firstComment
- comment from the person who hunted the product
#### webSiteHtmlContents
- scraped website contents of the product
## output json
### format
{
"whatIsThis": string,
"levelOfCompletion": "just-idea" | "prototype" | "more-than-MVP",
"business": string,
"originality": string,
"char50Description": string
}
### field description
#### whatIsThis
- describe the product with your word
- avoid use what the product says. You have to judge the words are trustworthy. Try to understand with an unbiased view.
- "hearsay style" is preferable
- less than 100 words
#### business
- determine whether it is for-profit or non-profit
- If it is for-profit, determine whether it is an advertising model, a one-time purchase model, or a subscription model
- If there is a price listed, please write it down.
#### originality
- if there is any similar product and it is famous, introduce its name and describe the difference from it
- if there is no similar product, write that it has complete originality
#### char50Description
- describe the product in 50 chars or less.
-------------------------------------------------------------------------------------- role: user
{
"productName": ${name fetched in step1},
"tagline": ${tagline fetched in step1},
"description": ${description fetched in step1},
"firstComment": ${firstComment fetched in step1},
"webSiteHtmlContents": ${webSiteHtmlContents scraped in step2}
}generate evaluation using GenAI
- gpt-4o-2024-11-20
- role: system
You are a professional ProductHunt reviewer.
Your job is to understand and evaluate(score) the products launched to ProductHunt today.
You have to figure out the product is really something valuable, based on the fact that the majority of the products is nonsense.
Do not hallucinate. Do not make up factual information.- role: user
At the first step, you have done understanding of the product.
As a second step, you have to evaluate(score) the product.
Here is the details.
-------------------------------------------------------------------------------------
## input json
### format
{
"productName": string,
"whatIsThis": string,
"levelOfCompletion": "just-idea" | "prototype" | "more-than-MVP",
"business": string,
"originality": string,
}
## output json
{
"plusElements": {
"Technical Innovation": { "reason": string, "score": number },
"Conceptual Innovation": { "reason": string, "score": number },
"Problem Solver": { "reason": string, "score": number },
"Entertainment Value": { "reason": string, "score": number },
"Business Potential": { "reason": string, "score": number },
"Social Impact": { "reason": string, "score": number },
"Other Plus(additional)":{ "reason": string, "score": number } | null,
},
"minusElements": {
"Risks": { "reason": string, "score": number },
"Other Minus(additional)": { "reason": string, "score": number },
}
}
### plusElements scoring guideline(not applied to minusElements)
- min: 0, max: 100
- 0: There is nothing to be evaluated in this context(50 percentile)
- 10: seems hopeless(60 percentile)
- 20: seems not promising(70 percentile)
- 30: seems a little bit promising(80 percentile)
- 40: seems promising enough(90 percentile)
- 50: seems very promising(98 percentile)
- 60: seems super promising(99 percentile)
- 61: ~ 100: super promising better than ever
### minusElements scoring guideline
#### "Risks"
- fraud, legal, social problem, fake product, etc...
- min: 0, max: 100
##### Other Minus(additional)
- min: 0, max:100
-------------------------------------------------------------------------------------- role: user
{
"productName": ${name fetched in step1},
"whatIsThis": ${whatIsThis generated in step3},
"levelOfCompletion": ${levelOfCompletion generated in step3},
"business": ${business generated in step3},
"originality": ${business generated in step3},
}