INCI Ingredient Principles-Based Beauty Advisor: A Practical Guide to Personalized Cosmetic Recommendations Using Ingredient Listing Order
Cosmetic ingredient labeling (INCI) in most countries follows the principle of listing ingredients in descending order of concentration. This single regulation provides a powerful signal. The first 3-5 ingredients essentially define the product's identity, and while the order of ingredients below 1% is flexible, the presence of specific ingredients is clearly indicated. This article leverages this regulatory principle to build a practical, personalized cosmetic recommendation tool using open data and AI techniques. Our example user is a "35-year-old woman, with oily skin, seeking to improve acne and minimize pores, and who is new to retinol but plans to become pregnant." To avoid personal information regulation issues, we will exclude genetic information (SNP) and instead combine ingredient embedding, facial lesion CNN, conflict rules, and LLM summarization.
📚 Recommended Prerequisite Reading (Strongly Recommended)
This article is an advanced and in-depth exploration of AI and biology. We strongly recommend that you review the following articles from DryBench before proceeding.
- DryBench ai-native #8 RAG and Context
- DryBench ai-native #9 Agents and Tool Use
- DryBench ai-native #13 HuggingFace and Commercial APIs
Attempting to proceed without reviewing the prerequisites will make it difficult to follow the practical code examples presented in this article, as we will not re-explain the RAG search, tool orchestration, embedding model loading, and CNN segmentation principles covered in those previous articles.
We Learned This at DryBench
In DryBench ai-native #8, we learned that RAG is an approach that retrieves and incorporates knowledge from external sources (documents, databases) into the context of a search result. In #9, we learned that an agent can autonomously call multiple tools to perform complex tasks. And in #13, we learned that HuggingFace provides vision and chemical embedding models through a standard API.
Beauty is a domain where these three principles combine in a surprisingly natural way. A cosmetic ingredient database can serve as the knowledge source for RAG, an LLM can mediate between user skin/preference and ingredient information in natural language, and chemical fingerprint embeddings can support KNN searches for similar ingredients. This article presents a pipeline for creating a practical tool from this combination, while also addressing the trade-offs required to avoid regulatory and privacy risks while maintaining practicality.
Defining the Hardcore Problem
Practical User Persona
Let's define the pipeline requirements with an example persona.
- Age: 35-year-old woman.
- Skin type: Oily (especially in the T-zone).
- Interests: Acne management, pore reduction, and early anti-aging.
- Usage history: Niacinamide (3 months, effective), BHA (1 month, irritating).
- New product candidate: Retinol serum for beginners.
- Special considerations: Planning pregnancy (retinoid avoidance necessary), salicylic acid allergy.
- Budget: Not concerned with brand; focuses on ingredients and texture.
Even with this single persona, there are many challenges that the pipeline needs to address.
- List potential retinol products (at least one of retinol, retinal, or retinyl palmitate in the top 5 INCI ingredients).
- Because of pregnancy plans, retinoids should be avoided; recommend alternatives (e.g., bakuchiol or other plant-based retinol alternatives).
- Explicitly state the salicylic acid allergy and immediately filter out products containing that ingredient.
- The user is already using niacinamide; prioritize products that have synergistic effects with niacinamide.
- For oily skin and acne management, avoid ingredients that could be comedogenic, such as cholesterol, myristyl alcohol, and coconut oil.
Information Structure Created by Regulatory Principles
Key principles of cosmetic INCI labeling:
- EU (Regulation EC 1223/2009): Ingredients are listed in descending order of concentration, with ingredients below 1% listed in random order. 26 allergenic ingredients are marked with an asterisk [1].
- US (21 CFR 701): Ingredients are listed in descending order of concentration, with ingredients below 1% listed in random order. Color additives are listed separately [2].
- KR (Cosmetics Act, MFDS): Ingredients are listed in descending order of concentration, with ingredients below 1% listed in random order. The 26 allergenic ingredients are listed according to EU standards (revised in 2020) [3].
- JP (Quasi-drugs and Cosmetics Labeling Standards): All ingredients are listed, in descending order of concentration [4].
Key takeaway: The order of ingredients is information itself. In particular, the top 5 ingredients essentially define the product's identity. This principle allows for the following:
- Identification of key active ingredients: Determine whether active ingredients such as hyaluronic acid or niacinamide are present in the top ingredients.
- Distinguishing between fillers and active ingredients: If the top ingredients are only water, glycerin, and butylene glycol, the product likely contains a high proportion of fillers.
- Detecting conflicting combinations: For example, the combination of retinoids and AHA/BHA (which can cause irritation) or vitamin C and retinol (which can reduce stability).
Practical Scope of This Article
- Collect data on 50,000 to 100,000 cosmetics from open databases (Open Beauty Facts).
- Create chemical fingerprint embeddings (RDKit ECFP) for each INCI ingredient.
- Allow users to input their skin profile, preferred ingredients, and ingredients to avoid.
- (Optional) Use facial images and CNNs to segment lesions and match them to relevant ingredients for specific areas of the face.
- Use a rule engine to warn users about conflicting ingredients, allergies, and ingredients to avoid during pregnancy.
- Have Claude LLM provide a natural language explanation for the top 3 recommended products.
Regulatory and privacy principles (adherence):
- Do not use SNP or genotype genetic information (to avoid the KR Personal Information Protection Act and EU GDPR Art. 9 regarding sensitive information).
- Do not promote or denigrate specific brands. Focus only on ingredients, textures, and open-label information.
- Process facial images locally to avoid transmitting them to external APIs. Obtain explicit user consent.
- State that the tool does not provide medical advice and encourage users to consult with a pharmacist, doctor, or dermatologist.
- Allow users to freely input information about pregnancy, allergies, and specific medical conditions, without profiling them based on this information.
Tools, Stack, and Infrastructure Requirements
| Tool | Role | License |
|---|---|---|
| Open Beauty Facts (Open Data) | Cosmetic ingredients, brands, barcodes | ODbL |
| CosIng (EU Commission) | INCI standard dictionary | Public |
| RDKit | Chemical fingerprints (ECFP, MACCS) | BSD-3-Clause |
| HuggingFace transformers | Facial CNN (Segformer, etc.) | Apache 2.0 |
| Claude API | Natural language recommendation summarization | Commercial |
| FAISS, scikit-learn | Similar ingredient KNN, clustering | MIT, BSD |
| FastAPI + Streamlit (Optional) | Service deployment | MIT, Apache 2.0 |
| PubChem REST | CAS to SMILES mapping | Public |
Infrastructure Requirements:
- Small consumer GPU (for facial CNN inference). CPU fallback is possible, but image processing will be 10x slower.
- 8GB RAM or more (for ingredient embedding index).
- Disk: Open Beauty Facts subset (~500MB), CNN model (~100MB).
Estimated learning cost: Based on the Claude API pricing, each user session is estimated to cost 0.15. Open data and CosIng are free.
Real-World Pipeline Implementation
Overall Flow:
Step 1. Open Beauty Facts Crawl · INCI Parsing
Open Beauty Facts is a community-driven, open data source [6].
from dataclasses import dataclass, fieldimport gzipimport jsonfrom pathlib import Pathimport re
import requests
OBF_DUMP_URL = "https://static.openbeautyfacts.org/data/openbeautyfacts-products.jsonl.gz"
@dataclassclass Product: barcode: str brand: str name: str ingredients_raw: str inci_list: list[str] # Ingredient list sorted by concentration categories: list[str] = field(default_factory=list) country: str = "" image_url: str = ""
def download_obf_dump(dest: Path) -> Path: """Download Open Beauty Facts JSON dump (approximately 500MB gzipped).""" dest.parent.mkdir(parents=True, exist_ok=True) if dest.exists() and dest.stat().st_size > 100_000_000: return dest print(f"Downloading... {OBF_DUMP_URL}") with requests.get(OBF_DUMP_URL, stream=True, timeout=600) as r: r.raise_for_status() with open(dest, "wb") as f: for chunk in r.iter_content(chunk_size=8192 * 1024): f.write(chunk) return dest
def parse_inci(ingredients_raw: str) -> list[str]: """Parse INCI ingredient raw text into a concentration-sorted list.
- Separates by comma, semicolon, or period. - Parentheses contain aliases or CI numbers (e.g., "AQUA (WATER)"'s "WATER"). - Removes brackets (e.g., for nano labeling). - Preserves asterisk tags for allergenic ingredients. """ if not ingredients_raw: return [] # Normalize and handle brackets parts = re.split(r"[,;·]", ingredients_raw) cleaned = [] for p in parts: # Remove brackets (e.g., "[nano]") p = re.sub(r"\[.*?\]", "", p) # Replace parenthesized aliases with separate tags (not removing entirely here) p = re.sub(r"\(.*?\)", "", p) p = p.strip().upper() # Determine whether to preserve special tags like allergy asterisks (**), tildes (~), etc. if p and len(p) > 1 and len(p) < 100: cleaned.append(p) return cleaned
def iter_cosmetics(dump_path: Path, min_ingredients: int = 3): """Stream Open Beauty Facts dump and filter by minimum number of ingredients.""" with gzip.open(dump_path, "rt", encoding="utf-8") as f: for line in f: try: d = json.loads(line) except json.JSONDecodeError: continue if not d.get("ingredients_text"): continue inci = parse_inci(d.get("ingredients_text", "")) if len(inci) < min_ingredients: continue yield Product( barcode=d.get("code", ""), brand=(d.get("brands") or "").split(",")[0].strip(), name=d.get("product_name", ""), ingredients_raw=d.get("ingredients_text", ""), inci_list=inci, categories=d.get("categories_tags", []), country=d.get("countries", ""), image_url=d.get("image_url", ""), )Step 2. CosIng Matching · Ingredient Normalization
INCI names have a lot of variation in notation. Use CosIng as a dictionary for normalization [7]. Partial matching fallback is essential.
import csv
@dataclassclass CosingEntry: inci_name: str cas: str einecs: str function: str # e.g., "Skin conditioning · Emollient" restriction: str # EU regulation (e.g., "Concentration limit 0.5%") allergen_flag: bool = False
def load_cosing_dict(cosing_csv: Path) -> dict[str, CosingEntry]: """Load CosIng CSV into a dictionary: {INCI_name(uppercased): CosingEntry}.""" result: dict[str, CosingEntry] = {} with open(cosing_csv, encoding="utf-8", errors="ignore") as f: reader = csv.DictReader(f) for row in reader: key = (row.get("INCI name") or "").upper().strip() if not key: continue result[key] = CosingEntry( inci_name=key, cas=row.get("CAS #", "").strip(), einecs=row.get("EINECS/ELINCS #", "").strip(), function=row.get("Function", "").strip(), restriction=row.get("Restriction", "").strip(), allergen_flag="allergen" in row.get("Restriction", "").lower(), ) return result
def normalize_inci(raw_name: str, cosing_dict: dict[str, CosingEntry]) -> str | None: """Normalize using the CosIng dictionary. Partial matching fallback.""" upper = raw_name.upper().strip() if upper in cosing_dict: return upper # Partial matching (e.g., "AQUA (WATER)" → "AQUA") for key in cosing_dict: if key in upper or upper in key: return key # Alternative: Fuzzy matching (Levenshtein distance) return NoneStep 3. Ingredient Chemical Fingerprint Embedding
Most INCI ingredients can have their chemical structure looked up by CAS number. Calculate fingerprints using RDKit.
from rdkit import Chem, RDLoggerfrom rdkit.Chem import AllChemimport numpy as np
RDLogger.DisableLog("rdApp.*")
def cas_to_smiles(cas_number: str, cache: dict | None = None) -> str | None: """CAS → SMILES. PubChem REST API [8]. It is recommended to use a cache to avoid repeated calls.""" if not cas_number: return None if cache and cas_number in cache: return cache[cas_number] url = f"https://pubchem.ncbi.nlm.nih.gov/rest/pug/compound/name/{cas_number}/property/CanonicalSMILES/JSON" try: resp = requests.get(url, timeout=15) if resp.status_code != 200: if cache is not None: cache[cas_number] = None return None data = resp.json() smiles = data["PropertyTable"]["Properties"][0]["CanonicalSMILES"] if cache is not None: cache[cas_number] = smiles return smiles except (requests.RequestException, KeyError): if cache is not None: cache[cas_number] = None return None
def compute_ecfp(smiles: str, radius: int = 2, n_bits: int = 2048) -> np.ndarray: """Calculate ECFP (Extended Connectivity Fingerprint). Morgan Fingerprint.""" mol = Chem.MolFromSmiles(smiles) if mol is None: return np.zeros(n_bits, dtype=np.uint8) fp = AllChem.GetMorganFingerprintAsBitVect(mol, radius, nBits=n_bits) return np.array(fp, dtype=np.uint8)
def build_ingredient_embeddings( cosing_dict: dict[str, CosingEntry], cache_path: Path, sleep_between: float = 0.2, # Respect PubChem (less than 5 requests per second)) -> dict[str, np.ndarray]: """Create a dictionary of INCI ingredient to ECFP embeddings. Use the PubChem cache.""" import time if cache_path.exists(): return np.load(cache_path, allow_pickle=True).item()
embeddings: dict[str, np.ndarray] = {} smiles_cache: dict[str, str | None] = {} for i, (inci_name, info) in enumerate(cosing_dict.items()): if i % 100 == 0: print(f"[{i}/{len(cosing_dict)}] Calculating fingerprints") smiles = cas_to_smiles(info.cas, cache=smiles_cache) if smiles: embeddings[inci_name] = compute_ecfp(smiles) time.sleep(sleep_between) # Respect PubChem rate limit np.save(cache_path, embeddings, allow_pickle=True) return embeddingsStep 4. Conflict, Regulation, and Allergy Rule Engine
Integrate known combinations of incompatible ingredients in dermatology and cosmetic chemistry, the 26 EU allergenic substances, and ingredients to avoid for pregnant women.
from typing import Literal, NamedTuple
Severity = Literal["info", "warn", "avoid"]
class Rule(NamedTuple): kind: str # "conflict" · "allergen" · "pregnancy" · "user_avoid" ingredient_a: str ingredient_b: str | None # None if it is a single-ingredient rule severity: Severity reason: str reference: str context: dict[str, str] = {}
CONFLICT_RULES: list[Rule] = [ Rule( kind="conflict", ingredient_a="RETINOL", ingredient_b="GLYCOLIC ACID", severity="warn", reason="The combination of retinoids and AHA can cause irritation and redness. It is recommended to use them separately, at night and in the morning.", reference="AAD (American Academy of Dermatology) retinoid guidelines", ), Rule( kind="conflict", ingredient_a="RETINOL", ingredient_b="SALICYLIC ACID", severity="warn", reason="The combination of retinoids and BHA can cause excessive irritation and exfoliation.", reference="Journal of the American Academy of Dermatology", ), Rule( kind="conflict", ingredient_a="RETINOL", ingredient_b="ASCORBIC ACID", severity="warn", reason="Retinol and pure vitamin C have conflicting pH requirements (retinol 5.5, vit C 3.5). This can affect stability and cause irritation.", reference="Skin Therapy Letter, vitamin C stability review", ), Rule( kind="conflict", ingredient_a="BENZOYL PEROXIDE", ingredient_b="RETINOL", severity="avoid", reason="BPO oxidizes and degrades retinol, reducing its effectiveness and irritating the skin.", reference="Journal of Drugs in Dermatology", ), Rule( kind="conflict", ingredient_a="NIACINAMIDE", ingredient_b="ASCORBIC ACID", severity="info", reason="There is a theory that high concentrations of the two together create nicotinic acid. Recent research generally shows that this is not a problem (controversial).", reference="Cosmetic Dermatology reviews · 2024 update", ),]
PREGNANCY_AVOID: list[Rule] = [ Rule( kind="pregnancy", ingredient_a=name, ingredient_b=None, severity="avoid", reason=f"{name} is recommended to be avoided during pregnancy (based on evidence of potential birth defects or as a precautionary measure).", reference="ACOG (American College of Obstetricians and Gynecologists) guidelines", ) for name in [ "RETINOL", "RETINALDEHYDE", "RETINYL PALMITATE", "TRETINOIN", "ADAPALENE", "TAZAROTENE", # retinoid family "SALICYLIC ACID", # high concentration BHA "HYDROQUINONE", # whitening ]]
EU_ALLERGENS_26 = [ "LIMONENE", "LINALOOL", "CITRAL", "GERANIOL", "EUGENOL", "COUMARIN", "CITRONELLOL", "FARNESOL", "BENZYL ALCOHOL", "BENZYL BENZOATE", "BENZYL SALICYLATE", "BENZYL CINNAMATE", "CINNAMAL", "CINNAMYL ALCOHOL", "AMYLCINNAMAL", "AMYLCINNAMYL ALCOHOL", "HEXYL CINNAMAL", "HYDROXYCITRONELLAL", "HYDROXYISOHEXYL 3-CYCLOHEXENE CARBOXALDEHYDE", "ISOEUGENOL", "METHYL 2-OCTYNOATE", "ANISYL ALCOHOL", "ALPHA-ISOMETHYL IONONE", "EVERNIA PRUNASTRI EXTRACT", "EVERNIA FURFURACEA EXTRACT", "3-P-CUMENYL-2-METHYLPROPIONALDEHYDE",]
def check_all_rules( inci_list: list[str], user_profile: dict,) -> list[Rule]: """Return relevant rules based on the product's ingredients and the rule set.""" triggered = [] upper_set = {name.upper() for name in inci_list}
# 1. Incompatible combinations for rule in CONFLICT_RULES: if rule.ingredient_a in upper_set and (rule.ingredient_b or "") in upper_set: triggered.append(rule)
# 2. Pregnancy planning if user_profile.get("pregnancy_planning", False): for rule in PREGNANCY_AVOID: if rule.ingredient_a in upper_set: triggered.append(rule)
# 3. User allergies and ingredients to avoid for avoid in user_profile.get("allergies", []) + user_profile.get("avoid_ingredients", []): if avoid.upper() in upper_set: triggered.append(Rule( kind="user_avoid", ingredient_a=avoid.upper(), ingredient_b=None, severity="avoid", reason=f"User input: Avoidance/allergy to {avoid}.", reference="user profile", ))
# 4. 26 EU allergenic substances (warning level) for allergen in EU_ALLERGENS_26: if allergen in upper_set: triggered.append(Rule( kind="allergen", ingredient_a=allergen, ingredient_b=None, severity="info", reason=f"Included in the list of 26 EU allergenic substances (does not necessarily mean an allergic reaction).", reference="EU Regulation (EC) 1223/2009 Annex III", ))
return triggeredStep 5. Facial Lesion Segmentation CNN (Optional)
If the user provides a facial image, segment lesions (redness, acne, dry areas).
import torchfrom transformers import SegformerImageProcessor, SegformerForSemanticSegmentation
class FacialLesionSegmenter: """Facial lesion segmentation based on Segformer (conceptual example).
In practice, it is recommended to use a fine-tuned model for skin conditions (e.g., fine-tuned on the ISIC dermatology benchmark). This example uses a baseline trained on natural images. This must be retrained for actual use. """
MODEL_ID = "nvidia/segformer-b0-finetuned-ade-512-512" # Example, in reality, skin-specific
def __init__(self, device: str = "cuda"): self.device = device self.processor = SegformerImageProcessor.from_pretrained(self.MODEL_ID) self.model = SegformerForSemanticSegmentation.from_pretrained(self.MODEL_ID).to(device).eval()
@torch.no_grad() def segment(self, image: np.ndarray) -> dict: """RGB image → lesion mask dictionary.""" inputs = self.processor(images=image, return_tensors="pt").to(self.device) outputs = self.model(**inputs) logits = outputs.logits upsampled = torch.nn.functional.interpolate( logits, size=image.shape[:2], mode="bilinear", align_corners=False ) pred_mask = upsampled.argmax(dim=1).squeeze().cpu().numpy() return { "mask": pred_mask, "class_ratios": { int(cls): float((pred_mask == cls).mean()) for cls in np.unique(pred_mask) }, }
LESION_TO_INGREDIENT_HINTS = { "acne": ["SALICYLIC ACID", "BENZOYL PEROXIDE", "NIACINAMIDE", "ZINC PCA"], "erythema": ["CENTELLA ASIATICA EXTRACT", "MADECASSOSIDE", "PANTHENOL"], "dryness": ["HYALURONIC ACID", "GLYCERIN", "CERAMIDE NP", "SQUALANE"], "hyperpigmentation": ["NIACINAMIDE", "ALPHA-ARBUTIN", "TRANEXAMIC ACID", "AZELAIC ACID"],}Important Disclaimer (explicitly stated in the main text): The CNN model is a baseline trained on natural images. For actual use, a fine-tuned model specific to dermatology is essential. Also, it is not a diagnostic tool (to avoid regulation as a medical device, this must be stated to the user).
Step 6. Claude LLM Natural Language Recommendation
import anthropic
RECOMMEND_PROMPT = """You are an AI assistant specializing in cosmetic ingredients and regulations.Review the user profile and candidate products, and recommend the top 3 products in natural language.
User Profile:- Skin type: {skin_type}- Preferred ingredients: {preferred}- Ingredients to avoid/allergies: {avoid}- Pregnancy planning: {pregnancy}- Usage history: {history}
Candidate products (top 5 INCI ingredients):{products_formatted}
Detected rules (conflicts, allergies, pregnancy avoidance):{rules_formatted}
Principles:1. Do not promote or denigrate specific brands. Base recommendations only on ingredients, properties, and rules.2. Each recommendation should include a short explanation (3-5 sentences): What top ingredients match the user's goals, and how conflicts were handled.3. Always include the following at the bottom: "This information is for reference only and is not medical advice. Consult a dermatologist for skin problems."4. If the user is planning a pregnancy, and a product containing retinoids is among the candidates, be sure to advise against it and suggest alternatives (e.g., bakuchiol).
Response format: Markdown."""
def format_products(products: list[dict]) -> str: lines = [] for i, p in enumerate(products, 1): top5 = ", ".join(p["inci_list"][:5]) lines.append(f"{i}. {p['brand']} / {p['name']}\n Top 5 ingredients: {top5}") return "\n".join(lines)
def format_rules(rules: list[Rule]) -> str: if not rules: return "No rules detected." lines = [] for r in rules: if r.ingredient_b: pair = f"{r.ingredient_a} × {r.ingredient_b}" else: pair = r.ingredient_a lines.append(f"- [{r.severity}] {pair}: {r.reason} (Source: {r.reference})") return "\n".join(lines)
def generate_recommendation( user_profile: dict, top_products: list[dict], all_rules: list[Rule], model: str = "claude-sonnet-4-5",) -> str: """Claude generates a natural language summary of the recommendations.""" client = anthropic.Anthropic() prompt = RECOMMEND_PROMPT.format( skin_type=user_profile.get("skin_type", ""), preferred=", ".join(user_profile.get("preferred", [])), avoid=", ".join(user_profile.get("avoid_ingredients", []) + user_profile.get("allergies", [])), pregnancy="Yes" if user_profile.get("pregnancy_planning", False) else "No", history="; ".join(user_profile.get("history", [])), products_formatted=format_products(top_products), rules_formatted=format_rules(all_rules), ) resp = client.messages.create( model=model, max_tokens=2048, messages=[{"role": "user", "content": prompt}], ) return resp.content[0].textStep 7. Integrated Pipeline · Persona Execution Example
def full_advisor_pipeline( user_profile: dict, face_image: np.ndarray | None, product_db: list[Product], ingredient_embeddings: dict[str, np.ndarray], top_k: int = 10,) -> dict: """User profile + facial image → recommended products + natural language explanation + rule report.""" # 1. Average embedding of user's preferred ingredients preferred = [p.upper() for p in user_profile.get("preferred", [])] pref_embs = [ingredient_embeddings[p] for p in preferred if p in ingredient_embeddings] user_emb = np.mean(pref_embs, axis=0) if pref_embs else None
# 2. Combine ingredients to avoid, allergies, and pregnancy avoidance hard_avoid = {a.upper() for a in user_profile.get("avoid_ingredients", []) + user_profile.get("allergies", [])} if user_profile.get("pregnancy_planning", False): for rule in PREGNANCY_AVOID: hard_avoid.add(rule.ingredient_a)
# 3. Score candidates scored = [] for prod in product_db: top5 = {i.upper() for i in prod.inci_list[:10]} # Exclude products that contain ingredients in hard_avoid if hard_avoid & top5: continue top5_embs = [ ingredient_embeddings[i] for i in prod.inci_list[:5] if i in ingredient_embeddings ] if not top5_embs: continue prod_emb = np.mean(top5_embs, axis=0) if user_emb is not None: similarity = float(np.dot(prod_emb, user_emb) / ( np.linalg.norm(prod_emb) * np.linalg.norm(user_emb) + 1e-8 )) else: similarity = 0.5 scored.append({ "brand": prod.brand, "name": prod.name, "inci_list": prod.inci_list, "score": similarity, "categories": prod.categories, "image_url": prod.image_url, })
# 4. Sort the top K scored.sort(key=lambda x: -x["score"]) top_products = scored[:top_k]
# 5. Check rules all_rules = [] for prod in top_products: all_rules.extend(check_all_rules(prod["inci_list"], user_profile))
# 6. Facial lesion analysis (if available) lesion_info = None if face_image is not None: segmenter = FacialLesionSegmenter() lesion_info = segmenter.segment(face_image)
# 7. Claude natural language recommendation recommendation = generate_recommendation(user_profile, top_products, all_rules)
return { "recommended_products": top_products, "triggered_rules": [r._asdict() for r in all_rules], "lesion_analysis": lesion_info, "recommendation_text": recommendation, "disclaimer": "This information is for reference only and is not medical advice. Consult a dermatologist for skin problems.", }
# Execution example (persona scenario)# user = {# "skin_type": "oily · acne-prone",# "preferred": ["NIACINAMIDE", "BAKUCHIOL", "ZINC PCA"],# "avoid_ingredients": ["ALCOHOL DENAT"],# "allergies": ["SALICYLIC ACID"],# "pregnancy_planning": True,# "history": ["Niacinamide worked for 3 months", "BHA caused irritation after 1 month"],# }# result = full_advisor_pipeline(user, face_image=None, product_db=[...], ingredient_embeddings={...})# print(result["recommendation_text"])
## Performance, Cost, and Known Failure Cases
### Performance Reference (Based on Public Benchmarks)
This section provides a complete benchmark, though due to the open data nature, a gold standard is unavailable. The following table shows performance metrics for each component:
| Component | Benchmark | Metric | Source ||----------|------|------|------|| ECFP fingerprint | ChEMBL similarity | Chemical similarity correlation r ≈ 0.75 | Rogers & Hahn 2010 [9] || Segformer (natural image) | ADE20K | mIoU 0.65 | Xie et al., NeurIPS 2021 [10] || Segformer (skin fine-tuned) | ISIC Benchmark | Dice 0.80~0.85 | Community fine-tuning benchmark [11] || Claude summarization | No benchmark | Requires user preference A/B testing | — || Open Beauty Facts data consistency | Self-validation | INCI notation error rate ~15% | OBF community report [6] |
### Estimated Cost for Replicating the System
- Claude API: Approximately 5-15 cents per user session.- CAS → SMILES PubChem API: Free (requires waiting between requests, limited to 5 requests per second).- Facial CNN inference: 100-500ms per image on a local GPU.- Hosting the entire system: Small VPS, $5-20 USD per month.
### Five Known Failure Cases (Collected from the Community and Papers)
1. **CosIng Matching Failure due to INCI Notation Variations and Typos** Symptoms: The INCI on the product label is slightly different from the CosIng standard name ("AQUA" vs "AQUA (WATER)" vs "WATER"). Cause: Differences in notation practices between brands, multilingual labeling, and typos. Mitigation: (a) Partial matching fallback (Step 2 in this document), (b) Fuzzy matching (Levenshtein distance, `rapidfuzz` library), (c) Community alternative name dictionary (using the OBF INCI dictionary), (d) LLM normalization (querying Claude: "What INCI ingredient does this notation refer to?"). Source: Open Beauty Facts GitHub — INCI dictionary issue [6].
2. **Missing CAS → SMILES Mapping (Especially for Natural Ingredients)** Symptoms: Plant extracts, fermentation products, and premium essential oils may not have a single CAS number or SMILES in PubChem. Cause: Natural ingredients are often mixtures of multiple compounds. A single SMILES is not appropriate. Mitigation: (a) Separate natural ingredients into a separate category, (b) Replace MACCS keys with text embeddings (embedding ingredient descriptions using an LLM), (c) Fingerprint only the representative active ingredient (e.g., Centella Asiatica → madecassoside). Source: PubChem "natural products" documentation [8].
3. **Overly Cautious Conflict Rules vs. Ignoring Individual Differences** Symptoms: The system automatically displays a retinol + niacinamide conflict warning, but recent research suggests that this combination is not actually problematic. This causes user confusion. Cause: Cosmetic chemistry research is constantly updated, and there are differing opinions within the dermatology community. Mitigation: (a) Display the reliability of the rule ("certain" / "controversial"), (b) Provide a URL for the reference, (c) Allow users to dismiss the rule via UI, (d) Regularly update the rule set (e.g., quarterly review), (e) Set the severity level of "info" to be collapsed by default in the UI. Source: Journal of Cosmetic Dermatology, reviews on the niacinamide-vitamin C debate [12].
4. **Domain Shift in Facial CNN (Lighting, Ethnicity, Angle)** Symptoms: A CNN trained on natural images performs poorly on user selfies (with varying smartphone lighting and angles). Cause: Insufficient diversity in lighting and demographics in the training data. Dataset bias towards certain ethnicities (e.g., ISIC is biased towards white skin). Mitigation: (a) Fine-tune with diverse ethnic and lighting data, (b) Provide a UI that guides users on lighting and angles, (c) Display confidence levels in the results, (d) Prioritize other aspects (core logic focuses on ingredient embeddings and rule engine). Source: Wen et al. "Characteristics of publicly available skin cancer image datasets: a systematic review." Lancet Digital Health 2022 [13].
5. **Personal Data Protection Regulations (KR Personal Information Protection Act, GDPR, CCPA)** Symptoms: Storing facial images, allergy profiles, or pregnancy status may violate regulations. Cause: Facial images are considered biometric data under the KR Personal Information Protection Act and special categories under GDPR Art. 9. Pregnancy status is considered health information (GDPR special category). Mitigation: (a) Process facial images only locally in the browser (no server transmission), (b) Store user profiles only within the session, encrypt data if stored in a database, and obtain explicit consent, (c) Clearly state that profiling and advertising will not be used, (d) Provide mandatory UI elements for privacy policies and consent forms, (e) Restrict use for children under 14 years of age. Source: Personal Information Protection Commission guidelines for cosmetics and beauty apps; GDPR Art. 9 and Art. 22.
## Expansion Ideas
- **Barcode Scanning App Integration:** Users scan barcodes in-store to instantly access ingredient information, matching, and warnings.- **Multilingual Label OCR:** Capture images of foreign cosmetic labels, use Tesseract OCR, and automatically parse the INCI.- **Allergy Profile Expansion:** Allow users to register allergy triggers and instantly filter products containing those substances, and also warn about similar ingredients (based on fingerprint similarity).- **Environmental Sustainability Filter:** Tag products based on microplastics, palm oil, and animal testing.- **Price-to-Ingredient Density Score:** Correlate the placement of active ingredients with price.- **Community Review Sentiment Analysis:** Scrape Open Beauty Facts reviews and Reddit r/SkincareAddiction (paying attention to licenses) and use an LLM for sentiment analysis.- **Korean-Specific KFDA Approval Status Display:** Tag functional cosmetics certified for wrinkle reduction, whitening, or UV protection.
## Next Section
- Section 07 `drug-target-gnn`: Quantify ingredient-skin receptor interactions using GNNs.- Section 09 `llm-vendor-benchmark`: Benchmark the natural language recommendations of this section using Claude, GPT, and Gemini.- Section 14 `bio-mcp-agent`: Expose this pipeline as an MCP tool to create a chatbot-style beauty advisor.
## References
1. EU Regulation (EC) No 1223/2009 (Cosmetic Products Regulation): `https://eur-lex.europa.eu/legal-content/EN/TXT/?uri=CELEX:02009R1223`2. FDA Cosmetic Labeling Guide (21 CFR 701): `https://www.fda.gov/cosmetics/cosmetics-labeling`3. Republic of Korea Cosmetics Act (MFDS): `https://www.mfds.go.kr/`4. Japan Quasi-Drugs and Cosmetics Labeling Standards: `https://www.mhlw.go.jp/`5. Anthropic Claude API pricing: `https://www.anthropic.com/pricing`6. Open Beauty Facts: `https://world.openbeautyfacts.org/` / GitHub: `https://github.com/openfoodfacts/openbeautyfacts-server`7. CosIng (EU Commission Cosmetic Ingredients Database): `https://ec.europa.eu/growth/tools-databases/cosing/`8. PubChem REST API documentation: `https://pubchem.ncbi.nlm.nih.gov/docs/pug-rest`9. Rogers D, Hahn M. "Extended-Connectivity Fingerprints." J Chem Inf Model 2010.10. Xie E, Wang W, Yu Z, et al. "SegFormer: Simple and Efficient Design for Semantic Segmentation with Transformers." NeurIPS 2021. `https://arxiv.org/abs/2105.15203`11. ISIC (International Skin Imaging Collaboration): `https://www.isic-archive.com/`12. Journal of Cosmetic Dermatology (niacinamide+vitamin C debate reviews): Various papers13. Wen D, Khan SM, Xu AJ, et al. "Characteristics of publicly available skin cancer image datasets: a systematic review." Lancet Digital Health 2022. `https://www.thelancet.com/journals/landig/article/PIIS2589-7500(21)00252-1`14. American Academy of Dermatology guidelines: `https://www.aad.org/`15. American College of Obstetricians and Gynecologists (ACOG) guidelines: `https://www.acog.org/`16. GDPR Art.9 (special categories): `https://gdpr-info.eu/art-9-gdpr/`17. RDKit Fingerprints documentation: `https://www.rdkit.org/docs/GettingStartedInPython.html`18. rapidfuzz (fuzzy matching): `https://github.com/rapidfuzz/RapidFuzz`19. INCI Beauty (community reference): `https://incibeauty.com/`20. EWG Skin Deep (reference, methodology is debatable): `https://www.ewg.org/skindeep/`