Overview
An agentic ML pipeline that automates data analysis. It dynamically interprets SQL databases or CSVs, generates its own preprocessing code on the fly and provides actionable business intelligence.
Tech Stack
OpenAI GPT-4oStreamlitPython (Pandas/Scikit-learn)PostgreSQL (Dockerized)SQLAlchemy
Features
- Autonomous Preprocessing: Uses LLMs to deduce dataset context and write custom Pandas cleaning logic.
- Dynamic Clustering: Automated K-Means segmentation to identify distinct data profiles.
- Database Agnostic: Connects to any SQL database via SQLAlchemy.
- AI Business Intelligence: Generates Markdown reports based on Z-score analysis.
Key Decisions
01
Used SQLAlchemy as a universal bridge to ensure the agent could analyze PostgreSQL, MySQL, or SQLite without modification.
02
Implemented a sandboxed Python execution environment using temporary file creation to safely test AI-generated cleaning scripts.
Learnings
→Learned to automate unsupervised learning workflows by letting the agent determine the optimal "K" in K-Means using the Elbow Method.
→Refined the ability to transform abstract Z-score outliers into plain-English business recommendations.