ORINEL — Autonomous Data Intelligence Platform

Design goal: turn raw enterprise data into automated insights, dashboards, predictions, and reports with minimal or zero manual work — more advanced than Databricks, Snowflake, and Palantir Foundry. Built as a modern enterprise SaaS (Stripe, Linear, Databricks UX).

Last updated: March 2025

Stack

  • Next.js — Frontend and BFF API (auth, CRUD, job triggers).
  • Python AI backend — FastAPI service: dataset ingestion (schema, column classification, anomaly), pipeline execution (cleaning, feature engineering), model engine (regression, classification, forecasting, anomaly), insights (stats, trends, correlations), report narrative, AI Copilot.
  • Supabase — PostgreSQL database, auth, object storage.
  • DuckDB — In-process analytics engine inside Python for fast SQL on ingested data and Copilot queries.

Six pillars

  • Dataset ingestion engine — CSV/Excel upload, schema inference, column classification, anomaly detection on ingest.
  • AI pipeline builder — Automatic data cleaning, feature engineering, pipeline orchestration.
  • AI model engine — Regression, classification, forecasting, anomaly detection; train and register models.
  • AI insights generator — Statistical summaries, trend detection, correlation analysis.
  • AI report generator — Executive summary, charts, recommendations, PDF/Excel/PPT export.
  • AI Copilot — Natural language queries, dataset exploration, insight explanations.

Core capabilities (current codebase)

  • Automatic schema detection — Infer types, constraints, and quality from raw data (CSV, Excel, DB, API).
  • Automatic data pipelines — DAG execution, scheduling, cleaning, and load; Redis/BullMQ workers.
  • Automated exploratory analysis — EDA (distributions, correlations, outliers) triggered on ingest or pipeline success.
  • ML model recommendation — Suggest algorithms and config from dataset profile and use case.
  • Autonomous insight generation — AI narratives, executive summaries, and recommendations from data + EDA + models.
  • Automatic dashboard generation — Create or refresh dashboards from EDA chart configs and KPIs.
  • AI Copilot — Natural language queries → intent + dataset resolution (vector) → analysis + charts + explanation + recommendations.

Architecture documents

Two design documents define the platform:

  • docs/ORINEL-DATA-INTELLIGENCE-ARCHITECTURE.md— Principal architecture: Next.js + Python AI backend + Supabase + DuckDB; six pillars; API contract; DuckDB integration; security and UX.
  • docs/ORINEL-AUTONOMOUS-PLATFORM.md— Full autonomy design: nine engines, data flows, Node/PostgreSQL/Redis mapping, roadmap.

Python AI backend skeleton: services/ai-backend/ (FastAPI, DuckDB helpers, API contract in README).

← Architecture overview