ULTIMATE Pinecone RAG ChatFlow~

ULTIMATE Pinecone RAG ChatFlow~ Workflow Overview This workflow automates the creation and querying of a sophisticated, multi-domain AI knowledge base. It functions as a dual-purpose system: first, as a data ingestion pipeline that processes information from various sources (files, YouTube, web searches), and second, as an intelligent chat agent that uses this knowledge to answer user questions. The system is designed to handle multiple, distinct categories of information, storing each in a separate, organized namespace within a central vector store. This allows the chat agent to be highly precise, retrieving information only from the relevant knowledge domain for a given query. Technologies & Automation - Automation Core: n8n coordinates all data flows, API calls, and logic. - AI Chat & Reasoning: OpenAI's GPT-4o model is used for conversational AI, text classification, and reasoning. - Vector Storage: Pinecone serves as the vector database, the long-term memory for the AI, enabling high-speed semantic search. - Data Ingestion: - Handles various file types (PDF, DOCX, TXT, XLSX, CSV, JSON, HTML) from user uploads via a webhook or from a monitored Google Drive folder. - Extracts transcripts from YouTube videos. - Performs real-time web searches for news and articles using the Tavily search API. - Data Management: Supabase is used for file storage (via download URLs) and for storing and retrieving dynamic system prompts from a Postgres database. - Conversational Memory: Zep provides persistent, long-term memory for chat sessions, allowing the agent to recall previous interactions and maintain context. This workflow automates the entire lifecycle of a Retrieval-Augmented Generation (RAG) system. It automates the collection, parsing, cleaning, categorizing, and vectorizing of knowledge. It then provides an intelligent, context-aware AI agent that can use this curated knowledge base to provide accurate, source-grounded answers, effectively creating a custom "expert" on demand. AI Integration AI is the cornerstone of this workflow, integrated at multiple critical points: 1.