We are seeking a highly skilled Data Analytics Engineer to join our upcoming AI-focused initiatives. The ideal candidate will have deep expertise in Python, Databricks, and Azure cloud services, with a strong background in data engineering and architecture design. This role involves building and optimizing data pipelines, integrating AI functionalities through OpenAI APIs, and leveraging Azure Search, RAG techniques, and Document Intelligence to support next-generation AI solutions. Key Responsibilities: Data Pipeline Development: Design, develop, and maintain robust and scalable data pipelines for ingesting, transforming, and loading data into Azure Search Index Index Optimization: Tune and configure index parameters to optimize Retrieval-Augmented Generation (RAG) workflows and ranking mechanisms for better AI performance Document Intelligence: Leverage Azure's Document Intelligence Services to process various document formats and extract relevant insights for downstream analytics AI Integration: Collaborate with OpenAI APIs and related services to enrich data systems with advanced NLP and AI capabilities Performance Tuning & Monitoring: Monitor data flow, detect anomalies, and improve efficiency through performance tuning and optimization Stakeholder Collaboration: Work autonomously while aligning technical decisions with the broader project goals and collaborating with internal teams to deliver high-quality solutions Requirements Must-Have Skills (Primary): Strong proficiency in Python for data manipulation, automation, and AI integrations Advanced experience with Databricks for scalable data processing and analytics Solid background in Data Engineering with proven experience in: Designing and building data pipelines Implementing scalable and efficient data architectures Experience integrating with Azure Search Index and optimizing for RAG (Retrieval-Augmented Generation) scenarios Hands-on experience with Azure cloud services (e.g., Azure Data Factory, Azure Functions, Azure Blob Storage, Azure SQL, Azure Search)