Data Analyst – Conversational Analytics (Retail Industry)

We are looking for an experienced Data Analyst to support and continuously improve a conversational analytics and AI platform within a large, complex retail organization.

In this role, you will work at the intersection of business, data, and conversational AI, ensuring that analytical insights, data visualizations, and AI-driven responses consistently meet real user needs. You will collaborate closely with business stakeholders and technical teams to test, evaluate, and optimize performance through structured experimentation and data-driven iteration.

Key Responsibilities

• Design and execute systematic testing of business questions and use cases within a conversational AI environment
• Develop, refine, and optimize prompts to improve response quality, accuracy, and relevance of data visualizations
• Conduct A/B testing and comparative analysis of different prompt and interaction strategies
• Identify edge cases, failure modes, and improvement opportunities through rigorous testing
• Define and maintain metrics and KPIs to evaluate conversational AI performance, including accuracy, relevance, user satisfaction, and task completion
• Perform regular performance reviews and regression testing to ensure sustained quality over time
• Analyze performance trends and patterns to proactively identify optimization opportunities
• Gather requirements from business users across functions and translate needs into actionable conversational AI features
• Communicate insights, findings, and recommendations clearly to both technical and non-technical stakeholders
• Document prompt engineering guidelines, testing frameworks, and best practices
• Maintain libraries of test cases, evaluation criteria, and performance benchmarks
• Create playbooks for common conversational scenarios and business use cases

Required Experience and Skills

• Strong analytical and problem-solving skills with high attention to detail
• Solid understanding of data visualization best practices
• Proficiency in SQL and Python for analysis, validation, and experimentation
• Experience working with stakeholders to gather and translate business requirements
• Strong communication skills across technical and business audiences

Preferred Experience

• Hands-on experience with prompt engineering, LLMs, or conversational AI systems
• Experience working in cloud environments, preferably GCP
• Knowledge of conversation design principles or UX research
• Background in customer support analytics or user-facing analytics roles