Data Analyst

You'll be the bridge between our business stakeholders/users and our conversational analytics AI system, ensuring that our chatbot consistently delivers value by understanding real user needs, testing solutions, and continuously optimizing performance through data-driven iteration. As a Data Analyst in Business Support domain, you will be responsible for the ongoing quality and effectiveness of our conversational analytics platform. You'll work hands-on with prompt engineering, evaluation framework development, and performance monitoring, while also serving as the key liaison between business users and the technical team to translate needs into actionable improvements. Your work will directly impact how well our conversational AI serves our users, combining analytical rigor with practical experimentation and stakeholder engagement.

Testing & Optimization:

1. Design and execute systematic tests of business questions and use cases within the conversational AI system

2. Develop, refine, and optimize prompts to improve response quality, accuracy, relevance of the data visualisations.

3. Conduct A/B testing and comparative analysis of different prompt strategies

4. Identify edge cases, failure modes, and opportunities for improvement through rigorous testing

Evaluation & Monitoring:

1. Develop metrics and KPIs to measure conversational AI performance across different dimensions (accuracy, relevance, user satisfaction, task completion)

2. Conduct regular performance reviews and regression testing to ensure sustained quality

3. Analyze trends and patterns in system performance to proactively identify areas for improvement

Stakeholder Engagement & Requirements Gathering:

1. Have meetings business users across departments to understand their needs and pain points

2. Translate business requirements into specific conversational AI feeatures

3. Communicate findings, insights, and recommendations to both technical and non-technical stakeholders

Documentation & Knowledge Management:

1. Document prompt engineering best practices and guidelines

2. Maintain a library of test cases, evaluation criteria, and performance benchmarks

3. Create playbooks for common conversational scenarios and business use cases

4. Share learnings and insights with the broader team to build organizational knowledge

Personal Qualities:

1. Curious and experimental mindset - you love testing hypotheses and channel with new ways of thinking.

2. Detail-oriented with a commitment to quality

3. Collaborative team player who can work across functions

4. Self-starter who can manage multiple priorities independently

What 3 things from the box above are most important?

• Strong analytical and problem-solving skills with attention to detail
• Familiarity with data visualization best practices
• Proficiency in SQL and Python