Data Scientist - Model Evaluation

Introduction

Rise to meet the challenge of solutioning the world’s problems! At IBM, work is more than a job - it's a calling: To build. To design. To code. To consult. To think along with clients and sell. To make markets. To invent. To collaborate. Not just to do something better, but to attempt things you've never thought possible. Are you ready to lead in this new era of technology and solve some of the world's most challenging problems? If so, lets talk.

Your Role And Responsibilities

We are looking for Data Scientist to work on dynamically evolving IBM watsonx.ai offering from Kraków (Poland) office in product area of Model Evaluation.

In this position, you'll be immersed in the forefront of the AI revolution. Your duties will include critically assessing generative AI models and processes/enhancements related to them, establishing quality benchmarks, and challenging models' capabilities.

Your Role And Responsibility Will Encompass

• Critically assessing and pushing the boundaries of generative AI models and tuning of such models, focusing on LLM models.
• Developing quality benchmarking framework and its components (in Python) assessing LLM models.
• Collaborate with the research team to develop state-of-the-art NLP solutions utilizing LLM models.
  
  

A Successful Candidate For This Role Should Demonstrate

• An intrinsic motivation, coupled with an insatiable appetite for continuous learning.
• Proficiency in crafting and sustaining tools to amplify productivity.
• Robust analytical capabilities and a keen problem-solving acumen.
• A tenacity to assume responsibility, driving problems to their resolution and tasks to fruition.
• Proficiency in collaborating seamlessly within diverse team dynamics.
  
  

For an overview of the product, see https://www.ibm.com/products/watsonx-ai

Preferred Education

Master's Degree

Required Technical And Professional Expertise

• Experience in AI/ML model development and evaluation with solid foundation in linear algebra, statistics, and optimization
• Proficiency & experience with coding & developing solutions in Python and Jupyter Notebook
• Demonstrated ability to critically assess and challenge AI (including LLM) models to ensure optimal performance
• Fluent in written and spoken English
  
  

Preferred Technical And Professional Experience

• Expertise & up-to-date knowledge of advances in Foundation Models area, focusing on Large Language and Multimodal Models
• Design, develop NLP models to extract insights and information from unstructured data
• Background in developing and implementing test automation frameworks and strategies
• Experience with cloud environments