Artificial Intelligence Engineer

Purview is a global leader in IT Engineering and Talent Solutions, trusted by Fortune 500 and mid-market clients across 21+ countries. Headquartered in Edinburgh, with delivery centres in India, our 1,200+ professionals drive meaningful digital transformation worldwide.

As a certified Ethnic Minority Business and Premier Member of MSDUK, we proudly blend diversity with deep technical expertise to deliver results that matter.

We’re currently hiring for the below role with one of our valued clients. If it aligns with your experience and aspirations, we’d love to hear from you!

Job Details:

Role: AI Platform & Databricks Engineer

Location: Barcelona, Catalonia, Spain

Type of Employment: Contract B2B

Place of Work: Hybrid- 2/3 days to the office a week

Experience Required: 6-12 Years

Summary:

We are seeking a hands-on AI Platform Engineer to design, build, and operate Databricks-based data and AI platforms on AWS. You will enhance AI capabilities by leveraging Databricks (Workspace, Unity Catalog, Lakehouse, MLflow), modern cloud services, and DevOps/MLOps practices to deliver reliable, secure, and scalable platforms.

• Administer and optimize Databricks workspaces: cluster policies, pools, job clusters vs. all-purpose clusters, autoscaling, spot/fleet usage, and GPU/accelerated compute where applicable.
• Implement CI/CD for notebooks, libraries, DLT pipelines, and ML assets; automate testing, quality gates, and promotion across workspaces using GitHub Actions and Databricks APIs.
• Develop and optimize Delta Lake pipelines (batch and streaming) using Auto Loader, Structured Streaming, and DLT; enforce data quality and SLAs with expectations and alerts.

Required Experience:

• Design and implement scalable Databricks platform solutions to support analytics, ML, and GenAI workflows across environments (dev/test/prod).
• Hands-on Databricks administration on AWS, including Unity Catalog governance and enterprise integrations.
• Strong AWS foundation: networking (VPC, subnets, SGs), IAM roles and policies, KMS, S3, CloudWatch; EKS familiarity is a plus but not required for this Databricks-focused role.
• Proficiency with Terraform (including databricks provider), GitHub, and GitHub Actions.
• Experience with MLflow for tracking and model registry; experience with model serving endpoints preferred.
• Familiarity with Delta Lake, Auto Loader, Structured Streaming, and DLT.
• Experience implementing DevOps automation and runbooks; comfort with REST APIs and Databricks CLI.

Job Description:

• Proven hands-on experience with Databricks on AWS: workspace administration, cluster and pool management, job orchestration (Jobs/Workflows), repos, secrets, and integrations.
• Strong experience with Databricks Unity Catalog: metastore setup, catalogs/schemas, data lineage, access control (ACLs, grants), attribute-based access control, and data governance.
• Expertise in Infrastructure as Code for Databricks and AWS using Terraform (databricks and aws providers) and/or AWS CloudFormation; experience with Databricks asset bundles or CLI is a plus.
• Experience with DevOps practices to enable automation strategies and reduce manual operations.
• Experience or awareness of MLOps practices; building pipelines to accelerate and automate machine learning will be viewed favorably.
• Proficient in cloud operations on AWS, with strong understanding of scaling infrastructure and optimizing cost/performance.