HubMub
-
Veracity Software Inc

Data scientists, AI engineering, LLM engineer, Machine Learning Engineers

Veracity Software Inc
United States · Full-time · Mid-Senior
Role: Data scientists, AI engineering, LLM engineer, Machine Learning Engineers

Charlotte, NC (Need Local candidate only)

Video Interview

Look for an ex-Client/Banking & Financial domain candidate.

Project Details:

  • Implemented a chatbot internally with the bank, build interface and now users can interact.
  • It's a RAG framework so instead of tuning it into the actual applications you can prompt it to give you prevectorized queries.
  • Able to feed it documents even if they don't know what your team is or who you are. Should have 1000 users by the end of the year and another 2000 next year.

Must Haves / Required Skills:

  • LLMs & Inference:
  • Experience with major LLMs, specifically Llama 3, Mistral, and possibly "Quinn."
  • Direct experience with VLLM (an inference engine) is a perfect match, as it's the core technology they are using to handle batched requests.
  • Experience with Nvidia Triton is a "big bonus" and a key part of their model serving infrastructure.
  • Core Development:
  • Python: A mandatory skill. They are using Python 3.12, but experience with 3.10 and above is sufficient.
  • Web Frameworks: Knowledge of Flask or FastAPI is required, as they are using a Python endpoint to host the LLM.
  • Java: A "secondary preferred" skill, used to create the REST service that interacts with the front-end UI.
  • Database & Data Management:
  • Vector Databases: Experience with Redis and other vector databases is essential for the RAG component.
  • SQL: Required.
  • RAG Skills: The candidate needs to understand how to handle the business-side parameters from the product team and push back if they are technically unfeasible. This shows they need to be a critical thinker, not just a code-jockey.
  • Infrastructure & Operations (MLOps)
  • Containers & Orchestration: Knowledge of containers and OpenShift (a Kubernetes platform) for CI/CD.
  • CI/CD Tools: Experience with XLR and Datical for pipeline deployments is required.
  • Hardware: A solid understanding of GPUs is necessary, as they are the most critical and challenging component of their infrastructure.
  • Agile: The team uses Agile methodology.
  • Scaling: The project's growth is tied to hardware availability. The initial deployment will be capped at 1,000 users, and scaling will only happen with more budget. This shows the importance of efficient resource management.

Skills / Experience That Are A Plus

"nice to have”, but not necessarily required:

  • Based on exp, likes to dig deep on exp.
  • General awareness of Vector DB as opposed to relational and others.
  • Pushing code to a controlled environment and seeing something go into production. Better if it's an AI application.
  • Any model experience they have had professionally, some quantitative models in the past or have written white papers before (required at Client)

Key Skills

Ranked by relevance

python cicd ai machine learning kubernetes fastapi redis flask java sql ui
Login to Apply
🇺🇸

Country Guide

United States

World’s deepest and highest-paying tech market

Posted
Aug 18, 2025
Type
Full-time
Level
Mid-Senior
Location
Charlotte
Company
Veracity Software Inc

Industries

IT Services IT Consulting

Categories

Other

Related Jobs

3 roles aligned with this opportunity

View all jobs
View Job Details
Veracity Software Inc
Related

Engineer, AI / ML & Analytics Platform Engineering

2026-02-18

Full-time
Mid-Senior
United States
IT Services
Other
View Job Details
Veracity Software Inc
Related

Data Scientist

2026-02-10

Full-time
Mid-Senior
United States
IT Services
Other
View Job Details
Veracity Software Inc
Related

Artificial Intelligence / Machine Learning Architect, Data Scientist, Machine Learning Engineer, Dat

2026-02-18

Full-time
Mid-Senior
Canada
IT Services
Other