Senior AI Test and Evaluation Engineer
December 12
Description
• Develop AI Test and Evaluation CONOPS, whitepapers and other documentation • Develop scalable Test and Evaluation plans for a variety of AI models, to include process, techniques and tools for evaluation • Design tests for each type of AI model and around risks related to each model • Works with data scientists, engineers and developers to: understand the AI system under test to develop test procedures, identify test data and strategy for data set quality analysis, integrate testing tools and procedures with the CI/CD pipeline • Develop and collect metrics to analyze evaluation process, as well as metrics related to the AI-Model under test • Collaborate with AI Governance team to maintain visibility and explainability through testing • Implement testing processes in the AI system design, development and deployment life cycles • Identify the risks in testing of projects, particularly for assessing the limitations of planned tests on complex AI systems • Work within teams of AI/ML researchers and engineers using Agile development processes
Requirements
• Bachelor's degree in Computer Science, Data Science or related field and 8+ years of relevant experience, Masters with 6+ years experience, or PhD with 4+ years experience. • Experience with MLOps tools such as Kubernetes, KubeFlow, MLFlow or similar tools. • Experience using deep learning tools such as Pytorch and TensorFlow • Practical experience with: • AI Performance and vulnerability assessment • Adversarial AI toolkits (e.g., ART, CleverHans) • Formal methods for AI model assurance evaluation • Applying and automating AI interpretability & explainability tools and methods • System and subsystem level test process and automation • Creating user acceptance test scenarios • SecDevOps tooling and MLOps pipeline development
Benefits
• Competitive compensation • Health and Wellness programs • Income Protection • Paid Leave • Retirement
Apply NowSimilar Jobs
December 11
NVIDIA seeks a Field Applications Engineer for datacenter system design and customer engagement. Join us to deliver AI computing solutions to enterprise clients.
December 11
As a Senior Formal Verification Engineer at NVIDIA, verify designs of CPUs and GPUs using formal verification tools.
December 11
Join CLARA Analytics as a Senior Data Engineer, optimizing Data Lakehouse architecture and data workflows. Utilize your 7+ years experience to ensure high performance, scalability, and reliability.
December 10
Senior Application Engineer at GFiber specializing in Billing and Payments, developing software components.
December 10
Join Safelite as a Senior SCCM Engineer, managing SCCM infrastructure in a diverse workplace. Ensure optimal deployment configurations and support for end users.