Apple AR/VR Job | Sr Software Engineer - LLM Evaluation and Test Infrastructure
Job（岗位）: Sr Software Engineer - LLM Evaluation and Test Infrastructure
Citys（岗位城市）: Santa Clara Valley (Cupertino), California, United States
Date（发布日期）: 2023-6-29
Summary（岗位介绍）
Do you believe generative models can transform smart assistants and creative workflows used by billions? Do you believe it can fundamentally shift how people interact with devices and communicate? We truly believe it can! We are looking for highly skilled senior software engineers experienced in architecting and deploying test and validation infrastructure for consumer-facing products using large-scale client/server ML systems.
You will play a critical role in designing and building LLMs online and offline validation infrastructure, tools, frameworks, and methodologies that enable product teams across Apple to develop state-of-the-art generative AI solutions that power amazingly intelligent user experiences on iPhone, Mac, Watch, iPad, and more. We are the Intelligence System Experience (ISE) team within Apple’s software organization. The team works at the intersection between multimodal machine learning and system experiences. System Experience (Springboard, Settings), Keyboards, Pencil & Paper, Shortcuts are some of the experiences that the team oversees. These experiences that our users enjoy are backed by production scale ML workflows. Visual Understanding of People, Text, Handwriting & Scenes, multilingual NLP for writing workflows & knowledge extraction, behavioral modeling for proactive suggestions, and privacy preserving learning are areas our multi disciplinary ML teams focus on.
Qualifications（岗位要求）
6+ years of proven expertise in software development with Java, Python, Swift, or Objective-C
Strong software engineering skills to create a scalable and robust infrastructure for generative AI systems training and inference pipelines leveraging on-device and server-side ML models.
Fundamental knowledge of ML concepts and understanding of ML-based product lifecycle and operations
Familiarity with ML toolkits such as PyTorch, TensorFlow, SkLearn, etc.
Hands-on experience in building an automation infrastructure, tools, and frameworks for large-scale AI/ML or cloud- based applications leveraging cross-functional partnerships
Demonstrated experience in evaluating and testing machine learning models. Familiarity with performance metrics, validation techniques, and statistical analysis.
A passion for staying up-to-date with the latest advancements in generative AI models and their applications. Ability to quickly learn new technologies, adapt to evolving project requirements, and think creatively to solve complex challenges.
Excellent verbal and written communication skills to effectively collaborate with multidisciplinary teams. Ability to articulate complex technical concepts to both technical and non-technical stakeholders.
Description（岗位职责）
This role is responsible for defining and developing generative AI systems validation infrastructure by collaborating with diverse cross-functional partners, including machine learning engineers, data scientists, software developers, and product managers. The evaluation infrastructure includes adopting or building new client and server CI pipelines and developing scalable automation tools and frameworks across all Apple platforms. The evaluation infrastructure should allow cross-functional teams to conduct rigorous testing to ensure the accuracy, performance, reliability, and stability of ISE's future generative AI models across various products and use cases.
This role will also work closely with product and program teams to ensure validation infrastructure development is done in lockstep with the new features and products roadmap and to drive a high adoption among product and ML engineers. This is a fast-paced role with high visibility, impact, and influence in building one of the industry's most advanced generative AI systems.
