Meta AR/VR Job | Research Engineer - Systems ML (Compiler), Live Telepresence with Codec Avatars
Job（岗位）: Research Engineer - Systems ML (Compiler), Live Telepresence with Codec Avatars
Type（岗位类型）: Computer Vision
Citys（岗位城市）: Pittsburgh, PA
Date（发布日期）: 2024-7-20
Summary（岗位介绍）
Meta Reality Labs Research (RLR) is dedicated to the research and development required to bring virtual and augmented reality to billions of people around the world. At our lab, we aspire to a vision of social VR and AR, where people are able to interact with each other across distances in a way that is indistinguishable from in-person interactions. Meta RLR is looking for a talented Research Engineer to accelerate the progression to authentic social presence in Virtual Reality and Augmented Reality. In this role, you will be a member of the Mobile Telepresence team which has been committed to comprehensive mobile optimization that delivers end2end mobile telepresence with high energy efficiency. Particularly, this compiler research engineer role will focus on real-time efficiency and latency for each component of the pipeline across CPU, DSP, NPU, and mobile GPU and build up a unified compilation framework to automate research validation and product prototyping.
You will be working on designing runtime libraries for state-of-the-art CV/ML/DL algorithms and large foundation models against different compute hardware with high energy efficiency and throughput, standardizing accelerated operators/modules into edge compiler and runtime engine, and creating a unified compilation flow to accelerate telepresence deployment on the current & next generation of ARVR hardware platforms. You will work closely with AI and telepresence researchers to analyze algorithms (which are mixed operators of computer vision, deep learning models, and graphic rendering processing) and apply codesign methodologies from the early phase for low latency and high efficiency. You will also partner with software engineering teams (e.g. Pytorch Edge) to land acceleration solutions on ARVR hardware which may also have inhouse CV/ML accelerators. You will apply software development best practices to feature design and performance tuning. You will gain valuable experience in developing product driven and industrial scale compilers and will help in driving next generation hardware software codesign for AI domain problems.
Qualifications（岗位要求）
Bachelor's degree in Computer Science, Computer Engineering, relevant technical field, or equivalent practical experience.
3+ years practical experience on building compilers or runtime libraries for CV/ML applications on resource constrained hardwares.
Proven C++ and Python programming skills.
Familiar with frameworks like OpenCV, PyTorch, TensorFlow Lite, and ONNX.
Familiar with Neon SIMD optimization and OpenCL general parallel computing.
Solid knowledge of ARM CPU, embedded DSP, and high efficient AI accelerator architectures with proven understanding of system performance evaluations from such as end2end latency, memory bandwidth, I/O access, compute utilization analysis with necessary tooling development.
Experience on optimizing image processing and computer vision applications for edge devices with optimizations such as vectorization, parallelization, and hardware specific SIMD and deep learning algorithms with techniques such as low bits quantization and sparse computation.
Description（岗位职责）
Research and implementation of parallel computing algorithms for achieving high energy efficiency on VRAR computing platforms.
Development of native toolchain for end2end mobile telepresence pipeline to accelerate compilation and deployment on the next generation of ARVR platforms.
Collaborate with other research teams to analyze ML and DL algorithms and co-design/customize operators and modules to maximize mobile energy efficiency.
Additional Requirements（额外要求）
Familiar with embedded systems, Linux OS and driver development, and basic knowledge on Android application development.
Experience with custom compiler design that’s similar to MLIR and TVM.
Experience with acceleration on Variational Auto-Encoder, Transformers, Graph CNN, and stable diffusion models.
Experience with algorithms optimization for Qualcomm SoCs is a plus.
Experience on GPU general computing with OpenGL/Vulkan is a plus.
