Biggest Challenges Faced By A Managing Director, Quantitative Strategies At O Asset Management
Trevor's biggest challenge as Managing Director of Quantitative Strategies is "bringing large-scale AI systems into production," citing difficulties in hiring individuals with specialized "ML Ops" skills and managing the substantial technical debt inherent in scaling AI prototypes to robust production systems. A significant portion of AI models never reach production, highlighting the difficulty of this process.
Machine Learning Operations (ML Ops), AI Production Systems, Technical Debt Management, Hiring and Team Scaling, Applied AI Research
Advizer Information
Name
Job Title
Company
Undergrad
Grad Programs
Majors
Industries
Job Functions
Traits
Trevor Richardson
Managing Director, Quantitative Strategies
O Asset Management
Arizona State University
M.S. Computer Science at Arizona State University
Engineering - Industrial
Finance (Banking, Fintech, Investing), Technology
Data and Analytics
Honors Student, Scholarship Recipient, Took Out Loans, Worked 20+ Hours in School
Video Highlights
1. Managing and scaling AI systems in production is challenging due to the difficulty in hiring and managing technical debt.
2. Developing robust and scalable AI systems requires expertise in ML Ops, focusing on transitioning prototypes into production-ready systems.
3. A significant portion of AI models fail to reach production, highlighting the difficulty of the process and the importance of addressing challenges like technical debt and scaling
Transcript
What is your biggest challenge in your current role?
The biggest challenge for bringing large-scale AI systems into production right now is hiring, scaling, and managing technical debt. For anyone wanting to work in applied AI, there are areas of research you should look into, such as ML Ops, or Machine Learning Operations.
It's one thing to build a prototype, but it's another to manage all of its components and scale them robustly into a production system. This requires hiring individuals with very difficult-to-find skills, and the team must grow. It's challenging to find people who fit these needs.
Scaling your approach requires a better job of automating all the artifacts from your technical workflow and managing them. This is often tedious and not the fun part of the job. The fun part is looking at the loss function, creating new equations, and seeing things work.
The less enjoyable but necessary part is writing many pages of versioning documents to meet production requirements. So, when you consider bringing AI into a production system, understand that the process is extremely difficult and fraught with many pitfalls.
Google conducted a research study on how many AI models actually make it into production, and the number was very low, around 15%. This low percentage highlights how much effort never actually gets deployed to produce value for organizations. This aspect of the process is a significant pain point that I would love to improve.