Project Overview
- I was brought in into this ongoing project once the customers and stakeholders realized the importance of integrating LLMs and GenAI into the solution
- My work started with understanding the progress and technical decisions made so far, understanding the customer's end goal and creating a project plan utilizing the knowledge and progress so far and closing gaps to reach the final goal
- The project implementation plan covered areas of:
- Data Engineering and Analytics: For multimodal, multi source data handling
- Analyzing Foundation Models: To fit the needs of problem space, solution suite and
- RAG: Amplifying model relevance with the use of statistical data
- LLMOps Tools: The comparison and integration of tools such as WandB, MLFlow etc. to fit our experiment and inference monitoring, transparency and governance
- With this plan in place, we were able to promise a deliverable to the client, which we managed to uphold within the original timeline
- As a team of 5 - A senior data scientist, a junior data scientist, an MLOps Engineer, a Data Analyst and myself as the Project Lead and Staff Data Scientist
Skills
LLMs and LLMOps
This project involved substantial research and deep dive into modeling (LLMs and GenAI) and operations (both data engineering and MLOps) to ingest, transform and generate from multimodal invoice documents, which was a leap forward for everyone involved
Leading a team of freelancers
As part of this project, I got the opportunity to work with and lead a remote team of talented freelancers - each of whom was highly motivated, driven and bringing in their own expertise. And it was delightful to actively experience the ML powerhouse they brought together
Full steam ahead
Since I was brought in at the middle of the project, but still had to maintain momentum - I got to gain and practice tech leadership and management skills that involved prioritizing what's best for the product, customer and their clients, without comprimising on features and quality