Tech Lead - AWS SageMaker Job

Experience

8+ Years

Job Description

Key Responsibilities:

Data Engineering:

·        Data LakeHouse:

o Simplify analytics and AI with unified data management across Amazon S3 data lakes, Amazon Redshift data warehouses, operational databases, and enterprise applications.

·        Data Processing:

o Use Visual ETL jobs and unified notebooks to perform data analysis, preparation, and integration with data sources like Amazon S3, Redshift, and AWS RDS.

·        Orchestration and Automation:

o Create and manage SageMaker Pipelines and Workflows to automate the ETL Pipelines, ML model continuous integration and continuous delivery (CI/CD) lifecycle.

·        Data Governance and Collaboration:

o Work within collaborative projects in SageMaker Unified Studio, leveraging the SageMaker Catalog (built on Amazon DataZone) to securely share data and AI artifacts.

o Implement various features in the project like Lineage, Data Quality, Metadata forms, Business Glossaries etc.

AI/ML:

·        End-to-End MLOps:

o Build, deploy, execute, and monitor complete data and AI workflows using SageMaker Unified Studio's integrated environment.

·        Model Development:

o Develop, train, and evaluate ML and generative AI models using SageMaker's purpose-built tools and algorithms.

o Utilize advanced features such as HyperPod for distributed training, JumpStart for pre-trained models, and SageMaker Experiments for tracking model versions.

·        Generative AI Development:

o Rapidly build and customize generative AI applications within the integrated Amazon Bedrock environment in Unified Studio.

o Implement advanced customization capabilities like Knowledge Bases, Guardrails, Agents, and Flows.

·        Security and Compliance:

o Implement and maintain security best practices, including fine-grained access controls and data protection, within the SageMaker environment.

·        Performance Optimization:

o Optimize the performance and cost of ML workloads by right-sizing compute resources and monitoring job metrics.

·        Troubleshooting:

o Utilize Amazon Q Developer's generative AI-powered assistance for code generation and resolving issues within the notebooks and jobs.

Required Skills

·        Experience with data processing and analytics using services integrated with Sagemaker Unified Studio (e.g., AWS Glue, Amazon EMR, Amazon Athena, AWS Redshift).

·        Demonstrable expertise with Amazon SageMaker Unified Studio and its core components (Domain Units, Projects, Data Assets, Jobs, Data Products).

·        Proficiency in PySpark, Python and relevant ML/AI libraries such as TensorFlow, PyTorch, and Scikit-learn.

·        Experience with generative AI application development using Amazon Bedrock services within Unified Studio.

·        Proven experience developing machine learning models and data pipelines on AWS.

·        Solid understanding of MLOps principles and experience building CI/CD pipelines for ML models.

·        Familiarity with AWS security best practices, including IAM roles, policies, and data governance, AWS Lake Formation

·        Knowledge of SQL and experience with the built-in SQL editor in Unified Studio for data querying

·        AWS Certified Data Engineer Associate / AWS Certified Machine Learning – Specialty or other relevant AWS certifications.

Required Technical/ Functional Competencies

Domain/ Industry Knowledge: 

• Basic knowledge of customer's business processes- relevant technology platform or product.
• Able to prepare process maps, workflows, business cases and simple business models in line with customer requirements with assistance from SME and apply industry standards/ practices in implementation with guidance from experienced team members.

Requirement Gathering and Analysis: 

• Working knowledge of requirement management processes and requirement analysis processes, tools and methodologies.
• Able to analyse the impact of change requested / enhancement / defect fix and identify dependencies or interrelationships among requirements and transition requirements for the engagement.

Product/ Technology Knowledge:

• Working knowledge of technology product/platform standards and specifications.
• Able to implement code or configure/customize products and provide inputs in design and architecture adhering to industry standards/ practices in implementation.
• Analyse various frameworks/tools, review the code and provide feedback on improvement opportunities.

Architecture tools and frameworks: 

• Basic knowledge of architecture Industry tools & frameworks
• Able to analyse available tools and frameworks for review by the SME and plan for tool configurations and development.

Architecture concepts and principles: 

• Basic knowledge of architectural elements, SDLC, methodologies.
• Able to apply various architectural constructs in the projects and identify various architectural patterns and implement.

Analytics Solution Design: 

• High-level awareness of a wide range of core data science/analytics techniques, their advantages, disadvantages, and areas of application.

Tools & Platform Knowledge: 

• Familiar with wide range of mainstream commercial and open-source data science/analytics software tools, their constraints, advantages, disadvantages, and areas of application.

Required Behavioral Competencies

Accountability:

• Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration:

• Participates in team activities and reaches out to others in team to achieve common goals.

Agility:

• Demonstrates a willingness to accept and embrace differing ideas or perceptions which are beneficial to the organization.

Customer Focus:

• Displays awareness of customers stated needs and gives priority to meeting and exceeding customer expectations at or above expected quality within stipulated time.

Communication:

• Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results:

• Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Certifications