Module Lead - Azure Job

Experience

Job Description

-Must Have-Databricks-Advanced PySpark, Spark SQL, Delta Lake, Unity Catalog, Databricks Workflows/Jobs, performance tuning
-Must Have-Azure Data Factory-Pipeline development, orchestration, parameterisation, monitoring, error handling, incremental loading.
-Must Have-SQL-Strong analytical SQL, data modelling, window functions, CTEs, performance optimisation
-Must Have-Python-Production-quality Python, type hints, modular code, reusable libraries, testing
-Must Have-Software Engineering Practices-Git, pull requests, code reviews, structured development practices
-Must Have-CI/CD-Azure DevOps pipelines, deployment automation, environment promotion
-Must Have-Data Pipeline Testing-Unit testing, integration testing, data quality validation
-Must Have-Documentation & Maintainability-Well-structured code, clear naming conventions, reusable components, comprehensive documentation
-Strongly Preferred-GitHub Copilot / AI-Assisted Development-Practical experience using AI coding assistants to accelerate delivery while maintaining quality.
-Strongly Preferred-AI-Maintainable Coding Practices-Writing code that can be easily understood and extended by AI agents and future developers.
-Strongly Preferred-Databricks AI Capabilities-Awareness of AI Functions, Genie, AI/BI Dashboards, Mosaic AI or similar capabilities.
-Strongly Preferred-DataOps / Infrastructure-as-Code-Terraform, Bicep, Databricks Asset Bundles (DABs), deployment as code.
-Strongly Preferred-Prompt Engineering for Development-Ability to specify requirements effectively for AI-assisted coding workflows.
-Preferred-Docker-Containerising Python applications and services
-Preferred-Kubernetes Literacy-Basic understanding of Kubernetes concepts and deployment patterns
-Preferred-DevOps Practices-Ability to deploy and support own solutions with minimal platform team involvement
-Preferred-Power BI-Azure-connected reporting, gateway configuration, scheduled refresh
-Nice to Have-Basic Geoscience Domain Knowledge    Drillhole, geochemistry, geoscience data structures and workflows.
-Nice to Have-Workflow Orchestration-Argo Workflows orchestration.
-Nice to Have-Data Governance & Metadata-Data cataloguing, lineage and metadata management

Required Technical/ Functional Competencies

Domain/ Industry Knowledge:

• Basic knowledge of customer's business processes- relevant technology platform or product.
• Able to prepare process maps, workflows, business cases and simple business models in line with customer requirements with assistance from SME and apply industry standards/ practices in implementation with guidance from experienced team members.

Requirement Gathering and Analysis:

• Working knowledge of requirement management processes and requirement analysis processes, tools & methodologies.
• Able to analyse the impact of change requested/ enhancement/ defect fix and identify dependencies or interrelationships among requirements & transition requirements for engagement.

Product/ Technology Knowledge:

• Working knowledge of technology product/platform standards and specifications.
• Able to implement code or configure/customize products and provide inputs in design and architecture adhering to industry standards/ practices in implementation.
• Analyze various frameworks/tools, review the code and provide feedback on improvement opportunities.

Architecture tools and frameworks:

• Working knowledge of architecture Industry tools & frameworks.
• Able to identify pros/ cons of available tools & frameworks in market and use those as per Customer requirement and explore new tools/ framework for implementation.

Architecture concepts and principles:

• Working knowledge of architectural elements, SDLC, methodologies.
• Able to provides architectural design/ documentation at an application or function capability level and implement architectural patterns in solution & engagements and communicates architecture direction to the business.

Analytics Solution Design:

• Knowledge of statistical & machine learning techniques like classification, linear regression modelling, clustering & decision trees.
• Able to identify the cause of errors and their potential solutions.

Tools & Platform Knowledge:

• Familiar with wide range of mainstream commercial & open-source data science/analytics software tools, their constraints, advantages, disadvantages, and areas of application.

Required Behavioral Competencies

Accountability:

• Takes responsibility for and ensures accuracy of own work, as well as the work and deadlines of the team.

Collaboration:

• Shares information within team, participates in team activities, asks questions to understand other points of view.

Agility:

• Demonstrates readiness for change, asking questions and determining how changes could impact own work.

Customer Focus:

• Identifies trends and patterns emerging from customer preferences and works towards customizing/ refining existing services to exceed customer needs and expectations.

Communication:

• Targets communications for the appropriate audience, clearly articulating and presenting his/her position or decision.

Drives Results:

• Sets realistic stretch goals for self & others to achieve and exceed defined goals/targets.

Resolves Conflict:

• Displays sensitivity in interactions and strives to understand others’ views and concerns.

Certifications