Support Python Engineer

About the Role:

We are seeking a skilled Support Python Engineer to develop, maintain, and optimize Python-based data applications in Azure environments. The role focuses on designing ETL/ELT pipelines, implementing scalable data solutions, and supporting real-time and batch data processing. The ideal candidate combines strong Python development expertise with Azure DevOps experience, database proficiency, and a passion for high-quality, reliable data solutions.

Key Responsibilities:

• Develop, maintain, and enhance Python-based data applications running in Azure environments.

• Design, implement, and optimize ETL/ELT pipelines for ingesting and processing data from multiple sources.

• Build, maintain, and optimize CI/CD pipelines using Azure DevOps for automated deployment of data applications.

• Implement data transformation logic using Pandas, PySpark, SQL, and Azure Data Factory.

• Ensure data quality and reliability through validation, cleansing, and error handling mechanisms.

• Optimize database queries, indexes, and performance for Azure SQL Database, SQL Server, and NoSQL/IoT data stores.

• Monitor data pipeline performance, troubleshoot failures, and perform root cause analysis.

• Collaborate with DevOps and infrastructure teams on deployment automation and containerized applications (Docker, AKS).

• Document data schemas, pipeline workflows, transformation logic, and technical solutions.

• Participate in code reviews and promote best practices for Python development and data engineering.

• Support integration of IoT and streaming data into data warehouses and analytics platforms.

• Implement data security measures, including encryption, access control, and auditing.

• Create and maintain automated tests for data pipelines and transformations.

• Provide technical guidance on data architecture, modeling, and best practices.

Required Skills & Experience:

Python Development:

• 3–5 years professional Python development experience (Python 3.x).

• Advanced knowledge of data processing libraries (Pandas, NumPy, Polars, PySpark).

• Experience with Python frameworks for APIs (FastAPI, Flask) and async programming.

• Knowledge of Python best practices, type hints, documentation, and package management (pip, poetry, conda).

Azure & DevOps:

• Hands-on experience with Azure DevOps Pipelines (YAML and Classic), Repos, and Artifacts.

• Experience designing automated CI/CD pipelines for Python applications, including testing integration.

• Knowledge of Infrastructure as Code (IaC) concepts and deployment strategies (blue-green, canary, rolling updates).

Database & Data Processing:

• Strong experience with Azure SQL Database, SQL Server, and NoSQL databases (Cosmos DB, MongoDB).

• Proficient in SQL query optimization, database schema design, stored procedures, and indexing.

• Knowledge of time-series databases for IoT/telemetry data (InfluxDB, TimescaleDB).

• Experience with data modeling techniques (star, snowflake, data vault) and data validation/cleansing frameworks.

Data Integration & Messaging:

• Experience with data transformation frameworks (Pandas, PySpark) and ETL workflows.

• Familiarity with streaming data processing (Azure Stream Analytics, Apache Kafka) and messaging protocols (MQTT).

• Understanding of data serialization formats (JSON, Parquet, Avro, CSV).

Development Tools:

• Proficiency with Git, Python IDEs (VS Code, PyCharm, Jupyter), debugging, profiling, Linux CLI, and Bash scripting.

• Experience with Docker, AKS, Azure CLI, and PowerShell for automation.

Soft Skills:

• Strong analytical and problem-solving skills.

• Ability to translate business requirements into technical solutions.

• Excellent communication and collaboration with business stakeholders and cross-functional teams.

• Attention to detail for data quality and accuracy.

• Self-motivated, willing to learn, and able to work independently and in distributed teams.

Preferred Qualifications:

• Microsoft Certified: Azure Data Engineer Associate (DP-203)

• Microsoft Certified: Azure Developer Associate (AZ-204)

• Microsoft Certified: Azure Database Administrator Associate (DP-300)

• Microsoft Certified: Azure Fundamentals (AZ-900)

• Experience with Apache Spark and big data processing.

• Familiarity with data visualization tools (Power BI, Grafana, Plotly).

• Knowledge of machine learning pipelines, MLOps, and IoT platforms.

• Understanding of data governance, cataloging, and Azure Purview.

• Prior experience in railway, transportation, or critical infrastructure industries.

• Agile/Scrum development experience.