Python/Spark/AI Developer

Our client is seeking an experienced Python and Spark Data Engineer to develop production-grade data pipelines and support the replatforming of legacy workloads into a modern lakehouse environment. This role is suited to an autonomous engineer who values clean, tested code and can work confidently from technical specifications and architecture decision records.

Requirements

• 4+ years' professional Python development experience.
• 2+ years' production experience building Spark/PySpark or comparable distributed data pipelines.
• Strong Python 3.12 skills, including type hints, Pydantic, ABC-based patterns and modern packaging.
• Strong Apache Spark knowledge, including DataFrame API, Spark SQL, partitioning, tuning and driver/executor architecture.
• Production experience with Delta Lake or equivalent technologies such as Iceberg or Hudi.
• Strong SQL skills and the ability to translate complex legacy T-SQL logic into Spark SQL.
• Experience with pytest, fixtures, test markers and tiered testing strategies.
• Practical FastAPI or equivalent REST API development experience.
• Experience with Docker and Compose-based development environments.
• Strong Git and CI/CD discipline, including GitHub flow, Ruff, static type checking and automated test gates.
• Bachelor's degree in Computer Science, Engineering or equivalent experience.

Should you meet the requirements for this position, please email your updated CV attached to Sign in to view email, alternatively contact 031 350 4018 or visit our website, www.tumaini.co. Correspondence will only be conducted with short listed candidates. Should you not hear from us within 3 days, please consider your application unsuccessful.