Python/Spark/AI Developer - Remote

MSP Staffing Pty Ltd

Date: 1 week ago
City: Paarl, Western Cape
Contract type: Full time

Our client is seeking an experienced Python/Spark/AI Developer to join a technically driven team focused on modernising data platforms and building scalable AI-enabled solutions. This role is ideal for a developer who enjoys working with distributed data, clean and testable Python, APIs, and modern lakehouse technologies.

Key Requirements

  • Bachelor's degree in Computer Science, Engineering or equivalent experience.
  • 4+ years' professional Python development experience.
  • 2+ years' production experience developing Spark or comparable distributed data pipelines.
  • Strong Python 3.12 skills, including type hints, Pydantic and modern packaging practices.
  • Solid Apache Spark/PySpark knowledge, including Spark SQL, DataFrame APIs, partitioning and performance optimisation.
  • Experience with Delta Lake or equivalent technologies such as Iceberg or Hudi.
  • Strong SQL skills and the ability to translate legacy T-SQL logic into Spark SQL.
  • Practical experience with pytest and automated testing.
  • Experience developing REST APIs with FastAPI or equivalent technologies.
  • Familiarity with Docker-based development environments.
  • Strong Git and CI/CD practices.

Advantageous

  • Experience integrating LLMs and AI services.
  • Knowledge of data privacy, PII handling, POPIA/GDPR concepts and data governance.
  • Exposure to Hive Metastore, Trino or Apache Ranger.
  • Azure data platform experience, including ADLS Gen2, Synapse or ADF.
  • Experience with observability tools and structured logging.
  • Previous data migration and pipeline parity experience.
  • Databricks or equivalent Spark certification.

Should you meet the requirements for this position, please email your CV to . You can also contact the IT team on 031 350 3607 or visit our website at https://mspstaffingza.co/ NOTE: When replying to the advert, also include the reference number in the subject line. Correspondence will only be conducted with short listed candidates. Should you not hear from us within 3 days, please consider your application unsuccessful.

For employers only

Is this your company's job post? Verify ownership to manage this listing and receive applications directly.

Claim this listing

Looking to apply for this job? Use the Apply button above.