We are seeking a highly capable Data Engineer with strong experience in scalable data platforms, large-scale scraping, LLM integrations, and agentic AI systems. This role is ideal for a hands-on builder who enjoys solving complex data challenges and developing intelligent automation solutions that operate reliably in production environments.
You will play a key role in designing and building modern data pipelines, AI-powered workflows, and autonomous systems that can interact with tools, APIs, databases, and enterprise data sources.
Responsibilities:
Design and build scalable batch and near-real-time data pipelines.
Ingest and process data from APIs, databases, files, event streams, and web sources.
Develop and maintain large-scale scraping and data acquisition systems.
Clean, validate, transform, and model data for analytics and AI applications.
Build LLM-powered workflows and agentic AI solutions using:
Tool calling
Structured outputs
Retrieval-Augmented Generation (RAG)
API and database integrations
Support modern data lake, warehouse, and lakehouse architectures.
Implement monitoring, observability, schema validation, and data quality controls.
Collaborate with engineering teams to deliver secure, scalable, and maintainable production systems.
Qualifications & Experience
Strong experience with Python and/or Node.js/TypeScript.
Advanced SQL and relational database skills.
Proven experience building production-grade data pipelines.
Experience with AWS or similar cloud platforms.
Hands-on experience with data lakes, warehouses, or lakehouse technologies.
Practical experience using LLM platforms such as OpenAI, Anthropic, Gemini, Bedrock, or similar.
Experience building agentic workflows involving:
Tool use
Task decomposition
Retrieval systems
Memory
Human-in-the-loop workflows
Experience with scraping frameworks such as Playwright, Puppeteer, Scrapy, or Selenium.
Strong understanding of APIs, containers, CI/CD, testing, and secure engineering practices.
The Reference Number for this position is NG61354 which is a Contract Remote role offering a salary of up to R550 per hour salary negotiable based on experience. E-mail Nokuthula on or call her for a chat on to discuss this and other opportunities.
Are you ready for a change of scenery? e-Merge IT recruitment is a niche recruitment agency. We offer our candidates options so that we can successfully place the right people with the right companies, in the right roles. Check out the e-Merge IT website www.e-merge.co.za for more great positions.
Do you have a friend who is a developer or technology specialist? We pay cash for successful referrals
Originally posted on WakandaJobs.com - Source: www.wakandajobs.com