| Date | 2024-07-07 |
|---|---|
| Job Title | Aws Data Engineer With Databricks |
| Employer | PBT Group More Information |
| Salary | 0 |
| Category | Finance Accounting |
| Location |
Gauteng / Johannesburg |
Job Summary
PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience.
Duties:
- Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd parties – Spark, EMR, DynamoDB, RedShift, Kinesis, Lambda, Glue, Snowflake.
- Analyze, re-architect and re-platform on-premise data warehouses to data platforms on AWS cloud using AWS or 3rd party services.
- Design and build production data pipelines from ingestion to consumption within a big data architecture, using Java, Python, Scala.
- Design and implement data engineering, ingestion and curation functions on AWS cloud using AWS native or custom programming.
- Perform detail assessments of current state data platforms and create an appropriate transition path to AWS cloud.
- Design, implement and support an analytical data infrastructure providing ad-hoc access to large datasets and computing power.
- Interface with other technology teams to extract, transform, and load data from a wide variety of data sources using SQL and AWS big data technologies.
- Creation and support of real-time data pipelines built on AWS technologies including Glue, Redshift/Spectrum, Kinesis, EMR and Athena
- Continual research of the latest big data and visualization technologies to provide new capabilities and increase efficiency
- Working closely with team members to drive real-time model implementations for monitoring and alerting of risk systems.
- Collaborate with other tech teams to implement advanced analytics algorithms that exploit our rich datasets for statistical analysis, prediction, clustering and machine learning
- Help continually improve ongoing reporting and analysis processes, automating or simplifying self-service support for customers
Experience:
- Advanced working SQL knowledge and experience working with relational databases, query authoring (SQL) as well as working familiarity with a variety of databases.
- Experience building/operating highly available, distributed systems of data extraction, ingestion, and processing of large data sets
- Experience working with distributed systems as it pertains to data storage and computing
- Experience performing root cause analysis on internal and external data and processes to answer specific business questions and identify opportunities for improvement.
- Strong analytic skills related to working with unstructured datasets.
- Build processes supporting data transformation, data structures, meta data, dependency and workload management.
- A successful history of manipulating, processing and extracting value from large disconnected data sets.
- Working knowledge of message queuing, stream processing, and highly scalable Big Data, data stores.
- Strong project management and organizational skills.
- Experience supporting and working with cross-functional teams in a dynamic environment.
- Experience in a Data Engineer or similar roles
- Experience with big data tools is a must: Hadoop, Spark, Kafka, etc.
- Experience with relational SQL and NoSQL databases, including Postgres and Cassandra.
- Experience with data pipeline and workflow management tools: Azkaban, Luigi, Airflow, etc.
- Experience with AWS cloud services: EC2, EMR, RDS, Redshift
- Experience with stream-processing systems: Storm, Spark-Streaming, etc.
- Experience with object-oriented/object function scripting languages: Python, Java, C++, Scala, etc
Qualifications/ Certification:
- Bachelor`s Degree in Computer Science, Information Technology or other relevant fields
- Has experience in any of the following AWS Athena and Glue Pyspark, EMR, DynamoDB, Redshift, Kinesis, Lambda, Snowflake
- Proficient in AWS Redshift, S3, Glue, Athena, DynamoDB, EMR
- Knowledge of software engineering best practices across the development lifecycle, including agile methodologies, coding standards, code reviews, source management, build processes, testing, and operations
* In order to comply with the POPI Act, for future career opportunities, we require your permission to maintain your personal details on our database. By completing and returning this form you give PBT your consent
Aws Data Engineer With Databricks position available in Gauteng, Johannesburg. This job position was posted by PBT Group. The job has been posted as a premium ad on 2024-07-07 at 13:08:17 in the Finance Accounting category
Click Go Apply to apply online!
You might also like to look at the available jobs in the same area.
Apply directly on jobmail for this position
Some posts have contact information included with other instructions. All applicants are urged to read the instructions carefully.
We will list jobs at the bottom of each job, so feel free to scroll down and continue your search.
We do not accept any job applications and merely offer the service of daily jobs being mined and displayed on our job portal.
Each job post has a link to take you directly to the original post to apply on their website or the original website where the post came from.
Here are some more related positions
Pretoria: Aws Data Engineer
PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and se
View Job
Aws Data Engineer
Cape Town City Centre: Aws Data Engineer
PBT Group is currently offering an opportunity for an AWS Data Engineer with 2 to 5 years of relevant experience. The role of a Data Engineer involves constructing and maintaining data pipelines and datamarts, emphasizing scalability, repeatability, and se
View Job
Aws Data Engineer
Centurion: Aws Data Engineer – Centurion– Up To R700 Per Hour
Our valued client, a leader in African capital markets and a powerhouse in the automotive industry, is renowned for its diverse product offerings and global reach. We are seeking a talented AWS Data Engineer to become a crucial part of their dynamic team.
View Job
Aws Data Engineer – Centurion– Up To R700 Per Hour
Pretoria: Aws Data Engineer – Pretoria – R500 To R700 Per Hour
A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant p
View Job
Aws Data Engineer – Pretoria – R500 To R700 Per Hour
Pretoria: Aws Data Engineer –Pretoria – R800k To R1mil Per Annum
An AWS Advanced partner that augments and accelerates organisational capabilities with agile, modern, data-driven solutions and services built according to well architected principles while providing technical thought leadership and innovation to go above
View Job
Aws Data Engineer –Pretoria – R800k To R1mil Per Annum
Johannesburg: Senior Data Engineer (Aws)- Johannesburg – R800k To R1mil Per Annum
A leading African Corporate Investment Bank and part of one of the largest financial services groups that are passionate Solutionists who offer clients innovative advisory, funding, trading, corporate banking and principal investing solutions have a vacant
View Job
Senior Data Engineer (Aws)- Johannesburg – R800k To R1mil Per Annum
Johannesburg: Aws Data Engineer With Databricks posted by PBT Group
PBT Group has an opportunity for an AWS Data Engineer, with DataBricks experience. Duties:Design, build and operationalize large scale enterprise data solutions and applications using one or more of AWS data and analytics services in combination with 3rd p
View Job
Aws Data Engineer With Databricks
Johannesburg: Aws Data Engineer/Jhb posted by Rooftop Recruitment
AWS Data Engineer/JHB Contract Type: 12 months Our client is seeking a highly motivated and experienced AWS Data Engineer to join their Global Markets division. The role is responsible for closely working the traders, quantitative analysts, and Data Scient
View Job
Aws Data Engineer/Jhb
Johannesburg: Aws Data Engineer – Johannesburg – R900k To R1.1Mil Per Annum
Pioneers in harnessing the power of Artificial Intelligence (AI) to deliver actionable insights and drive transformative change for businesses worldwide whose and vision is to provide next generation actionable insights to every company, globally, by harne
View Job
Aws Data Engineer – Johannesburg – R900k To R1.1Mil Per Annum
Randburg: Aws Senior Data Engineer posted by Planet10
Purpose of the Position To collect and validate business information (which can then be used by data scientists and business stakeholders). To design and implement data models and architectures that enable business stakeholders to gain insights from large-
View Job
Aws Senior Data Engineer
Randburg: Aws Senior Data Engineer posted by Planet10
Purpose of the Position To collect and validate business information (which can then be used by data scientists and business stakeholders). To design and implement data models and architectures that enable business stakeholders to gain insights from large-
View Job
Aws Senior Data Engineer
Johannesburg: Aws Data Cloud Engineer (Big Data Engineer) posted by Planet10
A leader in the insurance space is on the lookout for a AWS Data Cloud Engineer (big Data Engineer) Knowledge /Qualifications High knowledge in AWS security groups, permissions, VPC, roles, services. Experience developing engineering applications in Co
View Job
Aws Data Cloud Engineer (Big Data Engineer)
Johannesburg: Aws Data Cloud Engineer (Big Data Engineer) posted by Planet10
A leader in the insurance space is on the lookout for a AWS Data Cloud Engineer (big Data Engineer) Knowledge /Qualifications High knowledge in AWS security groups, permissions, VPC, roles, services. Experience developing engineering applications in Co
View Job
Aws Data Cloud Engineer (Big Data Engineer)
Johannesburg: Aws Data Engineer/Jhb posted by Rooftop Recruitment
AWS Data Engineer/JHB Contract Type: 12 months Our client is seeking a highly motivated and experienced AWS Data Engineer to join their Global Markets division. The role is responsible for closely working the traders, quantitative analysts, and Data Scient
View Job
Aws Data Engineer/Jhb
South Africa: Aws Data Engineer – Semi Remote – R740 Per Hour posted by E-Merge
A fantastic opportunity for an AWS Data Engineer to join a team of specialists developing the next generation of software systems.You will be responsible in performing thorough testing and data validation to ensure the accuracy of data transformationsCore
View Job
Aws Data Engineer – Semi Remote – R740 Per Hour
Bedfordview: Aws Redshift Data Engineer – Bedfordview – R1.1M Pa posted by E-Merge
e-Commerce environment for fast growing FCMG group which is a differentiated and diversified retailer of Furniture, Household appliances, Consumer electronic and technology goods. In addition, they are a leading consumer finance business providing innovati
View Job
Aws Redshift Data Engineer – Bedfordview – R1.1M Pa
Johannesburg: Aws Data Engineer – Johannesburg – Up To R800 Per Hour posted by E-Merge
This technology company is enthusiastic about resolving business challenges for their clients and is actively seeking an AWS Data Engineer to join their team. By employing advanced data analytics, actuarial services, software development, and automation, t
View Job
Aws Data Engineer – Johannesburg – Up To R800 Per Hour
South Africa: Data Engineer/Etl Developer (Aws) – Johannesburg North Up To R1.1M Per Annum posted by E-Merge
Are you currently responsible for data engineering solutions on a Cloud Data Platform?Can you code and develop and deploy software solutions?Are you highly Analytical thinker who can streamline processes?The successful incumbent will be responsible of the
View Job
Data Engineer/Etl Developer (Aws) – Johannesburg North Up To R1.1M Per Annum
Johannesburg: Senior Data Engineer (Aws)- Johannesburg – R800k To R1mil Per Annum posted by E-Merge
A leading African Corporate Investment Bank and part of one of the largest financial services groups that are passionate Solutionists who offer clients innovative advisory, funding, trading, corporate banking and principal investing solutions have a vacant
View Job
Senior Data Engineer (Aws)- Johannesburg – R800k To R1mil Per Annum
South Africa: Aws Data Engineer -Pretoria – R800k To R1mil Per Annum posted by E-Merge
An AWS Advanced partner that augments and accelerates organisational capabilities with agile, modern, data-driven solutions and services built according to well architected principles while providing technical thought leadership and innovation to go above
View Job
Aws Data Engineer -Pretoria – R800k To R1mil Per Annum
Centurion: Aws Data Engineer – Centurion- Up To R700 Per Hour posted by E-Merge
Our valued client, a leader in African capital markets and a powerhouse in the automotive industry, is renowned for its diverse product offerings and global reach. We are seeking a talented AWS Data Engineer to become a crucial part of their dynamic team.
View Job
Aws Data Engineer – Centurion- Up To R700 Per Hour
Johannesburg: Aws Data Engineer – Johannesburg – R900k To R1.1Mil Per Annum posted by E-Merge
Pioneers in harnessing the power of Artificial Intelligence (AI) to deliver actionable insights and drive transformative change for businesses worldwide whose and vision is to provide next generation actionable insights to every company, globally, by harne
View Job
Aws Data Engineer – Johannesburg – R900k To R1.1Mil Per Annum
South Africa: Data Engineer (Aws And Redshift) – Remote – R500 To R 700 Per Hour posted by E-Merge
A Group providing value conscious mass-marketing to customers in Southern Africa and leaders in providing innovative financial services have an opportunity for a Data Engineer who will take responsibility for driving, designing and building scalable ETL sy
View Job
Data Engineer (Aws And Redshift) – Remote – R500 To R 700 Per Hour
Pretoria: Aws Data Engineer – Pretoria- R500 To R700 Per Hour posted by E-Merge
A leader in African capital markets with a comprehensive suite of products, a vast global reach while and an acute positioning as a local bank with the expertise ability to leverage on local presence with regional and international expertise has a vacant p
View Job
Aws Data Engineer – Pretoria- R500 To R700 Per Hour
Johannesburg: Data Engineer (Aws And Redshift) – Johannesburg – Up To R1mil posted by E-Merge
A Group providing value conscious mass-marketing to customers in Southern Africa and leaders in providing innovative financial services have an opportunity for a data engineer who will take responsibility for driving, designing and building scalable ETL sy
View Job
Data Engineer (Aws And Redshift) – Johannesburg – Up To R1mil
