senior aws data engineer

Company: UsefulBI Corporation

Location: San Francisco Bay Area

Posted on: November 21

Role: Senior AWS Data Engineer

Location: San Fransisco (on-site)

Job Description:

Experience Required:

• Minimum 8+ years Experience in Data Engineering.
• Must have good knowledge and experience in Python with Py spark.
• Good knowledge and experience in SQL.
• Should have knowledge of using OpenAI Apis with Lang chain.
• Should be familiar with Hugging face model APIs.
• Mush have good Knowledge of Databricks.
• Must have good experience in AWS (Glue, Redshift, EMR)
• Typically requires relevant analysis work and domain-area work experience.
• Expert in the management, manipulation, and analysis of very large datasets.
• Proficient user of specialized data manipulation and analysis tools including R, Python, SAS, SQL, etc.
• Experience projects involving ML and/or DS.
• Experience in Machine Learning, Statistics, Data Science, or Neural Networks.
• Superior verbal and written communication skills, ability to convey rigorous mathematical concepts and considerations to non-experts.
• Good knowledge of scientific programming in scripting languages like Python/R/Matlab.

Key Responsibilities:

Create and maintain optimal data pipeline architecture.

Assemble large, complex data sets that meet functional / non-functional business requirements.

Identify, design, and implement internal process improvements: automating manual processes, optimizing data delivery, re-designing infrastructure for greater scalability, etc.

Build the infrastructure required for optimal extraction, transformation, and loading of data from a wide variety of data sources using SQL and AWS big data technologies.

Build analytics tools that utilize the data pipeline to provide actionable insights into customer acquisition, operational efficiency and other key business performance metrics.

Work with stakeholders including the Executive, Product, Data and Design teams to assist with data-related technical issues and support their data infrastructure needs.

Create data tools for analytics and data scientist team members that assist them in building and optimizing our product into an innovative industry leader.