JOB SUMMARY:The Data Engineer will sift through structured and unstructured data from diverse sources to design and build solutions to support BI, data science and operational needs. Will work closely with business leaders, product managers and data scientists to understand the potential business value of data sets and ultimately build data processing pipelines around those data sources. In addition to development around new data sources, the candidate will deliver provisioning capabilities to end users as well as core frameworks and infrastructure that supports a rapidly growing number of use cases.
REPORTS TO:Director
JOB DUTIES/RESPONSIBILITIES:
- Inform, influence, support, and execute our product decisions and product launches
- Manage data warehouse plans for a product or a group of products.
- Interface with engineers, product managers and product analysts to understand data needs.
- Partner with Product and Engineering teams to solve problems and identify trends and opportunities.
- Build data expertise and own data quality for allocated areas of ownership.
- Design, build and launch new data extraction, transformation and loading processes in production. Support existing processes running in production.
- Define and manage SLA for all data sets in allocated areas of ownership.
- Work with data infrastructure to triage infra issues and drive to resolution.
- Build platform infrastructure and interfaces between and within GCP, AWS, and other big data platforms in support of BI and operational needs
- Provide guidance and support to big data ETL engineer on performance and facilitate ETL performance tuning
- Performs other duties as assigned
JOB REQUIREMENTS:
- Bachelors Degree
- 5-10 years of related experience
- Valid Driver License for the State of employment
- 18 years of age or older (except some locations which may allow for 16 and 17 year old individuals)
REQUIRED SKILLS:
- BS/BA in Technical Field, Computer Science or Mathematics.
- Experience with DBT (data build tools) required
- Experience and focus on operations data stores to build out fundamental data models
- 4+ years experience in the data warehouse space.
- 4+ years experience in custom ETL design, implementation and maintenance.
- 4+ years experience working with either a Map Reduce or an MPP system.
- 4+ years experience with schema design and dimensional data modeling.
- 4+ years experience in writing SQL/ NoSQL statements.
- 4+ years experience using Python or Java
- Ability to analyze data to identify deliverables, gaps and inconsistencies.
- Communication skills including the ability to identify and communicate data driven insights.
- Ability in managing and communicating data warehouse plans to internal clients
- Experience or demonstrated interest in big data technologies
- Experience with Hadoop or other big data platforms
- Experience in coding
- Strong communication and presentation skills