Position Details: Lead Data Engineer - 971698Z
Looking for a lead engineer. 2+ years leading an engineering team along with the technicalities abilities.
Become a Part of the Client Team
Client does more than outfit the world’s best athletes. It is a place to explore potential, obliterate boundaries and push out the edges of what can be. The company looks for people who can grow, think, dream and create. Its culture thrives by embracing diversity and rewarding imagination. The brand seeks achievers, leaders and visionaries. At client, it’s about each person bringing skills and passion to a challenging and constantly evolving game.
Do you have a passion for digital technology, innovation and problem solving? Are you curious about how to turn billions of events and signals into meaningful information that not only provides insights into the present but also help predict the future? Are you interested in applying Data Engineering, Data Streaming and Big Data Technology to help deliver personalized experiences? If so, come join the talented team of engineers that are a driving force behind data engineering solutions at client on the Consumer Services Data Engineering Team.
The Consumer Services Data Engineering Team is building a new data warehouse and needs a talented lead engineer to help drive the team across milestones and support our product roadmap.
- MS/BS in Computer Science, or related technical discipline
- 5+ years of industry experience and 5+ years of relevant big data/ETL data warehouse experience building data pipelines
- 2+ years leading a team of data engineers – ie mentor, improve processes, peer training, long term best practices, coding standards
- 5+ year experience in Python and Snowflake. Strong programming experience in Python
- Professionally worked with APIs to extract data for data pipelines
- Extensive experience working with Hadoop and related processing frameworks such as Spark, Hive, Sqoop, etc.
- Ability to architect, design and implement solutions with AWS Virtual Private Cloud, EC2, AWS Data Pipeline, AWS Cloud Formation, Auto Scaling, AWS Simple Storage Service, EMR and other AWS products, HIVE, Athena
- Troubleshooting production issues and performing On-Call duties, at times.
- Working knowledge with workflow orchestration tools like Apache Airflow
- Hands on experience with performance and scalability tuning
- Professional experience in Agile/Scrum application development using JIRA
- Experience working in a public cloud environment, particularly AWS
- Professional experience with source control, merging strategies and coding standards, specifically Bitbucket/Git
- Professional experience in data design and modeling specifically working with ErWin
- Demonstrated experience developing in a continuous integration environment using tools like Jenkins, Bamboo, or TeamCity CI Frameworks.
- Demonstrated ability to maintain the build and deployment process through the use of build integration tools
- Working experience and communicating with business stakeholders and architects
- Demonstrated experience implementing security around sensitive data
- Experience designing instrumentation into code and using and integrating with software and logging analysis tools like log4Python, Signal FX and/or Splunk
Required Soft Skills:
- Desire to lead collaboratively with your teammates to come up with the best solution to a problem
- Demonstrated experience and ability to deliver results on multiple projects in a fast-paced, agile environment
- Excellent problem-solving and interpersonal communication skills
- Strong desire to learn and share knowledge with others
- Passionate about data and striving for excellence
- Desire to learn and understand the business and communicate with business stakeholders to accomplish business rules transformations and data validation while coding
- Understands the importance of data security and privacy
Nice to Have:
* Call center data at a global level
- Lead, design and implement features in collaboration with team engineers, product owners, data analysts, and business partners using Agile / Scrum methodology
- Contribute to overall architecture, frameworks and patterns for processing and storing large data volumes
- Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the EMR ecosystem
- Build utilities, user defined functions, and frameworks to better enable data flow patterns
- Research, evaluate and utilize new technologies/tools/frameworks centered around high-volume data processing
- Define and apply appropriate data acquisition and consumption strategies for given technical scenarios
- Build, incorporate, and drive teams to write automated unit tests and integration scripts using tools like the Python test unit framework
- Work with architecture/engineering leads and other teams to ensure quality solutions are implemented, and engineering best practices are defined and adhered to
- Work across teams to resolve operational and performance issues and to deliver data for reporting in Tableau
- Implement security control around sensitive data