Position Details: Sr Data Engineer - 931237F
Client is embracing Big Data technologies to enable data-driven decisions. We’re looking to expand our Hadoop Engineering team to keep pace. As a Sr. Bigdata Data science developer you will work with a variety of talented Client teammates and be a driving force for building solutions for Client Digital. You will be working on development projects related to consumer behavior, commerce, and web analytics.
- Design and implement distributed data processing pipelines using Spark, Hive, Sqoop, Python, and other tools and languages prevalent in the Hadoop ecosystem. Ability to design and implement end to end solution.
- Build utilities, user defined functions, and frameworks to better enable data flow patterns.
- Research, evaluate and utilize new technologies/tools/frameworks centered around Hadoop and other elements in the Big Data space.
- Build and incorporate automated unit tests, participate in integration testing efforts.
- Work with teams to resolving operational & performance issues
- Work with architecture/engineering leads and other teams to ensure quality solutions are implements, and engineering best practices are defined and adhered to.
- MS/BS degree in a computer science field or related discipline
- 6+ years’ experience in large-scale software development
- 2+ year experience in Hadoop
- 2+ year experience in Data Science
- Strong development skills around Hadoop, Spark, MapReduce, Hive
- Strong Java programming, Python, shell scripting, and SQL
- Good understanding of file formats including Parquet, Avro, JSON and others
- Good understanding of R, TensorFlow, SAS or similar
- Experience with performance/scalability tuning, algorithms and computational complexity
- Experience (at least familiarity) with data warehousing, dimensional modeling and ETL development
- Proven ability to work cross functional teams to deliver appropriate resolution
Nice to have:
- Experience with AWS components and services, particularly, EMR, S3, and Lambda
- Front end UI development experience, specifically Node JS or Angular JS
- Machine learning frameworks
- DATA WAREHOUSING
- APACHE HADOOP MAPREDUCE
- APACHE HADOOP SQOOP
- CONSUMER BEHAVIOR
- CUSTOMER BEHAVIOR
- DATA SCIENCE
- FRONT END
- INTEGRATION TESTING
- MACHINE LEARNING
- SERIAL ATTACHED SCSI
- SHELL SCRIPTING
- SOFTWARE DEVELOPMENT
- STRUCTURED SOFTWARE
- UNIT TESTS
- USER INTERFACE