Data Engineer - Open Application
Core
Design, build, and maintain scalable data pipelines, warehouses, and streaming solutions for U.S. clients using cloud technologies.
Role type
Senior Data Engineer
Builds
Data ingestion pipelines, ETL processes, data warehouses, and real-time streaming data services
Domain
Cloud data engineering, big data processing, and data warehousing
Deliverable
production ML models | product features | dashboards & analysis | infrastructure
Required skills
Python, PySpark, SQL, Kafka, AWS/GCP/Azure, Snowflake/Redshift, Hadoop, Flink, Airflow, Pandas, REST APIs, NoSQL databases (Cassandra, MongoDB, HBase)
Preferred skills
Lambda Architecture, SolR/ElasticSearch, Jupyter Notebook to production conversion, BI tools (Looker, Power BI, Tableau), Data Lake architecture
Technologies
Python, PySpark, Kafka, Kinesis, Flume, SQS, SNS, RabbitMQ, HBase, Cassandra, MongoDB, S3, Spark, Flink, Hadoop, Talend, Informatica, SSIS, Snowflake, Redshift, Hive, AWS, GCP, Azure, Looker, Power BI, Tableau, Airflow
Responsibilities
Consume data from various sources including REST APIs; Design and implement scalable data models and distributed data stores; Build data ingestion and streaming pipelines; Develop ETL solutions using tools like Talend, Informatica, or SSIS; Implement data warehouse solutions providing near real-time data; Ensure data model scalability and high performance; Work with public cloud PaaS platforms
Seniority
Senior, hands-on IC
