Data Engineer

Greenwich, CT
New York, NY

We are representing an interesting Data Engineering position with a reputable hedge fund firm in Midtown Manhattan / Greenwich, CT.

 

Responsibilities

 

  • Build and maintain robust data pipelines that ingest TBs of data.
  • Build new Airflow DAGs that manage different operators or individual tasks.
  • Create logical and physical data models for big data to be stored in cloud data warehouses.
  • Optimize Spark and SQL queries.
  • Create new SQL tables for reports and dashboards.
  • Work with data warehouses: Snowflake, AWS Redshift, and other cloud data warehouses.
  • Work with data analysts on implementing dashboards and performing analysis.
  • Use Databricks Spark and data visualization tools to analyze business problems.
  • Generate reports and dashboards for business insights.
  • Write analytical queries to extract insights for large datasets (billions of rows, multi-TB in size).

 

Requirements

 

  • 3+ years of professional working experience
  • A degree or advanced degree in Computer Science, Engineering, Physics, Mathematics, Statistics, or Machine Learning, with a record of academic success.
  • Excellent development experience in Python, Apache Spark, and SQL.
  • Extensive experience with large-scale data processing solutions.
  • Extensive working knowledge of Apache Spark, Databricks and pandas.
  • Extensive working experience with AWS ecosystem.
  • Excellent computer science fundamentals and problem-solving skills.
  • Strong ability to work in Linux environment.
  • Experience in the fields of data warehousing, business intelligence and big data related technologies.
  • Experience in creating logical and physical data models.
  • Experience with one or more cloud MPP data warehouses including: Snowflake, AWS Redshift, Azure Data Warehouse (Synapse), or Google BigQuery.

 

 

 

 

Note: Qualified candidates will be contacted within 2 business days of application. If an applicant does not meet the above criteria, we will keep your resume on file for future opportunities and may contact you for further discussion.

Date Posted 6/10/2021
Salary $90,000-$150,000







    (2MB Max; allowed file formats: doc, docx, pdf, pptx, txt)



    characters left