Azure Data Engineer

Job Overview

Our client in the Bedminster area of NJ has an excellent opportunity for an Azure Data Engineer!

 

Responsibilities:

  • Build and manage a new single version of truth data architecture leveraging the Microsoft Azure data stack.
  • Assist in the creation of end-to-end data pipelines from internal sources to the new Data Lakehouse.
  • Collaborate and work closely with DevOps, application services, infrastructure, and product teams to identify and build new data requirements.
  • Create and manage documentation on data dictionary, data architecture and meta data.
  • Establish and implement best practices in database design, data quality and data architecture.
  • Gather information from business stakeholders and convert into technical requirements.
  • Provide consultation regarding data platform terminology, concepts, and practices.
  • Evaluate new industry data engineering technologies and practices.

 

Requirements:

  • 2 years minimum experience with Azure (DP-200/DP-201 or DP-203 certification is a plus).
  • 3+ years of experience working with both relational: Oracle, SQL Server, Postgres, MySQL, or equivalent and NoSQL databases like MongoDB, DocumentDB, Cassandra or equivalent.
  • 3+ years of experience with ETL/ELT: using any of the following or equivalent: Informatica, Matillion, CloverDx, SSIS, Apache NiFi, Airflow, Spark, Talend, Kaafka, Oracle Integrator, etc.
  • 3+ years of experience in one or more of the following of the following: Python, R, Scala, Java, VB.NET, C#, C++.
  • 2+ years of experience in creating APIs / “data-as-a-service” layer for data ingestion.
  • 2+ years of experience working on analytics platforms: Snowflake, Amazon Redshift, Azure Synapse, Vertica, Greenplum or similar.
  • 2+ years of AWS / Azure experience working on one or more of the cloud native technologies like S3, AWS RDS, DynamoDB, Glue, Lambda, Kinesis, Azure Blob, Athena, Azure functions, Cosmos DB, Azure data factory.
  • 2+ years of experience working on reporting tools: Cognos, Tableau, Qlikview, SSRS, PowerBI a plus.
  • Experience architecting, developing, and deploying (both hybrid cloud & full cloud) data architectures.
  • Hands on experience with Azure SQL, Data Lake, Blob, Synapse, Functions and Data Factory.
  • Proficient in creating Azure Data Factory pipelines for ETL processing and mapping data flows.
  • Knowledge of Databricks, Spark, Data Frames, Apache Parquet.
  • Experience working with one or more of the data modeling tools: ERWin, ER Studio, etc.
  • Knowledge of any of the data warehouse methodologies: Dimensional (Kimball), Data Vault, Inmom.
  • Experience creating a data lake using S3, Azure Blob, Hadoop, or similar platforms.
  • Working knowledge of containers: Dockers, Kubernetis.
  • Knowledge of graph databases: Neo4J, AWS Neptune.
  • Experience with one or more of the following: Redis, Memcache, Elasticsearch.
  • Good understanding of database performance tuning, optimization, scalability, security, latency, and reliability.
  • Knowledge of agile methodologies using CI/CD techniques in a Dev/Ops environment using Azure DevOps and Git.
  • Ability to lead proofs-of-concepts and effectively transition and scale those concepts into production.
Job Detail
Shortlist Never pay anyone for job application test or interview.