12 month contract Azure Data Engineer (Azure Databricks & PySpark)
Job Description
We’re hiring a talented Azure Data Engineer enthusiast to work in our platform to help
ensure that our data quality is flawless. As a company, we have millions of new data
points every day that come into our system. You will be working with a passionate team
of engineers to solve challenging problems and ensure that we can deliver the best data
to our customers, on-time. You will be using the latest cloud data lake technology to
build robust and reliable data pipelines.
Job Responsibilities
● Develop expertise in the different upstream data stores and systems across the
company.
● Design, develop and maintain data integration pipelines for the organization growing
data sets and product offerings.
● Build unit testing and QA plans for data processes.
● Build data validation testing frameworks to ensure high data quality and integrity.
● Write and maintain documentation on data processes.
● Developing and maintaining data models and schemas.
● Strong analytical experience with database in writing complex queries, query
optimization, debugging, user defined functions, views, indexes etc.
● Write code that adheres to coding standards, procedures, and techniques. Maintain
the integrity of existing program logic according to specifications.
● Actively participate in the code review process to ensure development work adheres
to standards and specifications (including peer review and code review external to
team).
● Respond to all inquiries and issues in a timely manner as developed code/program
moves through the testing process.
● Participate in scrum, sprints, and backlog grooming meetings.
● Evaluate interrelationships between applications to determine whether a change in
one part of a project would impact or cause undesirable results in related applications
and design for effective interfaces between interrelated applications.
● Improve the health of system assets by identifying enhancements to improve
performance through tuning and monitoring, reliability, and resource consumption.
● Evaluate and troubleshoot root-cause analysis for production issues and
system failures; determine corrective action(s) and propose improvements to
prevent their recurrence.
● Maintain up-to-date business domain knowledge and technical skills in software
development technologies and methodologies.
● Provide input in the selection, implementation and use of development tools and best
practices.
Technical:
● BS or MS in Computer Science or equivalent experience.
● 4 + years of experience in Databricks/Apache Spark with Azure data storage
solutions handling large datasets.
● Expert in SQL and Spark SQL, including advanced analytical queries.
● Proficiency in Python (data structures, algorithms, object-oriented programming,
using API’s) and familiarity with PySpark.
● Experience working with Databricks Delta tables, unity catalog, Dataframe API, read
and write from/to various data sources and data formats.
● Experience with both Batch and streaming data pipeline.
● Knowledge of Azure Data Factory, Azure Data Lake, Azure SQL DW, Azure SQL is a
plus.
Nice to haves
● Understanding of PostgreSQL and MS SQL
● Experience working in a fast-paced environment.
● Experience in an Agile software development environment.
● Ability to work with large datasets and perform data analysis
● Worked on migration project to build Unified data platform.
● Experience working with JIRA, Azure DevOps CI/CD pipelines.
MediaRadarJob Description Job Description Job Title: Garden Center Merchandiser (HD) Locations: NH, ME and VT Position: Seasonal Hourly non-exempt Reports to: Merchandising Regional Manager (HD)
...service standards by providing customers with prompt, quality coffee selection, purchase, and preparation. Demonstrate the company's... ...values of respect, honesty, integrity, diversity, inclusion and safety. Coffee, Barista, Restaurant, Grocery Kroger Family of Companies
Job Description Allied Universal, North Americas leading security and facility services company, provides rewarding careers that give you a sense of purpose. While working in a dynamic, diverse and inclusive workplace, you will be part of a team that fuels a culture...
...(SIEM) analyst for the AvMC contract. The candidate should have a background in cybersecurity compliance monitoring tools such as Elastic and Splunk, as well as extensive Linux system administration knowledge. The candidate will provide support and administration for...
We have many opportunities available on our other career site pages. Click here to link to our careers page! At Kay, we know that love is unstoppable. Which is why we're devoted to helping customers Celebrate Life and Express Love in ways every bit as dynamic as they ...