Job Description

Position: Data Engineer

Location: Reston, VA(100% remote)

Duration: 1+ year contract

Responsibilities :

• Design, build, and optimize data pipelines using Databricks Runtime, Delta Lake, Autoloader, and Structured Streaming.
• Implement secure and governed data access using Unity Catalog, workspace controls, and audit configurations.
• Integrate and manage data from diverse structured and unstructured sources (e.g., REST APIs, SQL, cloud storage).
• Develop notebook-based workflows and manage jobs in Databricks Workflows and Jobs API.
• Implement best practices for performance, scalability, and cost optimization on Databricks.
• Collaborate with data scientists, analysts, and stakeholders to deliver clean, curated, and reliable datasets.
• Support CI/CD and infrastructure-as-code approaches for Databricks jobs and configurations.

Qualifications :

• 5 7 years of experience in data engineering or software development.
• 3+ years hands-on experience with Databricks, Delta Lake, and Spark (PySpark/PySQL/Python).
• Experience implementing and managing Unity Catalog for multi-tenant environments.
• Strong knowledge of ETL/ELT, SQL, and big data performance optimization techniques.
• Experience with GitHub, CI/CD, and deployment automation for Databricks assets.
• Familiarity with AWS, Azure, or GCP cloud storage and networking.
• Excellent problem-solving skills and ability to work in Agile teams.

Certification-

• Databricks Certified Data Engineer-Professional

Job Tags

Similar Jobs