Data Engineer – Databricks Engineer with Data Scientist experience

We are searching for a seasoned Databricks Engineer to pioneer Databricks solutions and having good experience in Machine Learning(ML).

• Building, and maintaining medallion architecture-based data pipelines in Databricks platforms.
• Handling raw data ingestion through Delta Live Tables while ensuring data quality and access latency.
• Utilizing knowledge of Snowflake to offer insight into data warehousing and management.
• Deep expertise in writing and reviewing production code in Python.
• Experience with designing scalable end-to-end Machine Learning/NLP systems.
• Experience on distributed, high throughput and low latency architecture.
• Understanding of NLP techniques around text cleaning/pre-processing, entity extraction, encoder-decoder architectures, similarity matching etc.
• Familiarity with Continuous Integration tools such as Jenkins.
• Work in conjunction with data science teams to implement complex statistical models.
• Ensuring data security and compliance in Databricks and Snowflake environments.
• Developing and deploying ETL job workflows with an emphasis on automation and scalability.
• Keeping up to date with industry best practices and emerging trends in Databricks technology.

• Bachelor’s degree in computer science, Information Technology, or a related field
• Experience in Databricks platform, data engineering, and using Delta Live Tables
• Experience with medallion architecture in a data lake environment
• Good to have knowledge on Snowflake data warehousing platform
• Strong understanding of ETL workflows and data pipeline automation.
• Skilled in SQL, Python, or Scala.
• Experience with LLM tooling and frameworks like LangChain, LlamaIndex etc.
• Knowledge of basic algorithms, object-oriented and functional design principles, and best-practice patterns
• Experience building software on top of containerization technology (Kubernetes, Docker etc.), and familiarity with frameworks/tools such as FastAPI, Uvicorn.
• Good problem-solving abilities, with an innovative approach to tackling complex challenges.
• Exceptional communication skills with proven ability to bridge communication gaps between technical and non-technical stakeholders.

• Savings and Investment plans
• Upskilling and career growth opportunities
• Health and Welfare Benefits
• Employee Assistance Program (EAP)
• Yearly Free Health campaigns
• Comprehensive Insurance benefits

At Chubb India, we are on an exciting journey of digital transformation driven by a commitment to engineering excellence and analytics.