Data Architect / Data Engineering Lead 

Description:

Must have a deep understanding of data and its usage, storage, security, governance along with engineering. Proven ability to deliver results with a high sense of ownership

Responsibilities:

• Own InCred's data platform including data pipelines, the business intelligence portal
• Create long term roadmap to enable more data sources, building of ML models and their usage in workflows
• Communicate the data platform roadmap and resulting tradeoffs/benefits to business stakeholders
• Be an innovator in technical architecture, along with demonstrated ability to take accountability for achieving results
• Stay on top of latest developments in tech stack and propose potential upgrades to existing systems
• Strong analytical skills.

Mandatory Skills:

·

• 5+ years of experience in Data Engineering - Designing databases, building data pipelines, and maintaining data governance protocols in cloud platforms-Mandatory
• 5+ year experience in PySpark / Scala (preferably PySpark) Mandatory
• 5+ year experience in ETL Mandatory
• 2+ year experience in Kafka/Spark streaming/AWS Kinesis or any other streaming counterpart-Optional
• Experience in SQL - PostgreSQl / MySQL / Aurora and NoSQL DynamoDB / DocumentDB / MongoDB- SQL is needed/NoSQL is optional
• Experience working with various file formats Mandatory
• Experience working with ETL tools like EMR, Glue or Databricks-Any other tools work
• Experience working with Presto / Athena- Optional

Bonus skills:

• Strong understanding of AWS Services - S3, Redshift, Lambda, Athena, SNS, DMS, security protocols
• Experience working on Graph database
• Experience with Delta lakes