Core Responsibilities
- Develop data integration pipelines using Informatica IICS, and Python, ensuring rigorous data quality controls and exception handling.
- Design and optimize Snowflake-based Medallion architectures, leveraging advanced stored procedures, window functions, and dynamic schema evolution strategies, including SCD Type 2.
- Engineer effective ingestion frameworks for semi-structured data (e.g., JSON), enabling dynamic structure handling and late-arriving dimension management.
- Establish automated CI/CD workflows for database and data pipeline deployments, incorporating version control and Snowflake dynamic table capabilities.
- Model manufacturing domain datasets, such as inventory, order management, and sales, ensuring seamless application integration and data replication.
- Drive performance and scalability across data lake environments through partitioning, parallelism, and efficient file format utilization (Parquet, JSON, CSV).