Software Engineer

Required Qualifications

• Master's degree in Information Systems, Computer Engineering, Computer Science, Information Technology, or related field of study plus 2 years of experience in the job offered or as Software Engineer, Senior Software Engineer, or related occupation.
• Alternatively, a Bachelor's degree in Information Systems, Computer Engineering, Computer Science, Information Technology, or related field of study plus 4 years of experience in the job offered or as Software Engineer, Senior Software Engineer, or related occupation.
• Designing and implementing large data pipelines for ETL.
• Utilizing Python to design ETL pipelines that ingest data from multiple sources, applying business-rule transformations, and loading data into Salesforce to integrate databases, APIs, and cloud platforms to build unit tests and validation scripts.
• Utilizing Shell scripts to automate file operations, monitor system resources, log ETL job statuses, and send alerts in case of failures.
• Configuring environments, managing permissions, and handling pre- and post-processing steps in ETL workflows.
• Utilizing Talend Studio for maintaining, auditing, and accurately migrating legacy ETL pipelines into PySpark-based architectures ensuring data integrity and consistency during the decommissioning and modernization of enterprise-scale data systems.
• Utilizing Spark framework to architect scalable, distributed data architectures that execute high-performance transformations and aggregations on massive datasets across both real-time streaming and batch processing environments.
• Designing data models (Dimensional Modeling, Relational Modeling), data integration, data security, and metadata management to support analytics and business intelligence.
• Working on data loads and extracts from Salesforce objects using SOQL and ETL processes to ensure effective data handling.
• Utilizing JIRA to track and manage software development tasks.
• Ensuring software quality through comprehensive testing including functional, manual, performance, regression, smoke, system integration, unit, and user acceptance testing.
• Using Waterfall and Agile SDLC methodologies to provide a structured approach to project execution.
• Developing and optimizing SQL and PySpark queries across multiple data sources; utilizing advanced window functions, recursive CTEs, and multi-stage transformations to efficiently process large-scale datasets and ensure high-performance data retrieval for business-critical intelligence.
• Implementing database solutions using SQL on MS SQL Server, Oracle 11g, PostgreSQL, and Cassandra to enhance data management and access.
• Building robust applications and automation scripts using Python to improve software functionality and operational efficiency including extracting and analyzing data from REST APIs.
• Maintaining data warehousing solutions and data marts for efficient storage and retrieval using Erwin Data Modeler, Talend Open Studio, Python, and Cassandra.
• Incorporating data governance principles including data quality, security, and compliance into design specifications using Python Libraries.
• Leveraging command line interfaces including IntelliJ CLI and Putty for automated development tasks, efficient cloud management, and secure Linux server access.
• Automating data ingestion and processing pipelines in cloud.
• Cloud services including AWS for migrating on-premise applications to AWS cloud.
• Leveraging AWS for scalable data storage as a processing solution, and managing large datasets.
• Leveraging command line interfaces including AWS CLI for automated development tasks, efficient cloud management, and secure Linux server access.
• Handling formats including Parquet and JSON.
• Managing source code using GIT for version control and CI/CD practices.