Data Solution Architect Job Description:Activities:Design and develop scalable data pipelines from multiple sources that includes batch data and event data.Design, develop, test, deploy and maintain data platform (storage, API, pipelines, catalogues).Work with various types of data - JSON, XML, Parquet, Avro, Delimited files.Implement strategy on data reliability, efficiency, performance, and quality.Perform peer design reviews, code reviews, pair programming and functional testing to ensure quality releases.Deploy data pipelines following DevOps principles and incorporating automated testing.Automate monitoring of data pipelines and data integrity.Showcase designs for new features and get approvals from the stakeholders.Support team members with design, development and testing when blocked.Must-have:Knowledge and working experience of modern data platform architectures including event-based architecture.Experience in configuration-based design and development of large data projects.Extensive and proven experience with Python, SQL, Shell Scripting, API, and processing large datasets.Excellent understanding of data engineering concepts and data modelling.Experience working with AWS environment including S3, EC2, Redshift, Aurora, SQS, SNS, IAM etc.Experience with streaming data platforms like Kafka.Experience with CI/CD Methodologies and Building Infrastructure as a Code.Experience with container-based development & deployment such as Docker.Adhere to data governance standards and principles.Nice-to-have skills:Experience with Kubernetes and Argo Workflows.Experience with AWS Glue, Lambda, EMR, DMS, CloudWatch, KMS, Secrets Manager.Experience working with microservices architecture.Job description:Understand the existing data architecture of the data platform.Design, develop, test and deploy features for generic data pipeline using Kubernetes, Argo Workflow, Python, Aurora PostgreSQL, Redshift, S3, Kafka.Work in a fast-paced and agile environment with tight deadlines.Work in a team environment with efficient communication and collaboration.Lead by empowering team members to deliver quality results on time.About the companyVirtusa Corporation is an American information technology services company founded in 1996 in Sri Lanka and has its headquarters in Southborough, Massachusetts, United States.
#J-18808-Ljbffr