Who are we?
Quanteam, founding entity of RAINBOW PARTNERS, is a consulting firm specialized in Banking and Financial Services. With more than 1200 consultants across 10 countries, we combine business and IT expertise to support leading institutions in transformation projects.
Our core areas:
➡ Quantitative Finance
➡ Risk & Compliance
➡ Operations & Finance
➡ Transformation & Organization
➡ Information Systems
We are looking for a Senior Big Data Engineer
CONTEXT
In this role, you will join an international and distributed team, working closely with counterparts based in Paris to design, develop, and maintain scalable Big Data solutions.
You will contribute to the development and optimization of data-processing pipelines, working with large volumes of data and modern Big Data technologies such as Spark, Python, Hadoop, Hive, Kafka, and SQL.
This is an excellent opportunity for an experienced Big Data Engineer who is looking to take ownership of complex technical challenges, contribute to performance improvements, and work within a collaborative and international environment.
MAIN MISSION
To design, develop, maintain, and optimize scalable Big Data solutions and data-processing pipelines, ensuring high levels of performance, reliability, automation, and alignment with technical and business requirements.
MAIN RESPONSIBILITIES
- Design, develop, and maintain scalable Big Data pipelines and applications.
- Build data-processing solutions using Spark, Python, Hadoop, Hive, and SQL.
- Process and transform large volumes of structured and unstructured data.
- Develop and maintain integrations with internal and external systems through APIs.
- Work with Kafka to support real-time or event-driven data-processing solutions.
- Analyse existing processes and identify opportunities for optimization and performance improvement.
- Troubleshoot and resolve data-processing, application, and infrastructure-related issues.
- Develop and optimize SQL, Hive, and Spark queries.
- Contribute to software development activities throughout the full development lifecycle.
- Participate in code reviews, testing, deployment, and release-management processes.
- Apply CI/CD and DevOps practices to improve development and deployment efficiency.
- Manage source code and development workflows using Git.
- Support build, release, and deployment activities using XL Release and XL Deploy.
- Schedule and monitor data-processing workflows using Control-M.
- Work within Linux environments and use relevant development and diagnostic tools.
- Produce and maintain technical documentation.
- Collaborate closely with technical teams and stakeholders based in Portugal and Paris.
- Share technical knowledge, best practices, and improvement recommendations with the wider team.
PROFILE & SKILLS
- Degree in Computer Science, Information Systems, Data Engineering, Software Engineering, or a related field.
- More than 6 years of professional experience in Data Engineering, Big Data, Software Development, or a similar technical role.
- Strong professional experience working with Big Data technologies and large-scale data environments.
- Solid background in software development and engineering best practices.
- Strong experience designing, developing, and maintaining data-processing pipelines.
- Experience working with distributed data-processing frameworks.
- Good understanding of data integration, data transformation, and data architecture concepts.
- Experience working with CI/CD pipelines and automated deployment processes.
- Familiarity with DevOps principles and practices.
- Experience identifying and resolving performance and scalability issues.
- Strong analytical and problem-solving skills.
- Autonomous, organised, and able to effectively manage priorities.
- Proactive and innovative approach to technical challenges.
- Positive and collaborative mindset.
- Ability to work effectively within an international and distributed team.
- Willingness to learn, collaborate, and share technical knowledge.
- Good command of English.
- French language skills are highly valued but not mandatory.
TECHNICAL SKILLS
- Apache Spark
- Python
- Hadoop
- Hive
- SQL
- Linux
- REST APIs
- Apache Kafka
- Software Development
- Data Pipelines
- Big Data Technologies
- CI/CD
- DevOps
- Git
- XL Release
- XL Deploy
- Control-M
WHAT WE OFFER
- Gross annual salary aligned with your experience and seniority.
- Permanent contract (CTI).
- Meal allowance.
- Health Insurance.
- Extra holiday: Christmas Eve or New Year's Eve.
- Access to Quanteam Academy for continuous learning and certifications.
- Elevate Program to support professional development and career growth.
- Career progression plan.
- Ongoing follow-up and technical support provided by a Consultant Expert.
Pagamento: 24 500,00€ - 60 500,00€ por ano
Localização do trabalho: Trabalho remoto híbrido em Lisboa
