Big Data Engineer – Distributed Systems, Cloud Pipelines & Advanced Analytics

About arenaflex: Where Innovation Meets Opportunity

At arenaflex, technology is more than a tool — it is the engine that drives exceptional customer experiences, smarter business decisions, and next-generation financial solutions. We are a forward-thinking organization committed to building robust digital ecosystems that empower millions of customers to take control of their financial lives. Our engineering culture is rooted in collaboration, curiosity, and continuous learning, making arenaflex a destination for technologists who want to make a tangible impact at scale.

As a member of our diverse and dynamic technology division, you will engineer, code, and deploy software solutions that form a foundational part of our customers' digital journeys. At arenaflex, you will work alongside highly skilled engineers in an open, supportive, and inclusive environment where your voice is valued, and you have the autonomy to choose the best technologies to solve complex challenges. We actively encourage our engineers to contribute to the broader developer community through open-source initiatives, and we provide dedicated time and resources to invest in your professional development so your skills remain sharp, relevant, and future-ready. Discover your place in technology with #Teamarenaflex.

Position Overview

We are seeking a passionate and high-performing Big Data Engineer to join our fast-paced, entrepreneurial engineering team. In this role, you will design, build, and optimize large-scale batch and real-time data processing solutions using cutting-edge technologies such as Hadoop, Spark, Kafka, Python, and SQL. You will play a key role in enabling sophisticated analytics, machine learning workflows, and business intelligence capabilities across multiple lines of business at arenaflex.

Reporting to an Engineering Manager, you will be part of a product engineering, delivery, and integration team focused on building resilient, scalable, and high-performance data platforms. This is an exceptional opportunity for an engineer who thrives in an Agile environment and is excited about leveraging data as a strategic asset.

What You Will Do: Key Responsibilities

Software Development & Engineering Excellence

• Design, develop, test, and maintain high-quality software components and data pipelines using Core Java, Python, or Scala.
• Contribute as a strong individual contributor within an Agile team, adhering to consistent development practices including the use of common tools, reusable components, and thorough documentation.
• Participate actively in code reviews, pair programming sessions, and automated testing to ensure code quality and maintainability.
• Troubleshoot and debug complex software components, identifying and remediating code defects efficiently.
• Enable seamless deployment, support, and monitoring of software solutions across test, integration, and production environments.
• Automate solutions across test and production environments to improve operational efficiency.
• Implement auto-scaling mechanisms for applications based on demand projections and usage patterns.

Data Engineering & Platform Development

• Build and optimize advanced data pipelines for both batch and real-time data processing at scale.
• Develop and maintain solutions using Hadoop, Spark (Core, SQL, and PySpark), Hive, and related big data technologies.
• Write and tune complex SQL queries (including HiveQL and PySpark DataFrames), optimizing joins and transformations for large datasets.
• Design and architect data warehousing solutions aligned with strong data modeling principles.
• Apply visualization and analytical techniques to rapidly prototype and validate concepts.
• Identify opportunities to adopt innovative technologies that improve platform performance and developer productivity.

Leadership & Collaboration

• Lead a small team of engineers, mentoring and motivating them to achieve their full potential.
• Influence peers and stakeholders with innovative ideas, improvements, and a willingness to challenge the status quo.
• Drive continuous improvement in code structure, quality, and engineering best practices.
• Collaborate with product managers, data scientists, and business stakeholders to understand requirements and translate them into scalable technical solutions.

Essential Qualifications

• Bachelor's degree in Computer Science, Engineering, Data Systems, or a related technical field OR Master's degree in Computer Applications or equivalent.
• 3+ years of hands-on software development experience, including experience leading engineering teams and Scrum teams.
• 3+ years of practical experience working with MapReduce, Hive, and Spark (Core, SQL, and PySpark).
• Strong understanding of data warehousing concepts and dimensional modeling.
• Expertise in distributed computing environments and parallel processing frameworks.
• Hands-on programming experience in Core Java, Python, or Scala.
• Deep knowledge of Hadoop and Spark architecture, including working principles and performance tuning.
• Proficiency in writing and understanding complex SQL (Hive/PySpark DataFrames) and optimizing joins for large-scale data processing.
• Solid experience with UNIX shell scripting for automation and operational tasks.
• Demonstrated ability to design and develop advanced data pipelines for both batch and streaming workloads.
• Experience in the analysis, design, development, testing, and deployment of enterprise-scale system applications.
• Proven ability to create and document technical and functional specifications, and to analyze software and system processing flows.

Preferred Qualifications

• Knowledge of cloud platforms such as GCP or AWS, with experience building microservices and scalable cloud-native solutions.
• 3+ years of experience designing and building solutions using Kafka streams or message queuing systems.
• Experience with version control systems like GitHub or Bitbucket, and with CI/CD pipelines.
• Familiarity with NoSQL databases such as HBase, Couchbase, or MongoDB.
• Exposure to financial reporting environments or financial services data ecosystems.
• Excellent technical and analytical problem-solving skills.
• Strong communication and interpersonal abilities.
• Exceptional project management and organizational skills.

Skills and Competencies for Success

• Technical Depth: Strong foundation in distributed systems, big data ecosystems, and modern programming languages.
• Problem Solving: Ability to break down complex problems, identify root causes, and deliver elegant, scalable solutions.
• Agile Mindset: Comfortable working in fast-paced, iterative environments with shifting priorities.
• Innovation: Naturally curious, willing to experiment with emerging technologies, and driven to find better ways of doing things.
• Collaboration: A team player who thrives in cross-functional settings and values diverse perspectives.
• Communication: Able to articulate complex technical concepts clearly to both technical and non-technical audiences.
• Ownership: Takes accountability for deliverables and drives initiatives from concept through completion.

Career Growth and Learning Opportunities at arenaflex

At arenaflex, we believe that investing in our people is the key to building world-class technology. As a Big Data Engineer, you will have access to:

• Dedicated time and a generous budget for professional development, certifications, and conference attendance.
• Mentorship programs pairing you with senior engineers and technical leaders.
• Internal mobility opportunities across product lines, platforms, and geographies.
• Exposure to cutting-edge projects involving machine learning, real-time analytics, and cloud-native architectures.
• A clear career progression framework that rewards both technical excellence and leadership impact.
• Active open-source contribution culture, with company support for community engagement.

Work Environment and Culture

arenaflex fosters a culture built on trust, transparency, and teamwork. We embrace remote and hybrid work models that give our engineers the flexibility to do their best work from anywhere. Our inclusive environment celebrates diversity of thought, background, and experience — because we know that great ideas come from great teams. Whether you are collaborating in a virtual stand-up, contributing to an open-source project, or brainstorming the next big architectural improvement, you will be supported by colleagues who are genuinely invested in your success.

Compensation and Benefits

arenaflex offers a competitive compensation package designed to attract and retain top engineering talent. This position includes a base pay rate of $28/hour, along with a comprehensive benefits package that typically includes:

• Health, dental, and vision insurance coverage.
• Retirement savings plans with company matching contributions.
• Generous paid time off, holidays, and parental leave.
• Wellness programs and mental health support resources.
• Tuition reimbursement and continuous learning stipends.
• Employee resource groups and community engagement programs.

Note: An offer of employment with arenaflex is contingent upon the successful completion of a background verification check, subject to applicable laws and regulations.

Join arenaflex and Shape the Future of Data

If you are an engineer who is passionate about building scalable data platforms, loves solving complex technical challenges, and wants to work with some of the brightest minds in the industry, arenaflex is the place for you. Bring your curiosity, your craftsmanship, and your drive — and help us engineer the future of intelligent, data-driven solutions. Apply today and take the next step in your career with arenaflex.