Sparkland, a fully remote and fast-growing company in the algorithmic trading space, is seeking a skilled Data Engineer to join their technically driven team. This is a full-time position focused on building and maintaining high-volume data pipelines, optimizing infrastructure reliability, and preparing systems for future machine learning initiatives. You'll manage real-time and batch data streams, oversee Clickhouse performance, and work closely with cross-functional teams. Sparkland offers a flat structure, a culture of continuous learning, and the exciting opportunity to work on intellectually challenging problems with top-tier talent.
The ideal candidate has at least 5 years of experience in data engineering or backend infrastructure. Proficiency in Python and SQL is required, along with hands-on experience in Clickhouse, Kafka, and workflow orchestration tools like Airflow or Argo. Familiarity with CI/CD pipelines and monitoring tools (e.g., Grafana) is also important. Bonus points for candidates experienced with AWS, Kubernetes, and ML infrastructure.
This is a unique opportunity to join a high-impact team where your work directly influences advanced trading systems. The role is ideal for those who thrive in a data-centric, innovative, and collaborative remote environment.
We are a team of highly-driven individuals who are passionate about technology, algorithmic trading, and solving intellectually challenging problems. Being a part of Sparkland means you get to work with some of the brightest people in one of the world’s fastest-growing and most exciting industries. We are fully remote and have a flat corporate structure that values open-mindedness, entrepreneurial spirit, commitment to excellence, and continuous learning.
The Role
Responsibilities
- Design and maintain robust data pipelines to support real-time and batch processing.
- Manage and optimize our Clickhouse data warehouse, including cluster performance and schema tuning.
- Ensure data quality, observability, and governance across critical pipelines.
- Collaborate with backend engineers, trading teams, and data stakeholders to align on data requirements.
- Support internal initiatives by building tooling and monitoring for business and technical metrics.
- Take ownership of scheduling and workflow orchestration (Argo, Airflow, etc.) and contribute to CI/CD automation.
Required Skills & Experience
- At least 5 years of professional experience in data engineering or backend infrastructure.
- Proficiency in Python, including object-oriented programming and testing.
- Solid experience with SQL: complex joins, window functions, and performance optimization.
- Hands-on experience with Clickhouse (especially the MergeTree engine family) or similar columnar DBs.
- Familiarity with workflow schedulers (e.g., Argo Workflows, Airflow, or Kubeflow).
- Understanding of Kafka architecture (topics, partitions, producers, consumers).
- Comfortable with CI/CD pipelines (GitLab CI, ArgoCD, GitHub Actions).
- Experience with monitoring and BI tools such as Grafana for technical/business dashboards.
Bonus Points
- Experience with AWS services (S3, EKS, RDS).
- Familiarity with Kubernetes and Helm for deployment and scaling.
- Exposure to data quality/observability frameworks.
- Experience supporting ML infrastructure (e.g., feature pipelines, training data workflows).
Similar Jobs
