Scala/AWS/Spark Backend Engineer (Remote)

Published date Posted on WeWorkRemotely on Nov 19, 2021 (13 d ago)

Headquarters: New York
URL: https://narrative.io


What You Will Do

We currently have a marketplace that connects buyers and sellers of adtech data, and we are building on this success to make it possible to transact any other kinds of data.


As a result, here are the kinds of projects you will likely work on in the foreseeable future:

  • Make our core systems generic/agnostic to the actual data types that are being ingested, transacted and delivered. For instance, we are building Iceberg-backed data puddles, schema/validation tools, making the transaction process more generic, etc.
  • Improve the monitoring and reliability of our systems with the ever-increasing amount of data being handled (over 500GB per day)
  • Extract abstractions, modularize the code and improve the tests.
  • Enhance the API we expose to partners and increasingly rely on it for internal use, in order to "eat our own dogfood".

Technical Stack


In a nutshell, our technical stack looks like:

  • Frontend: Vuejs, Sass, Pug and Functional JavaScript
  • Backend: Big Data, scala, AWS, spark, cats, cats-effect, http4s and doobie
  • Ops: EC2, Fargate, Lambda, Terraform, EMR, DynamoDB, S3, RDS, Step Functions, Jenkins, and Datadog

Read more about our technical stack


The Ideal Candidate

We are not looking for a 100% fit on all the technology buzzwords, but we are looking for someone with strong personal and technical skills who is eager to pick up new technologies as necessary. We are obviously going to expect much more from a senior candidate than we would from a junior one.


The ideal candidate should:

  • Have experience in a typed functional language such as Scala, F# or Haskell, or significant experience in their non-functional equivalents (Java, C#) with an interest in Scala and Functional Programming.
  • Have experience working with non-trivial quantities of data. As of this writing, our ingestion pipelines are handling something on the order of 500GB .snappy.parquet files per day. Prior work with Spark would be ideal, but experience with similar MapReduce-based technologies would also be helpful.
  • Have experience operating in a cloud environment like Amazon Web Services, Google Compute Engine, or similar.
  • Be able to work across all aspects of back end systems, from application code to SQL to systems administration.
  • Not be afraid of contributing to the entire stack (from the UI to Devops) when the need arises.
  • Have strong experience using a version management system and continuous integration (CI) development process. We use Git/Github for version management and Gitlab for our CI pipelines.
  • Have the ability to lead the creation of architectural and design documents,
    collect requirements as well as feedback from the development and product teams and evalute new technologies as needed.
  • Be able to transform product designs into coherent, working and robust code solutions.
  • Communicate potential technical issues to relevant teams and adapt to changing requirements.
  • Be able to interface with technical and non-technical team members in order to bring business ideas to fruition.
  • Be mindful of the compromises that need to be done to be reactive on the business side while keeping the systems manageable in the long run.
  • Live/work within +/- 3 hours of EST

The Team (Remote US and Canada)


We are a small, remote-first team looking for great developers who want to jump in and take major systems and user-facing features from design to launch.



To apply: https://weworkremotely.com/remote-jobs/narrative-i-o-scala-aws-spark-backend-engineer-remote


Let us know

Help us maintain the quality of jobs posted on RemoteTechJobs and let us know if:

Loading...
Success
Error on reporting

Related jobs

THE CHALLENGEFinancial Engineering is incredibly important for any global marketplace. The success of the Eventbrite as well as our creators depends on building and maintaining a best-in-class financial ecosystem that produces, monitors, and validates Eventbrite’s financial

Avalara, Inc., (www.Avalara.com), is the leading provider of cloud-based software that delivers a broad array of compliance solutions related to sales tax and other transactional taxes.What is it like to work at Avalara? Come find out! We are committed to the following success tr

UNITED STATES / CANADA (REMOTE)ENGINEERING — DEVELOPMENT / FULL-TIMEEveryone wants a career with impact. At Nuvalence, you’ll be helping some of the world's most renowned and ambitious companies build winning cloud platforms.Nuvalence software engineers are problem so

Freedom Forever Freedom Forever |
Today

* FULLY REMOTE*Position SummaryThe Back End Software Engineer is responsible for writing, documenting, testing and maintaining software. The Back End Software Engineer is responsible for writing the web services and APIs used by front-end developers and mobile application develop

Plotly Technologies Plotly Technologies |
Yesterday

About the RoleAt Plotly, our customers depend on us for their mission-critical data analysis needs. In this TechOps engineer role, you'll work with code, colleagues, and customers to build Plotly's core products and improve their robustness and scalability.Duties & Responsibi