Data Engineer (Remote - North America)
Paytm LabsToronto, ON24 days ago
About Paytm Labs:
At Paytm Labs, we’re on a mission to provide useful technological solutions that enrich and empower millions of people in their daily lives. We apply big data, artificial intelligence and machine learning to bring the next generation of financial products and services to the Indian, Japanese and Canadian markets.
As a company, we’re committed to offering the most transparent, secure, and personalized consumer experience to over 500 million users and over 20 million merchants. Since our journey began 6 years ago, we’ve launched the Paytm Canada app (our bill management app), and PayPay (a QR-based payment app in Japan), all while powering the Paytm India app.
If working with billions of events, petabytes of data and optimizing for last millisecond is something that excites you then read on! We are looking for Data Engineers who have seen their fair share of messy data sets and have been able to structure them for building useful AI products.
You will be working on writing frameworks building for real time and batch pipelines to ingest and transform events(108 scale) from 100’s of applications every day. Our ML and Software engineers consume these for building data products like personalization and fraud detection. You will also help optimize the feature pipelines for fast execution and work with software engineers to build event driven microservices.
You will get to put cutting edge tech in production and freedom to experiment with new frameworks, try new ways to optimize and resources to build next big thing in fintech using data!
- Work directly with Machine Learning Engineers and Platform Engineering Team to create reusable experimental and production data pipelines.
- Understand, tune, and master the processing engines (like Spark, Hive, Samza, etc) used day-to-day.
- Keep the data whole, safe, and flowing with expertise on high volume data ingest and streaming platforms (like Spark Streaming, Kafka, etc).
- Sheppard and shape the data by developing efficient structures and schema for the data in storage and transit.
- Explore as many new technology options for data processing, storage, and share them with the team.
- Develop tools and contribute to open source wherever possible.
- Adopt problem solving as a way of life – always go to root cause
- Degree in Computer Science, Engineering or a related field
- You have previously worked on building serious data pipelines ingesting and transforming > 10 ^6 events per minute and terabytes of data per day.
- You are passionate about producing clean, maintainable and testable code part of real-time data pipeline.
- You understand how microservices work and are familiar with concepts of data modelling.
- You can connect different services and processes together even if you have not worked with them before and follow the flow of data through various pipelines to debug data issues.
- You have worked with Spark and Kafka before and have experimented or heard about Flink/Druid/Ignite/Presto/Athena and understand when to use one over the other.
- On a bad day maintaining zookeeper and bringing up cluster doesn’t bother you.
- You may not be a networking expert but you understand issues with ingesting data from applications in multiple data centres across geographies, on-premise and cloud and will find a way to solve them.
- Proficient in Java/Scala/Python/Spark
What we Offer!
Due to the pandemic, we have been and will continue to WFH until it is safe to open our office. Our company culture and values remain at the core of everything we do.
- For the third year in a row, we are proud to announce that we have been certified as a Great Place to Work
- We were also certified as one of the Best Workplaces for Mental Wellness in 2020
- We are an open work environment that fosters collaboration, ownership, creativity, and urgency
- We ensure flexible hours outside of our core working hours
- Enrolment in the Group Health Benefits plan right from day 1, no waiting period
- To keep things fun and stress-free during COVID-19 we started Virtual Daily, Virtual Weekly and Monthly team bonding activities including: Trivia, Games Nights, Movies Nights, Arts & Crafts (e.g. Origami), Lunch & Learns (e.g. Sign Language 101), Virtual Wellness Sessions (e.g. Meditation, Morning stretches), Virtual Team Ubereats Lunches, and so much more
When we are able to open our office, our in-office experience consists of:
- Team building events (anything from axe throwing, go-karting, bike riding, etc.)
- Fuel for the day: Weekly delivery of groceries, and all types of snacks
- Catered lunches and desserts on a monthly basis
- Flexibility with WFH
- Daily fun in the office with our competitive games of Ping Pong, Pool, Smash Bros competitions, or FIFA
- And of course, an unlimited amount of freshly made coffee! We’re pretty serious about our coffee beans
Notice for Job Applicants
Following the advice of Canadian health authorities, to mitigate the risk of the potential spread of COVID-19 and support social distancing, all recruiting activities including interviews and new hire onboarding will be conducted remotely. While we are doing our best to ensure reasonable response times, please expect potential delays during the recruiting process due to the current situation.
We are an equal opportunity employer and value diversity and uniqueness at our company. We thank all applicants, however, only those selected for an interview will be contacted.
Paytm Labs is committed to meeting the accessibility needs of all individuals in accordance with the Accessibility for Ontarians with Disabilities Act (AODA) and the Ontario Human Rights Code (OHRC). Should you require accommodations during the recruitment and selection process, please let us know.
Don't have Paytm Canada App yet?