Want to improve this content? Edit this content
Python Developer @ Scrapinghub

Description

About Scrapinghub:

Founded in 2010, Scrapinghub is on pace to grow revenue more than 5x in the next 3 years, with the largest growth coming from our SaaS product business lines.

We are a fast growing and diverse technology business turning web content into useful data with a cloud-based web crawling platform, off-the-shelf datasets, and turn-key web scraping services.Our comprehensive web crawling stack powers crawls of over 9 billion pages per month and is used by everyone from individual developers to Fortune 100 companies.

We’re a globally distributed team of over 190 Shubbers who are passionate about web data and data science.

 

About the job:

We are looking for software engineers to join our Delivery team to work on web crawler development with Scrapy, our flagship open source project. 

Are you interested in building web crawlers harnessing the Scrapinghub platform, which powers crawls of over 3 billion pages a month?

Do you like working in a company with a strong open source foundation?

 

Job Responsibilities:

  • Design, develop and maintain Scrapy web crawlers
  • Leverage the Scrapinghub platform and our open source projects to perform distributed information extraction, retrieval and data processing
  • Identify and resolve performance and scalability issues with distributed crawling at scale
  • Help identify, debug and fix problems with open source projects, including Scrapy
Requirements
  • 2+ years of software development experience.
  • Solid Python knowledge.
  • Familiarity with Linux/UNIX, HTTP, HTML, Javascript and Networking.
  • Good communication in written & spoken English.
  • Availability to work full time.

 

Bonus points for:

  • Scrapy experience is a big plus.
  • Familiarity with techniques and tools for crawling, extracting and processing data (e.g. Scrapy, NLTK, pandas, scikit-learn, mapreduce, nosql, etc).
Benefits

As a new SHubber, you will:

Become part of a self-motivated, progressive, multi-cultural team.

Have the opportunity to work remotely.

Have the opportunity to go to conferences and meet with the team from across the globe.

Get the chance to work with cutting-edge open source technologies and tools.

 

Active: Yes
Last Modified: 2020-11-17 1:37:36
Contributors of this content: jobs