tagged with crawler
Displaying 1-3 of 3 total results

miry

miry/medup Archived

Download all content from Medium and Dev.to to local folder

tool markdown cli json crawler medium devto sync
  • 36
  • 9
  • 16
  • 0
  • 3
  • over 2 years ago
squirm-framework

squirm-framework/squirm

This was the night of the crawling terror!

crystal web scrape crawler crawling spider scraper data-engineering squirm
  • 31
  • 2
  • 0
  • 0
  • 7
  • 11 months ago
confact

confact/Spider.cr

Spider.cr is a spider crawler in Crystal. It handles collecting, scraping, and parsing. So you can spend your time collecting the data you want on a big scale.

crawler spider
  • 9
  • 0
  • 1
  • 0
  • 1
  • almost 3 years ago