This was the night of the crawling terror!

Squirm is a framework for crawling web sites and extracting structured data which can be used for a wide range of useful applications, like data mining, information processing or historical archival.

Installation

  1. Add the dependency to your shard.yml:

    dependencies:
  squirm:
    github: squirm-framework/squirm

  2. Run shards install

Usage

View the usage in the examples directory.

Contributing

  1. Fork it (https://github.com/squirm-framework/squirm/fork)
  2. Create your feature branch (git checkout -b my-new-feature)
  3. Commit your changes (git commit -am 'Add some feature')
  4. Push to the branch (git push origin my-new-feature)
  5. Create a new Pull Request

Contributors