Skip to content

Downloads and stores a given URL (including js, css, and images) for offline use.

License

Notifications You must be signed in to change notification settings

johnernaut/webhog

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

45 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

webhog

webhog is a package that stores and downloads a given URL (including js, css, and images) for offline use and uploads it to a given AWS-S3 account (more persistance options to come).

##Installation go get github.com/johnernaut/webhog

##Usage Make a POST request to http://localhost:3000/scrape with a header set to value X-API-KEY: SCRAPEAPI. Pass in a JSON value of the URL you'd like to fetch: { "url": "http://facebook.com"} (as an example). You'll notice an Ent dir: /blah/blah/blah printed to the console - your assets are saved there. To test, open the given index.html file.

##Configuration Create a webhog.yml file in the running directory. The following options are supported:

development:
  mongodb: mongodb://127.0.0.1:27017/webhog
  api_key: SCRAPEAPI
  aws_key: <env var name for s3 key>
  aws_secret: <env var name for s3 secret>
  bucket: mybucket
production:
staging:

The setting root-key is established via a MARTINI_ENV environment variable that you should set.

About

Downloads and stores a given URL (including js, css, and images) for offline use.

Topics

Resources

License

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published