Skip to content
forked from rockets/rockets

Broadcasts new posts and comments as they are created on reddit.com

Notifications You must be signed in to change notification settings

Uduchi2nd/rockets

 
 

Folders and files

NameName
Last commit message
Last commit date

Latest commit

 

History

38 Commits
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 
 

Repository files navigation

Rockets Author Support Stability

Mission

Provide a way to continuously monitor new content on reddit.com in near real-time.

Many reddit bots rely on monitoring new content, constantly sending requests to keep up. Unfortunately this means that bots can't use their precious rate-limit tokens to then do something with that content.

Rockets allows you to subscribe to a channel, with the ability to specify content filters. All you need to do is open a web socket connection to the command center at ws://rockets.cc:3210 and transmit your subscription.

The client receives JSON models exactly as they appear in reddit listings (with kind and data keys). These will be sent one at a time, but are not guaranteed to be in perfect chronological order.

For a detailed overview of the JSON models, see the reddit JSON wiki. Note that the wiki might not always match the actual model data, because reddit doesn't always update it when they make changes to the models.

Demo

See rockets-demo.

Client

See rockets-client.

Subscriptions

Subscriptions are sent as JSON in the following format:

{
    "channel": "",
    "filters": {

    }
}

Channels

  • comments
  • posts

Filters

All filters can be provided as either a single value or an array of values. A filter is considered a match if any of the values match the corresponding value in the model.

Comments

Key Type Description
contains string (regex) Comment body (markdown), case-insensitive
subreddit string Subreddit in which the comment was made, eg. "subreddit"
author string The user who made the comment, eg. "username"
post string Comments that are replies to a specific post, eg. "t3_abcd"
root boolean Comments that are not replies to other comments

Posts

Key Type Description
contains string (regex) Link title and selftext (markdown), case-insensitive
subreddit string Subreddit in which the post was made, eg. "subreddit"
author string The user who made the post, eg. "username"
domain string Link's domain, or "self.subreddit" if it's a selfpost
url string Link's URL, or the post's permalink if it's a selfpost
nsfw boolean Flagged NSFW at the time of creation

Example

{
    "channel": "comments",
    "filters": {

        "subreddit": [
          "space",
          "spacex",
          "science"
        ],

        "contains": [
          "mars",
          "rockets",
          "\\bion\\b"
        ]
    }
}

Limitations

Dropped connections

The server will drop all connections when being updated or restarted, which can't be avoided. Make sure that your client will attempt to reconnect automatically when a connection is dropped. Updates will be posted to /r/redditdev in the event of unexpected downtime.

Latency

Even though the command center is in low reddit orbit, there will always be some delay between a model's creation and its broadcast. Most of the time this delay will only be a few seconds, but could be several minutes in some cases. This occurs when reddit.com goes down, or during very busy periods. The command center will catch up again when things are back to normal or when busy periods subside. Data will not be lost during this time.

Bandwidth

A single model is roughly 1kb of JSON. The command center receives an average of 30 models per second. This works out to about 2.6GB per day, per unfiltered connection. With a limit of 8TB downstream traffic per month, this equates to a maximum of roughly 100 concurrent unfiltered connections.

Subscriptions

You are only allowed one subscription per connection per channel, where new subscriptions replace previous ones. It's possible that some unwanted models may still be received after a subscription has been replaced, so you should open a new connection when you need a strict boundary between subscriptions.

Support

There are a few ways to resupply and assist the CC:

  • Use filters to avoid receiving unwanted data.
  • Donate a few dollars to help cover server costs.

Credits

Illustrations by Ken Samonte.

About

Broadcasts new posts and comments as they are created on reddit.com

ws://rockets.cc:3210

Resources

Stars

Watchers

Forks

Releases

No releases published

Packages

No packages published

Languages

  • CoffeeScript 98.3%
  • JavaScript 1.7%