Merge pull request #11 from EVEprosper/v2-refactor

V2 refactor
EVEprosper · Dec 2, 2017 · c07594b · c07594b
2 parents d33d220 + 352dbcf
commit c07594b
Show file tree

Hide file tree

Showing 37 changed files with 1,933 additions and 61 deletions.
diff --git a/.coveragerc b/.coveragerc
@@ -1,5 +1,10 @@
 [report]
-    omit = prosper/warehouse/_version.py
+    omit = 
+        prosper/datareader/_version.py
+        prosper/datareader/__init__.py
+        prosper/datareader/exceptions.py
+        prosper/datareader/robinhood/__init__.py
+        prosper/datareader/google/__init.py
 
 [paths]
-    source = prosper/warehouse
+    source = prosper/datareader
diff --git a/docs/coins_help.rst b/docs/coins_help.rst
@@ -4,19 +4,19 @@ prosper.datareader.coins
 
 Meant as an extension of `pandas-datareader`_, ``prosper.datareader.coins`` provides the ability to fetch and parse data pertaining to crypto currencies.
 
-``prosper.datareader.coins`` relies on services from `hitBTC`_.
+``prosper.datareader.coins`` relies on services from `hitBTC`_ and `CryptoCompare`_
 
-Info
-====
+to_yahoo()
+----------
 
-General metadata and feed testing tools.
+Do you love how `pandas-datareader`_ displays data and want cryptocoin data in a similar shape?  Most fetchers include a ``to_yahoo`` bool value to convert results to a friendlier format. Not all keys are guaranteed to be returned and is offered only as a "nice to have".
 
-**NOTE**: will implement caching layer for info, since this data should only refresh daily
+Also, since Yahoo and Google have deprecated their financial APIs, do not expect continued coverage.
 
 get_symbol()
 ------------
 
-    ``symbol_name = coins.info.get_symbol('COIN_TIKER', 'CONVERT_TICKER')``
+    ``symbol_name = coins.get_symbol('COIN_TIKER', 'CONVERT_TICKER')``
 
 Price of a crypto currency is measured in relation to other currencies a la FOREX.  `hitBTC`_ requires a smash-cut version of coin + currency.
 
@@ -36,37 +36,53 @@ Expected supported currencies:
 
 - ``USD``
 - ``EUR``
-- ``ETH``
 - ``BTC``
 
-For more info, try ``info.supported_currencies()`` for a current list
-
 get_ticker_info()
 -----------------
 
-    ``ticker_info = coins.info.get_ticker_info('TICKER')``
+    ``ticker_info = coins.get_ticker_info('TICKER')``
 
 If working backwards from a ticker, this function returns the original `hitBTC symbols`_ data.  
 
-Prices
-======
-
 get_quote_hitbtc()
 ------------------
 
-    ``quote_df = coins.prices.get_quote_hitbtc(['BTC', 'ETH'])``
+    ``quote_df = coins.get_quote_hitbtc(['BTC', 'ETH'])``
 
 Get a peek at the current price and trend of your favorite crypto currency.  This feed helps get OHLC data as well as mimic `pandas-datareader`_ quote behavior with keys like ``pct_change``.
 
 get_orderbook_hitbtc()
 ----------------------
 
-    ``orderbook = coins.prices.get_orderbook_hitbtc('BTC', 'asks')``
+    ``orderbook = coins.get_orderbook_hitbtc('BTC', 'asks')``
 
 When you absolutely, positively, need all the data... go to the orderbook.  This supports ``asks`` and ``bids`` for lookup.
 
 ## TODO: add ``both`` behavior
 
+get_quote_cc()
+--------------
+
+    ``quote = coins.get_quote_cc(['BTC', 'ETH'])``
+
+Gets general metadata about requested coins from `CryptoCompare`_.  Supports ``USD`` and ``EUR`` as currency values.  Also returns current prices for desired coins.
+
+get_ohlc_cc()
+-------------
+
+    ``history = coins.get_ohlc_cc('BTC', 30)``
+
+Provides up to 2000 "units" of data for a given ``frequency`` (default=day).
+
+frequency:
+
+- ``'day'``
+- ``'hour'``
+- ``'minute'``
+
+
 .. _pandas-datareader: https://pandas-datareader.readthedocs.io/en/latest/index.html
+.. _CryptoCompare: https://www.cryptocompare.com/api/#introduction
 .. _hitBTC: https://hitbtc.com/
 .. _hitBTC symbols: https://hitbtc.com/api#symbols
diff --git a/docs/getting_started.rst b/docs/getting_started.rst
@@ -16,10 +16,31 @@ It is encouraged to import the smallest scope required.
 
    import prosper.datareader.stocks as stocks
 
-   todays_news = stocks.news.company_news_google('MU')
-
    todays_quote = stocks.prices.get_quote_rh('MU')
 
+Contributing Data Sources
+=========================
+
+The purpose of this project is to provide a common framework to fetch/parse data and put appropriate testing scaffolding around feeds.  New data sources are always encouraged!
+
+The template for adding new functions is:
+
+Consider Top-Level Needs
+------------------------
+
+If you are extending an existing framework, think of the common questions/data returned.  More stock functions should go with ``prosper.datareader.stocks``.  
+
+Adding a wild new segment?  A new top-level module should be considered.  EX: adding `Albion`_ data parsing should be added to ``prosper.datareader.albion``.  
+
+All functions in top-level scopes should return ``pandas.DataFrame``.
+
+Consider Data Sources
+---------------------
+
+Though functionality might be mixed between scopes, sources should be split by API provider.  ``prosper.datareader.cryptocompare`` contains all the API parsing required to interface with `CryptoCompare`_.  These individual REST interfaces should then be imported into their useage scope like ``prosper.datareader.coins`` to return the data users expect.
+
+Most data-source feeds should probably have `jsonschema`_ tests.
+
 Developing ProsperDatareader
 ============================
 
@@ -78,3 +99,5 @@ Release message should include useful update notes, and versions should follow `
 .. _currently broken: https://github.com/EVEprosper/ProsperUtilities/issues/2
 .. _PyPI: https://pypi.python.org/pypi/ProsperDatareader
 .. _Semantic Versioning: http://semver.org/
+.. _CryptoCompare: https://www.cryptocompare.com/api/#introduction
+.. _Albion: https://albion-data.com/
diff --git a/docs/index.rst b/docs/index.rst
@@ -24,6 +24,7 @@ Supported Feeds
 * `Utils`_: General utilities for additional insights
 * `Stocks`_: Parse IRL stock quote data
 * `Coins`_: Data utilities for cryptocoin price quotes
+* `News`_: Fetch relevant news data
 
 Index
 =====
@@ -49,6 +50,7 @@ Indices and tables
 .. _Stocks: stocks_help.html
 .. _Utils: utils_help.html
 .. _Coins: coins_help.html
+.. _News: news_help.html
 
 .. |Build Status| image:: https://travis-ci.org/EVEprosper/ProsperDatareader.svg?branch=master
    :target: https://travis-ci.org/EVEprosper/ProsperDatareader

diff --git a/docs/news_help.rst b/docs/news_help.rst
@@ -0,0 +1,18 @@
+=======================
+prosper.datareader.news
+=======================
+
+Fetching news feeds should be easy.  These endpoints return dataframes with articles, links, and some metadata.  These feeds save you from a few pitfalls.
+
+- Non-compliant JSON 
+- Weird single-character identifiers
+- Strange tree-shaped feeds
+
+company_news_rh()
+-----------------
+
+    ``news_dataframe = news.company_news_rh('TICKER')``
+
+Fetches articles from `Robinhood`_ for desired ticker.  NOTE: ``page_limit`` may be needed for popular stocks.
+
+**NOTE**: Google datasources have been deprecated
diff --git a/docs/stocks_help.rst b/docs/stocks_help.rst
@@ -4,48 +4,14 @@ prosper.datareader.stocks
 
 Meant as an extension of `pandas-datareader`_, ``prosper.datareader.stocks`` provides utilities for collecting news and quotes from other sources.
 
-``prosper.datareader.stocks`` relies on services from `Google`_ and `Robinhood`_ to collect data.
-
-News
-====
-
-Fetching news feeds should be easy.  These endpoints return dataframes with articles, links, and some metadata.  These feeds save you from a few pitfalls.
-
-- Non-compliant JSON 
-- Weird single-character identifiers
-- Strange tree-shaped feeds
-
-company_news_google()
----------------------
-
-    ``news_dataframe = stocks.news.company_news_google('TICKER')``
-
-Fetches all the news articles for a single ticker.  Human readable headers are enabled by default, but ``pretty=False`` will return columns to original short-names
-
-market_news_google()
---------------------
-
-    ``news_dataframe = stocks.news.market_news_google()``
-
-Fetches all articles on `Google`_ general market feed.  Technically wrapped around ``company_news_google()`` function.  
-
-company_news_rh()
------------------
-
-    ``news_dataframe = stocks.news.company_news_rh('TICKER')``
-
-Fetches articles from `Robinhood`_ for desired ticker.  NOTE: ``page_limit`` may be needed for popular stocks.
-
-
-Prices
-======
+``prosper.datareader.stocks`` relies on services from `Robinhood`_ to collect data.
 
 Though `pandas-datareader`_ is a powerful tool for generic data fetching, it lacks some support for `Robinhood`_.
 
 get_quote_rh()
 --------------
 
-    ``quote_dataframe = stocks.prices.get_quote_rh('TICKER')``
+    ``quote_dataframe = stocks.get_quote_rh('TICKER')``
 
 Default quote data:
 
@@ -59,7 +25,6 @@ Default quote data:
 Designed to mirror ``pandas_datareader.data.get_quote_yahoo()`` from `pandas-datareader`_
 
 .. _pandas-datareader: https://pandas-datareader.readthedocs.io/en/latest/index.html
-.. _Google: https://www.google.com/finance
 .. _Robinhood: https://support.robinhood.com/hc/en-us
 .. _company_news_google(): source/datareader.stocks.html#datareader.stocks.news.fetch_company_news_google
 .. _company_news_rh(): source/datareader.stocks.html#datareader.stocks.news.fetch_company_news_rh