Skip to content

Commit

Permalink
1)add docs about adding tradable entity and compare intent 2)Index me…
Browse files Browse the repository at this point in the history 
…ta data from provider em
  • Loading branch information
@foolcage
foolcage committed Apr 27, 2022
1 parent 7e069c5 commit a6832ae
Show file tree
Hide file tree
Showing 22 changed files with 388 additions and 147 deletions.
Binary file added docs/source/_static/compare_cn_us.png
View file
Loading
Sorry, something went wrong. Reload?
Sorry, we cannot display this file.
Sorry, this file is invalid so it cannot be displayed.
27 changes: 27 additions & 0 deletions docs/source/contributor.rst
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,27 @@
====================
Contributor
====================

Thanks, my friends.

* 隋鹏飞
* `Tiger <https://github.com/Tigerno1>`_
* 蓝小球
* 青玄
* \*崇
* \*阳
* 张凯
* Rubicon_陈小虾
* `看门大爷 <https://github.com/Mr-later>`_
* 芝华
* 雨多田光
* 叶金荣
* 冷冷
* Landy
* Sunny
* Ccc国毅
* Jim Tong
* tom
* 小七
* 小强
* Cascara Latte- 刘可烁
197 changes: 96 additions & 101 deletions docs/source/data/adding_new_entity.rst
View file
Original file line number Diff line number Diff line change
@@ -1,72 +1,34 @@
.. _add_new_entity:
.. _adding_new_entity:

=================
Adding new entity
=================

It's human nature to like the new and hate the old. Adding new TradableEntity is easy in zvt.

And from a higher perspective, trading is everywhere. you make trading everytime when you make the
decision.

So you could treat Country as TradableEntity and make trading when making decision where to live or invest.


Adding new entity is nothing than a specific case of :ref:`Adding data <extending_data.add_data>`.
Let's show the key steps below.
Let's show the key steps below which add :class:`~.zvt.domain.meta.future_meta.Future`.

Define entity Schema
--------------------------

::

# -*- coding: utf-8 -*-
from sqlalchemy.ext.declarative import declarative_base

from sqlalchemy import Column, String, Float
from sqlalchemy.orm import declarative_base

from zvt.contract.schema import TradableEntity
from zvt.contract.register import register_schema, register_entity
from zvt.contract.schema import TradableEntity

CountryMetaBase = declarative_base()


@register_entity(entity_type="country")
class Country(CountryMetaBase, TradableEntity):
__tablename__ = "country"

#: 区域
#: region
region = Column(String(length=128))
#: 首都
#: capital city
capital_city = Column(String(length=128))
#: 收入水平
#: income level
income_level = Column(String(length=64))
#: 贷款类型
#: lending type
lending_type = Column(String(length=64))
#: 经度
#: longitude
longitude = Column(Float)
#: 纬度
#: latitude
latitude = Column(Float)


register_schema(providers=["wb"], db_name="country_meta", schema_base=CountryMetaBase)

entity_type, exchange and code define the entity, for country, it's in following way:
FutureMetaBase = declarative_base()

::

entity_type = "country"
exchange = "galaxy"
code = "iso code"
@register_entity(entity_type="future")
class Future(FutureMetaBase, TradableEntity):
__tablename__ = "future"

e.g. country_galaxy_CN = China, country_galaxy_US = United States of America

register_schema(providers=["em"], db_name="future_meta", schema_base=FutureMetaBase)

Implement recorder for the entity
---------------------------------
Expand All @@ -75,88 +37,121 @@ Implement recorder for the entity

from zvt.contract.api import df_to_db
from zvt.contract.recorder import Recorder
from zvt.domain.meta.country_meta import Country
from zvt.recorders.wb import wb_api
from zvt.domain import Future
from zvt.recorders.em import em_api


class WBCountryRecorder(Recorder):
provider = "wb"
data_schema = Country
class EMFutureRecorder(Recorder):
provider = "em"
data_schema = Future

def run(self):
df = wb_api.get_countries()
df = em_api.get_tradable_list(entity_type="future")
self.logger.info(df)
df_to_db(df=df, data_schema=self.data_schema, provider=self.provider, force_update=self.force_update)

Define schema for the entity
----------------------------

e.g treasury yield of the country
Define OHLC schema(kdata) for the entity
----------------------------------------

zvt provide a standard way to generate OHLC schema for the tradable entity.
All `OHLC schemas <https://github.com/zvtvz/zvt/blob/master/src/zvt/domain/quotes>`_ is generated by
`fill project script <https://github.com/zvtvz/zvt/blob/master/src/zvt/fill_project.py>`_.

e.g generate Future OHLC schema.

::

# -*- coding: utf-8 -*-
from sqlalchemy import Column, String, Float
from sqlalchemy.orm import declarative_base
gen_kdata_schema(
pkg="zvt",
providers=["em"],
entity_type="future",
levels=[IntervalLevel.LEVEL_1DAY],
entity_in_submodule=True,
)

The OHLC schema definition principle is: **one level one file**

from zvt.contract import Mixin
from zvt.contract.register import register_schema
So we would define a common OHLC schema for each entity type in `quotes module <https://github.com/zvtvz/zvt/blob/master/src/zvt/domain/quotes/__init__.py>`_.

CurrencyBase = declarative_base()
e.g. Future common OHLC schema

::

class FutureKdataCommon(KdataCommon):
#: 持仓量
interest = Column(Float)
#: 结算价
settlement = Column(Float)
#: 涨跌幅(按收盘价)
# change_pct = Column(Float)
#: 涨跌幅(按结算价)
change_pct1 = Column(Float)

And we could relate the common kdata schema with the recorder and route level to specific schema automatically.

class TreasuryYield(CurrencyBase, Mixin):
__tablename__ = "treasury_yield"
Implement recorder for OHLC schema(kdata)
-----------------------------------------

code = Column(String(length=32))
Check `em quotes recorder <https://github.com/zvtvz/zvt/blob/master/src/zvt/recorders/em/quotes/em_kdata_recorder.py>`_ for
the details.

# 2年期
yield_2 = Column(Float)
# 5年期
yield_5 = Column(Float)
# 10年期
yield_10 = Column(Float)
# 30年期
yield_30 = Column(Float)
::

class EMFutureKdataRecorder(BaseEMStockKdataRecorder):
entity_provider = "em"
entity_schema = Future

register_schema(providers=["em"], db_name="currency", schema_base=CurrencyBase)
data_schema = FutureKdataCommon

And the `recorder <https://github.com/zvtvz/zvt/blob/master/src/zvt/recorders/em/macro/em_treasury_yield_recorder.py>`_ for the schema

Use them in zvt way
-------------------

Find the rich country:
Fetch the entity list:

::

>>> from zvt.domain import Country
>>> Country.record_data()
>>> df = Country.query_data()
>>> df[df['income_level']=='High income']

id entity_id timestamp entity_type exchange code name list_date end_date region capital_city income_level lending_type longitude latitude
0 country_galaxy_AW country_galaxy_AW None country galaxy AW Aruba None None Latin America & Caribbean Oranjestad High income Not classified -70.016700 12.516700
7 country_galaxy_AD country_galaxy_AD None country galaxy AD Andorra None None Europe & Central Asia Andorra la Vella High income Not classified 1.521800 42.507500
9 country_galaxy_AE country_galaxy_AE None country galaxy AE United Arab Emirates None None Middle East & North Africa Abu Dhabi High income Not classified 54.370500 24.476400
13 country_galaxy_AG country_galaxy_AG None country galaxy AG Antigua and Barbuda None None Latin America & Caribbean Saint John's High income IBRD -61.845600 17.117500
14 country_galaxy_AU country_galaxy_AU None country galaxy AU Australia None None East Asia & Pacific Canberra High income Not classified 149.129000 -35.282000
.. ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
277 country_galaxy_TW country_galaxy_TW None country galaxy TW Taiwan, China None None East Asia & Pacific High income Not classified NaN NaN
282 country_galaxy_UY country_galaxy_UY None country galaxy UY Uruguay None None Latin America & Caribbean Montevideo High income IBRD -56.067500 -34.894100
283 country_galaxy_US country_galaxy_US None country galaxy US United States None None North America Washington D.C. High income Not classified -77.032000 38.889500
287 country_galaxy_VG country_galaxy_VG None country galaxy VG British Virgin Islands None None Latin America & Caribbean Road Town High income Not classified -64.623056 18.431389
288 country_galaxy_VI country_galaxy_VI None country galaxy VI Virgin Islands (U.S.) None None Latin America & Caribbean Charlotte Amalie High income Not classified -64.896300 18.335800
>>> from zvt.domain import Future
>>> Future.record_data()
>>> df = Future.query_data()
>>> print(df)

[80 rows x 15 columns]
id entity_id timestamp entity_type exchange code name list_date end_date
0 future_cffex_IC future_cffex_IC NaT future cffex IC 中证当月连续 NaT None
1 future_cffex_IF future_cffex_IF NaT future cffex IF 沪深当月连续 NaT None
2 future_cffex_IH future_cffex_IH NaT future cffex IH 上证当月连续 NaT None
3 future_cffex_T future_cffex_T NaT future cffex T 十债当季连续 NaT None
4 future_cffex_TF future_cffex_TF NaT future cffex TF 五债当季连续 NaT None
.. ... ... ... ... ... ... ... ... ...
65 future_ine_LU future_ine_LU 2020-06-22 future ine LU 低硫燃油主力 2020-06-22 None
66 future_czce_PF future_czce_PF 2020-10-12 future czce PF 短纤主力 2020-10-12 None
67 future_ine_BC future_ine_BC 2020-11-19 future ine BC 国际铜主力 2020-11-19 None
68 future_dce_LH future_dce_LH 2021-01-08 future dce LH 生猪主力 2021-01-08 None
69 future_czce_PK future_czce_PK 2021-02-01 future czce PK 花生主力 2021-02-01 None

[70 rows x 9 columns]

Compare treasury yields of different maturities:
Fetch the quotes:

::

>>> from zvt.domain import TreasuryYield
>>> from zvt.api.intent import compare
>>> TreasuryYield.record_data()
>>> compare(codes=["US"], schema=TreasuryYield, columns=["yield_2", "yield_10", "yield_30"])

.. image:: ../_static/compare_yields.png
>>> from zvt.domain import Future1dKdata
>>> Future1dKdata.record_data(code="CU")
>>> df = Future1dKdata.query_data(code="CU")
>>> print(df)

id entity_id timestamp provider code name level open close high low volume turnover change_pct turnover_rate interest settlement change_pct1
0 future_shfe_CU_1996-04-03 future_shfe_CU 1996-04-03 em CU 沪铜主力 1d 22930.0 22840.0 23000.0 22840.0 353.0 0.000000e+00 0.0000 0.0 None None None
1 future_shfe_CU_1996-04-04 future_shfe_CU 1996-04-04 em CU 沪铜主力 1d 22700.0 22750.0 22820.0 22650.0 251.0 0.000000e+00 -0.0039 0.0 None None None
2 future_shfe_CU_1996-04-05 future_shfe_CU 1996-04-05 em CU 沪铜主力 1d 22520.0 22780.0 22820.0 22500.0 298.0 0.000000e+00 0.0013 0.0 None None None
3 future_shfe_CU_1996-04-08 future_shfe_CU 1996-04-08 em CU 沪铜主力 1d 22660.0 22650.0 22680.0 22600.0 98.0 0.000000e+00 -0.0057 0.0 None None None
4 future_shfe_CU_1996-04-09 future_shfe_CU 1996-04-09 em CU 沪铜主力 1d 22830.0 22810.0 22860.0 22810.0 56.0 0.000000e+00 0.0071 0.0 None None None
... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ... ...
6343 future_shfe_CU_2022-04-21 future_shfe_CU 2022-04-21 em CU 沪铜主力 1d 74140.0 74480.0 74750.0 74140.0 48008.0 1.787678e+10 -0.0004 0.0 None None None
6344 future_shfe_CU_2022-04-22 future_shfe_CU 2022-04-22 em CU 沪铜主力 1d 74800.0 75010.0 75200.0 74690.0 58874.0 2.205633e+10 0.0073 0.0 None None None
6345 future_shfe_CU_2022-04-25 future_shfe_CU 2022-04-25 em CU 沪铜主力 1d 74900.0 73660.0 75190.0 73660.0 107455.0 3.989090e+10 -0.0168 0.0 None None None
6346 future_shfe_CU_2022-04-26 future_shfe_CU 2022-04-26 em CU 沪铜主力 1d 73170.0 73260.0 73750.0 72500.0 113019.0 4.130931e+10 -0.0132 0.0 None None None
6347 future_shfe_CU_2022-04-27 future_shfe_CU 2022-04-27 em CU 沪铜主力 1d 72990.0 73100.0 73560.0 72910.0 61563.0 2.254089e+10 0.0000 0.0 None None None

[6348 rows x 18 columns]
1 change: 1 addition & 0 deletions docs/source/data/index.rst
View file
Original file line number Diff line number Diff line change
Expand Up @@ -22,3 +22,4 @@ Data
record_and_query
extending_data
adding_new_entity
trading_anything
Loading

0 comments on commit a6832ae

Please sign in to comment.