Fixed tensorflow upgrade issues

README.md

@@ -1,9 +1,8 @@
-This is the implementation of our paper, A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem ([arXiv:1706.10059](https://arxiv.org/abs/1706.10059)), together with a toolkit of portfolio management research.
+This is the original implementation of our paper, A Deep Reinforcement Learning Framework for the Financial Portfolio Management Problem ([arXiv:1706.10059](https://arxiv.org/abs/1706.10059)), together with a toolkit of portfolio management research.
 
-* The policy optimization method we described in the paper is designed specifically for portfolio management problem.
-  * Differing from the general-purpose reinforcement learning algorithms, it has similar efficiency and robustness to supervized learning.
-  * This is because we formulate the problem as an immediate reward optimization problem regularised by transaction cost, which does not require a monte-carlo or bootstrapped estimation of the gradients.
-* One can configurate the topology, training method or input data in a separate json file. The training process will be recorded and user can visualize the training using tensorboard.
+* The deep reinforcement learning framework is the core part of the library.
+The method is basically the policy gradient on immediate reward.
+ One can configurate the topology, training method or input data in a separate json file. The training process will be recorded and user can visualize the training using tensorboard.
 Result summary and parallel training are allowed for better hyper-parameters optimization.
 * The financial-model-based portfolio management algorithms are also embedded in this library for comparision purpose, whose implementation is based on Li and Hoi's toolkit [OLPS](https://github.com/OLPS/OLPS).
 
@@ -45,5 +44,10 @@ We welcome contributions from the community, including but not limited to:
 
 There is always risk of loss in trading. **All trading strategies are used at your own risk**
 
-* The project has been open-sourced for many years (since 2017). The market efficiency may have increased quite a lot since then. There is no guarantee the exact same algorithm can still work.
-* Although we tried to make assumptions closer to the situation in the real market, the results are all from backtest on static historical data. The slippage or market impact can always be a problem if you deploy it as a live trading bot.
+*The volumes of many cryptocurrency markets are still low. Market impact and slippage may badly affect the results during live trading.*
+
+## Donation
+If you have made some profits because of this project or you just love reading our codes, please consider making a small donation to our ongoing projects via the following BTC or ETH address. All donations will be used as student stipends. 
+
+* BTC: [1PEHK1nVi8x4HQM1A67anyfYzdDFoqUo21](https://blockchain.info/address/1PEHK1nVi8x4HQM1A67anyfYzdDFoqUo21)
+* ETH: [0xa8197289e16C0cCad0a4838719ce11C9A920cfB7](https://etherscan.io/address/0xa8197289e16C0cCad0a4838719ce11C9A920cfB7)

environment37.yml

@@ -0,0 +1,109 @@
+name: pgportfolio37
+channels:
+  - defaults
+  - anaconda
+dependencies:
+  - _libgcc_mutex=0.1=main
+  - _tflow_select=2.1.0=gpu
+  - absl-py=0.12.0=py37h06a4308_0
+  - astor=0.8.1=py37h06a4308_0
+  - blas=1.0=mkl
+  - c-ares=1.17.1=h27cfd23_0
+  - ca-certificates=2021.1.19=h06a4308_1
+  - certifi=2020.12.5=py37h06a4308_0
+  - coverage=5.5=py37h27cfd23_2
+  - cudatoolkit=10.0.130=0
+  - cudnn=7.6.5=cuda10.0_0
+  - cupti=10.0.130=0
+  - cvxopt=1.2.0=py37ha87f1a5_0
+  - cycler=0.10.0=py37_0
+  - cython=0.29.22=py37h2531618_0
+  - dbus=1.13.18=hb2f20db_0
+  - expat=2.3.0=h2531618_2
+  - fontconfig=2.13.1=h6c09931_0
+  - freetype=2.10.4=h5ab3b9f_0
+  - gast=0.2.2=py37_0
+  - glib=2.68.0=h36276a3_0
+  - glpk=4.65=h3ceedfd_2
+  - gmp=6.2.1=h2531618_2
+  - google-pasta=0.2.0=py_0
+  - grpcio=1.36.1=py37h2157cd5_1
+  - gsl=2.4=h14c3975_4
+  - gst-plugins-base=1.14.0=h8213a91_2
+  - gstreamer=1.14.0=h28cd5cc_2
+  - h5py=2.10.0=py37hd6299e0_1
+  - hdf5=1.10.6=hb1b8bf9_0
+  - icu=58.2=he6710b0_3
+  - importlib-metadata=3.7.3=py37h06a4308_1
+  - intel-openmp=2020.2=254
+  - jpeg=9b=h024ee3a_2
+  - keras-applications=1.0.8=py_1
+  - keras-preprocessing=1.1.2=pyhd3eb1b0_0
+  - kiwisolver=1.3.1=py37h2531618_0
+  - lcms2=2.11=h396b838_0
+  - ld_impl_linux-64=2.33.1=h53a641e_7
+  - libffi=3.3=he6710b0_2
+  - libgcc-ng=9.1.0=hdf63c60_0
+  - libgfortran-ng=7.3.0=hdf63c60_0
+  - libpng=1.6.37=hbc83047_0
+  - libprotobuf=3.14.0=h8c45485_0
+  - libstdcxx-ng=9.1.0=hdf63c60_0
+  - libtiff=4.1.0=h2733197_1
+  - libuuid=1.0.3=h1bed415_2
+  - libxcb=1.14=h7b6447c_0
+  - libxml2=2.9.10=hb55368b_3
+  - lz4-c=1.9.3=h2531618_0
+  - markdown=3.3.4=py37h06a4308_0
+  - matplotlib=3.3.4=py37h06a4308_0
+  - matplotlib-base=3.3.4=py37h62a2d02_0
+  - metis=5.1.0=hf484d3e_4
+  - mkl=2020.2=256
+  - mkl-service=2.3.0=py37he8ac12f_0
+  - mkl_fft=1.3.0=py37h54f3939_0
+  - mkl_random=1.1.1=py37h0573a6f_0
+  - ncurses=6.2=he6710b0_1
+  - numpy=1.19.2=py37h54aff64_0
+  - numpy-base=1.19.2=py37hfa32c7d_0
+  - olefile=0.46=py37_0
+  - openssl=1.1.1k=h27cfd23_0
+  - opt_einsum=3.1.0=py_0
+  - pandas=0.23.4=py37h04863e7_0
+  - pcre=8.44=he6710b0_0
+  - pillow=8.2.0=py37he98fc37_0
+  - pip=21.0.1=py37h06a4308_0
+  - protobuf=3.14.0=py37h2531618_1
+  - pympler=0.9=py_0
+  - pyparsing=2.4.7=pyhd3eb1b0_0
+  - pyqt=5.9.2=py37h05f1152_2
+  - python=3.7.10=hdb3f193_0
+  - python-dateutil=2.8.1=pyhd3eb1b0_0
+  - pytz=2021.1=pyhd3eb1b0_0
+  - qt=5.9.7=h5867ecd_1
+  - readline=8.1=h27cfd23_0
+  - scipy=1.6.2=py37h91f5cce_0
+  - seaborn=0.11.1=pyhd3eb1b0_0
+  - setuptools=52.0.0=py37h06a4308_0
+  - sip=4.19.8=py37hf484d3e_0
+  - six=1.15.0=py37h06a4308_0
+  - sqlite=3.35.3=hdfb4753_0
+  - suitesparse=5.2.0=h9e4a6bb_0
+  - tbb=2020.3=hfd86e86_0
+  - tensorboard=1.15.0=pyhb230dea_0
+  - tensorflow=1.15.0=gpu_py37h0f0df58_0
+  - tensorflow-base=1.15.0=gpu_py37h9dcbed7_0
+  - tensorflow-estimator=1.15.1=pyh2649769_0
+  - tensorflow-gpu=1.15.0=h0d30ee6_0
+  - termcolor=1.1.0=py37h06a4308_1
+  - tk=8.6.10=hbc83047_0
+  - tornado=6.1=py37h27cfd23_0
+  - typing_extensions=3.7.4.3=pyha847dfd_0
+  - webencodings=0.5.1=py37_1
+  - werkzeug=0.16.1=py_0
+  - wheel=0.36.2=pyhd3eb1b0_0
+  - wrapt=1.12.1=py37h7b6447c_1
+  - xz=5.2.5=h7b6447c_0
+  - zipp=3.4.1=pyhd3eb1b0_0
+  - zlib=1.2.11=h7b6447c_3
+  - zstd=1.4.9=haebb681_0
+  - pip:
+    - tflearn==0.5.0

pgportfolio/learn/network.py

@@ -3,7 +3,8 @@
 from __future__ import print_function
 from __future__ import absolute_import
 from __future__ import division
-import tensorflow as tf
+import tensorflow.compat.v1 as tf
+tf.disable_v2_behavior()
 import tflearn
 
 

pgportfolio/learn/nnagent.py

@@ -1,6 +1,7 @@
 from __future__ import absolute_import, print_function, division
 import tflearn
-import tensorflow as tf
+import tensorflow.compat.v1 as tf
+tf.disable_v2_behavior()
 import numpy as np
 from pgportfolio.constants import *
 import pgportfolio.learn.network as network

pgportfolio/learn/tradertrainer.py

@@ -10,7 +10,8 @@
 import tflearn
 import numpy as np
 import pandas as pd
-import tensorflow as tf
+import tensorflow.compat.v1 as tf
+tf.disable_v2_behavior()
 from pgportfolio.learn.nnagent import NNAgent
 from pgportfolio.marketdata.datamatrices import DataMatrices
 import logging

pgportfolio/marketdata/datamatrices.py

@@ -53,12 +53,15 @@ def __init__(self, start, end, period, batch_size=50, volume_average_days=30, bu
             raise ValueError("market {} is not valid".format(market))
         self.__period_length = period
         # portfolio vector memory, [time, assets]
-        self.__PVM = pd.DataFrame(index=self.__global_data.minor_axis,
-                                  columns=self.__global_data.major_axis)
+        panelxr = self.__global_data
+        self.__PVM = pd.DataFrame(index=panelxr.dim_2.values,
+                                  columns=panelxr.dim_1.values)
+
         self.__PVM = self.__PVM.fillna(1.0 / self.__coin_no)
 
         self._window_size = window_size
-        self._num_periods = len(self.__global_data.minor_axis)
+        self._num_periods = len(panelxr.dim_2.values)
+
         self.__divide_data(test_portion, portion_reversed)
 
         self._portion_reversed = portion_reversed
@@ -170,6 +173,8 @@ def setw(w):
 
     # volume in y is the volume in next access period
     def get_submatrix(self, ind):
+        p=self.__global_data
+        a = p.values[:, :, ind:ind+self._window_size+1]
         return self.__global_data.values[:, :, ind:ind+self._window_size+1]
 
     def __divide_data(self, test_portion, portion_reversed):
@@ -191,4 +196,4 @@ def __divide_data(self, test_portion, portion_reversed):
         # reversed and normal version
         self._train_ind = list(self._train_ind)
         self._num_train_samples = len(self._train_ind)
-        self._num_test_samples = len(self.test_indices)
+        self._num_test_samples = len(self.test_indices)

pgportfolio/marketdata/globaldatamatrix.py

@@ -3,9 +3,10 @@
 from __future__ import print_function
 
 from pgportfolio.marketdata.coinlist import CoinList
+import xarray as xr
 import numpy as np
 import pandas as pd
-from pgportfolio.tools.data import panel_fillna
+from pgportfolio.tools.data import panel_fillna, serial_data_fillna
 from pgportfolio.constants import *
 import sqlite3
 from datetime import datetime
@@ -69,8 +70,7 @@ def get_global_panel(self, start, end, period=300, features=('close',)):
         self.__checkperiod(period)
 
         time_index = pd.to_datetime(list(range(start, end+1, period)),unit='s')
-        panel = pd.Panel(items=features, major_axis=coins, minor_axis=time_index, dtype=np.float32)
-
+        panelxr = xr.DataArray(data=np.NaN, coords=[features, coins, time_index])
         connection = sqlite3.connect(DATABASE_DIR)
         try:
             for row_number, coin in enumerate(coins):
@@ -114,12 +114,15 @@ def get_global_panel(self, start, end, period=300, features=('close',)):
                     serial_data = pd.read_sql_query(sql, con=connection,
                                                     parse_dates=["date_norm"],
                                                     index_col="date_norm")
-                    panel.loc[feature, coin, serial_data.index] = serial_data.squeeze()
-                    panel = panel_fillna(panel, "both")
+
+                    serial_data_squeezed = serial_data.squeeze()
+                    serial_data_squeezed = serial_data_squeezed.reindex(time_index)
+                    serial_data_squeezed = serial_data_fillna(serial_data_squeezed, "both")
+                    panelxr.loc[feature, coin, :] = serial_data_squeezed
         finally:
             connection.commit()
             connection.close()
-        return panel
+        return panelxr
 
     # select top coin_number of coins by volume from start to end
     def select_coins(self, start, end):
@@ -162,7 +165,7 @@ def __checkperiod(self, period):
         elif period == DAY:
             return
         else:
-            raise ValueError('peroid has to be 5min, 15min, 30min, 2hr, 4hr, or a day')
+            raise ValueError('period has to be 5min, 15min, 30min, 2hr, 4hr, or a day {}'.format(period))
 
     # add new history data into the database
     def update_data(self, start, end, coin):
@@ -221,4 +224,4 @@ def __fill_part_data(self, start, end, coin, cursor):
                     cursor.execute('INSERT INTO History VALUES (?,?,?,?,?,?,?,?,?)',
                                    (c['date'],coin,c['high'],c['low'],c['open'],
                                     c['close'],c['volume'],c['quoteVolume'],
-                                    weightedAverage))
+                                    weightedAverage))

pgportfolio/resultprocess/plot.py

@@ -161,7 +161,7 @@ def _load_from_summary(index, config):
     @:param index: index of the training and backtest
     @:return: numpy array of the portfolio changes
     """
-    dataframe = pd.DataFrame.from_csv("./train_package/train_summary.csv")
+    dataframe = pd.read_csv("./train_package/train_summary.csv")
     history_string = dataframe.loc[int(index)]["backtest_test_history"]
     if not check_input_same(config, json.loads(dataframe.loc[int(index)]["config"])):
         raise ValueError("the date of this index is not the same as the default config")

pgportfolio/tools/data.py

@@ -120,9 +120,20 @@ def panel_fillna(panel, type="bfill"):
     frames = {}
     for item in panel.items:
         if type == "both":
-            frames[item] = panel.loc[item].fillna(axis=1, method="bfill").\
-                fillna(axis=1, method="ffill")
+            frames[item] = panel.loc[item].fillna(axis=1, method="bfill").fillna(axis=1, method="ffill")
         else:
             frames[item] = panel.loc[item].fillna(axis=1, method=type)
     return pd.Panel(frames)
 
+
+def serial_data_fillna(serial_data, type="bfill"):
+    """
+    fill nan
+    :param serial_data: the series to be filled
+    :param type: bfill or ffill
+    """
+    if type == "both":
+        ret_val = serial_data.fillna(method="bfill").fillna(method="ffill")
+    else:
+        ret_val = serial_data.fillna(method=type)
+    return ret_val

user_guide.md

@@ -1,6 +1,6 @@
 # User Guide
 ## Configuration File
-Under the `nntrader/nntrader` directory, there is a json file called `net_config.json`,
+Under the `PGPortfolio/pgportfolio` directory, there is a json file called `net_config.json`,
  holding all the configuration of the agent and could be modified outside the program code.
 ### Network Topology
 * `"layers"`
@@ -48,8 +48,8 @@ Under the `nntrader/nntrader` directory, there is a json file called `net_config
         * if it is online, new data that dose not exist in the database would be saved
 
 ## Training and Tuning the hyper-parameters
-1. First, modify the `nntrader/nntrader/net_config.json` file.
-2. make sure current directory is under `nntrader` and type `python main.py --mode=generate --repeat=1`
+1. First, modify the `PGPortfolio/pgportfolio/net_config.json` file.
+2. make sure current directory is under `/PGPortfolio/` and type `python main.py --mode=generate --repeat=1`
     * this will make 1 subfolders under the `train_package`
     * in each subfolder, there is a copy of the `net_config.json`
     * `--repeat=n`, n could followed by any positive integers. The random seed of each the subfolder is from 0 to n-1 sequentially.
@@ -103,7 +103,7 @@ There are three types of logging of each training.
 
 ## Download Data
 * Type `python main.py --mode=download_data` you can download data without starting training
-* The program will use the configurations in `nntrader/nntrader/net_config` to select coins and
+* The program will use the configurations in `PGPortfolio/pgportfolio/net_config` to select coins and
   download necessary data to train the network.
 * The downloading speed could be very slow and sometimes even have error in China.
 * For those who cann't download data, please check the first release where there is a `Data.db` file, put it in the database folder. Make sure the `online` in `input` in `net_config.json` to be `false` and run the example.
@@ -142,4 +142,4 @@ ons       1.000231      0.217216          1144              1360            731
 
 ```
 * use `--format` arguments to change the format of the table,
- could be `raw` `html` `csv` or `latex`. The default one is raw.
+ could be `raw` `html` `csv` or `latex`. The default one is raw.