Merge pull request #37 from fabian-sp/f-fda-and-mask

New features: Functional Graphical Lasso and lambda1 mask

Author: fabian-sp

.github/workflows/unit_test.yml

@@ -1,25 +1,31 @@
-# This is a basic workflow to help you get started with Actions
-
 name: Testing
 
-# Controls when the workflow will run
 on:
-  # Triggers the workflow on push or pull request events but only for the main branch
   push:
-    branches: [ main ]
+    branches: [ master ]
   pull_request:
 
   # Allows you to run this workflow manually from the Actions tab
   workflow_dispatch:
 
-# A workflow run is made up of one or more jobs that can run sequentially or in parallel
 jobs:
-  # This workflow contains a single job called "build"
   build:
-    # The type of runner that the job will run on
     runs-on: ubuntu-latest
 
-    # Steps represent a sequence of tasks that will be executed as part of the job
     steps:
-      - name: GitHub Action for pytest
-        uses: cclauss/[email protected]
+      - uses: actions/checkout@v3
+
+      - uses: conda-incubator/setup-miniconda@v2
+        with:
+          activate-environment: gglasso_env
+          environment-file: environment.yml
+          python-version: 3.8
+      
+      # Install package and run tests
+      - name: run tests
+        # This one is very important so we can reuse conda env from last step
+        shell: bash -l {0}
+        run: |
+          pip install -e .
+          conda list
+          pytest gglasso/tests -v

.readthedocs.yaml

@@ -21,7 +21,7 @@ formats:
 # Optionally set the version of Python and requirements required to build your docs
 python:
    install:
-    - requirements: requirements.txt
+    - requirements: readthedocs-requirements.txt
     - method: setuptools
       path: .
 

README.md

@@ -14,12 +14,17 @@ Graphical Lasso problems. <br>
 
 ## Getting started
 
-### Install via pip
+### Install via pip/conda
 
-The package is available on pip and can be installed with
+The package is available on pip and conda and can be installed with
 
     pip install gglasso
 
+or
+
+    conda install -c conda-forge gglasso
+
+
 ### Install from source
 
 Alternatively, you can install the package from source using the following commands:
@@ -35,9 +40,9 @@ Test your installation with
 
 ### Advanced options
 
-When installing from source, you can also install dependencies with `conda` via the command
+If you want to create a conda environment with full development dependencies (for building docs, testing etc), run:
 
-	$ while read requirement; do conda install --yes $requirement || pip install $requirement; done < requirements.txt
+	conda env create -f environment.yml
 
 If you wish to install `gglasso` in developer mode, i.e. not having to reinstall `gglasso` everytime the source code changes (either by remote or local changes), run
 
@@ -50,18 +55,22 @@ If you wish to install `gglasso` in developer mode, i.e. not having to reinstall
 
 ## Algorithms
 
-`GGLasso` contains algorithms for Single and Multiple Graphical Lasso problems. Moreover, it allows to model latent variables (Latent variable Graphical Lasso) in order to estimate a precision matrix of type **sparse - low rank**. The following algorithms are contained in the package.
-<br>
-1) ADMM for Single Graphical Lasso<br>
+`GGLasso` contains algorithms for solving a multitude of Graphical Lasso problem formulations. For all the details, we refer to the [solver overview in our documentation](https://gglasso.readthedocs.io/en/latest/solvers-overview.html).
+
+The package includes solvers for the following problems:<br>
+
+- **Single Graphical Lasso**<br>
+
+- **Group and Fused Graphical Lasso**<br>
+We implemented the ADMM (see [2] and [3]) and a proximal point algorithm (see [4]). 
 
-2) ADMM for Group and Fused Graphical Lasso<br>
-The algorithm was proposed in [2] and [3]. To use this, import `ADMM_MGL` from `gglasso/solver/admm_solver`.<br>
+- **Non-conforming Group Graphical Lasso**<br>
+A Group Graphical Lasso problem where not all variables exist in all instances/datasets.  
 
-3) A Proximal Point method for Group and Fused Graphical Lasso<br>
-We implement the PPDNA Algorithm like proposed in [4]. To use this, import `warmPPDNA` from `gglasso/solver/ppdna_solver`.<br>
+- **Functional Graphical Lasso**<br>
+A variant of Graphical Lasso where each variables has a functional representation (e.g. by Fourier coefficients).
 
-4) ADMM method for Group Graphical Lasso where the features/variables are non-conforming<br>
-Method for problems where not all variables exist in all instances/datasets.  To use this, import `ext_ADMM_MGL` from `gglasso/solver/ext_admm_solver`.<br>
+Moreover, for all problem formulation the package allows to model latent variables (Latent variable Graphical Lasso) in order to estimate a precision matrix of type *sparse - low rank*.
 
 ## Citation
 

docs/source/math-description.rst

@@ -114,6 +114,27 @@ where
 
 In ``GGLasso`` we implemented an ADMM algorithm for the above described problem formulation, possibly extended with latent variables. Have a look at the :ref:`Nonconforming Group Graphical Lasso experiment` in our example gallery.
 
+
+Functional Graphical Lasso
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+Functional Graphical Lasso is a variant of Single Graphical Lasso, where each variable is not represented as a scalar, but as a function (or time series) [ref13]_. Hence, assume that each variable has a :math:`M`-dimensional representation, for example coming from Functional PCA, Fourier transform or using a spline basis. 
+Functional Graphical Lasso can be understood as SGL but with each matrix entry being an :math:`M\times M` block, representing the association between two function variables. Therefore, the regularization results in each :math:`M\times M` block either being zero or non-zero, and typically being dense in the latter case.
+
+For :math:`p` variables, we compute the covariance matrix :math:`S \in \mathbb{R}^{pM\times pM}`. The problem formulation is 
+
+.. math::
+   \min_{\Theta \in \mathbb{S}^{p\cdot M}_{++}} - \log \det \Theta + \mathrm{Tr}(S\Theta) + \lambda_1 \sum_{j\neq l} \|\Theta_{jl}^M\|_{F}.
+
+With latent variables, our implementation solves
+
+.. math::
+   \min_{\Theta, L \in \mathbb{S}^{p\cdot M}_{++}} - \log \det (\Theta -L) + \mathrm{Tr}(S (\Theta -L)) + \lambda_1 \sum_{j\neq l} \|\Theta_{jl}^M\|_{F} + \mu_1 \|L\|_{\star}.
+
+
+We have a simple tutorial on this in the :ref:`Functional Graphical Lasso experiment`.
+
+
 Optimization algorithms
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
 
@@ -134,6 +155,7 @@ References
 .. [ref10] Foygel, R. and Drton, M. (2010). Extended Bayesian Information Criteria for Gaussian Graphical Models. In Lafferty, J., Williams, C., Shawe-Taylor, J.,Zemel, R., and Culotta, A., editors, Advances in Neural Information Processing Systems, volume 23. Curran Associates, Inc.
 .. [ref11] Condat, L. (2013). A Direct Algorithm for 1-D Total Variation Denoising, IEEE Signal Processing Letters, vol. 20, no. 11, pp. 1054-1057.
 .. [ref12] Kurtz,  Z.  D.,  Bonneau,  R.,  and  Mueller,  C.  L.  (2019).   Disentangling microbial associations from hidden environmental and technical factors via latent graphical models.
+.. [ref13] Qiao, X., Guo, S., and James, G. M. (2019) Functional graphical models, J. Amer. Statist. Assoc., 114, 211-222.
 
 
 

docs/source/solvers-doc.rst

@@ -22,3 +22,7 @@ GGL nonconforming
 .. autofunction:: gglasso.solver.ext_admm_solver.ext_ADMM_MGL
 
 
+Functional SGL
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+.. autofunction:: gglasso.solver.functional_sgl_admm.ADMM_FSGL

docs/source/solvers-overview.rst

@@ -52,6 +52,10 @@ Here, ``list_of_samples`` stands for your list of data samples as described abov
 
 We recommend to have a look at the :ref:`Nonconforming Group Graphical Lasso experiment` in our example gallery.
 
+Functional Graphical Lasso
+^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
+
+For Functional Graphical Lasso, we have implemented the ADMM algorithm, see ``from gglasso.solver.functional_sgl_admm import ADMM_FSGL``. Note that in [ref13]_ a block coordinate descent algorithm is proposed for solving.
 
 Further Remarks - proximal operators
 ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

environment.yml

@@ -0,0 +1,23 @@
+name: gglasso_env
+
+dependencies:
+  - python>=3.8
+  - numpy>=1.19.2
+  - numba>=0.53.1
+  - scipy>=0.11.0
+  - scikit-learn>=0.24.1
+  - pandas
+  - matplotlib
+  - seaborn
+  - networkx
+  - decorator==4.4.2
+  - sphinx==3.5.4
+  - pytest
+  - pip
+  - pip:
+    - regain
+    - scikit-fda
+    - sphinx-gallery==0.8.2
+    - sphinx_rtd_theme==0.5.2
+    - fdasrsf==2.3.11
+