Merge pull request #65 from RADAR-base/release-0.2.2

Release 0.2.2

Author: yatharthranjan

README.md

@@ -1,9 +1,9 @@
 # RADAR-CNS back-end
 
-[![Build Status](https://travis-ci.org/RADAR-CNS/RADAR-Backend.svg?branch=master)](https://travis-ci.org/RADAR-CNS/RADAR-Backend)
-[![Codacy Badge](https://api.codacy.com/project/badge/Grade/e21c0c25f43e4676a3c69bae444101ca)](https://www.codacy.com/app/RADAR-CNS/RADAR-Backend?utm_source=github.com&utm_medium=referral&utm_content=RADAR-CNS/RADAR-Backend&utm_campaign=Badge_Grade)
+[![Build Status](https://travis-ci.org/RADAR-base/RADAR-Backend.svg?branch=master)](https://travis-ci.org/RADAR-base/RADAR-Backend)
+[![Codacy Badge](https://api.codacy.com/project/badge/Grade/e21c0c25f43e4676a3c69bae444101ca)](https://www.codacy.com/app/RADAR-base/RADAR-Backend?utm_source=github.com&utm_medium=referral&utm_content=RADAR-CNS/RADAR-Backend&utm_campaign=Badge_Grade)
 
-RADAR-Backend is a Java application based on Confluent Platform to standardize, analyze and persist data collected by RADAR-CNS data sources. It supports the backend requirements of RADAR-CNS project. The data is produced and consumed in Apache Avro format using the schema stored inside the RADAR-CNS [schema repository](https://github.com/RADAR-CNS/RADAR-Schemas).
+RADAR-Backend is a Java application based on Confluent Platform to standardize, analyze and persist data collected by RADAR-CNS data sources. It supports the backend requirements of RADAR-CNS project. The data is produced and consumed in Apache Avro format using the schema stored inside the RADAR-CNS [schema repository](https://github.com/RADAR-base/RADAR-Schemas).
 
 RADAR-Backend provides an abstract layer to monitor and analyze streams of wearable data and write data to Hot or Cold storage. The Application Programming Interfaces (APIs) of RADAR-Backend makes the process of to integrating additional topics, wearable devices easier. It currently provides MongoDB as the Hot-storage and HDFS data store as the Cold-storage. They can be easily tuned using property files. The stream-monitors monitor topics and notify users (e.g. via emails) under given circumstances. 
 
@@ -21,7 +21,7 @@ The following are the prerequisites to run RADAR-Backend on your machine:
 2. Clone radar-backend repository.
 
     ```shell
-    git clone https://github.com/RADAR-CNS/radar-backend.git
+    git clone https://github.com/RADAR-base/radar-backend.git
     ```
 3. Build the project from project directory
 
@@ -210,12 +210,12 @@ This image requires the following environment variable:
 - `KAFKA_SCHEMA_REGISTRY`: a valid Confluent Schema Registry.
 - `KAFKA_BROKERS`: number of brokers expected (default: 3).
 
-For a complete use case scenario, check the RADAR-CNS `docker-compose` file available [here](https://github.com/RADAR-CNS/RADAR-Docker/blob/backend-integration/dcompose-stack/radar-cp-hadoop-stack/docker-compose.yml)
+For a complete use case scenario, check the RADAR-base `docker-compose` file available [here](https://github.com/RADAR-base/RADAR-Docker/blob/backend-integration/dcompose-stack/radar-cp-hadoop-stack/docker-compose.yml)
 
 ## Contributing
 
 Code should be formatted using the [Google Java Code Style Guide](https://google.github.io/styleguide/javaguide.html).
-If you want to contribute a feature or fix browse our [issues](https://github.com/RADAR-CNS/RADAR-Backend/issues), and please make a pull request.
+If you want to contribute a feature or fix browse our [issues](https://github.com/RADAR-base/RADAR-Backend/issues), and please make a pull request.
 
 There are currently two APIs in RADAR-Backend: one for streaming data (RADAR-Stream) and one for monitoring topics (RADAR-Monitor). To contribute to those APIs, please mind the following.
 
@@ -269,19 +269,19 @@ Monitors can be used to evaluate the status of a single stream, for example whet
     ```
 - the default log path is the jar folder
 
-[1]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/MasterAggregator.java
-[2]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/AggregatorWorker.java
-[3]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/AbstractKafkaMonitor.java
-[4]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/KafkaMonitorFactory.java
-[5]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/DisconnectMonitor.java
-[8]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/topic/GeneralStreamGroup.java
-[10]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/DeviceTimestampExtractor.java
-[11]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/E4Worker.java
-[14]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/topic/E4Streams.java
-[15]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4Acceleration.java
-[16]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4BatteryLevel.java
-[17]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4BloodVolumePulse.java
-[18]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4ElectroDermalActivity.java
-[19]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4HeartRate.java
-[20]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4InterBeatInterval.java
-[21]: https://github.com/RADAR-CNS/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4Temperature.java
+[1]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/MasterAggregator.java
+[2]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/AggregatorWorker.java
+[3]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/AbstractKafkaMonitor.java
+[4]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/KafkaMonitorFactory.java
+[5]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/monitor/DisconnectMonitor.java
+[8]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/topic/GeneralStreamGroup.java
+[10]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/stream/aggregator/DeviceTimestampExtractor.java
+[11]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/E4Worker.java
+[14]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/topic/E4Streams.java
+[15]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4Acceleration.java
+[16]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4BatteryLevel.java
+[17]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4BloodVolumePulse.java
+[18]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4ElectroDermalActivity.java
+[19]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4HeartRate.java
+[20]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4InterBeatInterval.java
+[21]: https://github.com/RADAR-base/RADAR-Backend/blob/master/src/main/java/org/radarcns/empatica/streams/E4Temperature.java

build.gradle

@@ -11,7 +11,7 @@ plugins {
 //---------------------------------------------------------------------------//
 
 group = 'org.radarcns'
-version = '0.2.1'
+version = '0.2.2'
 ext.description = 'Kafka backend for processing device data.'
 
 mainClassName = 'org.radarcns.RadarBackend'
@@ -34,8 +34,8 @@ ext.mathVersion = '3.0'
 ext.findbugVersion = '3.0.1'
 ext.commonsCliVersion = '1.2'
 ext.mockitoVersion = '2.2.29'
-ext.radarCommonsVersion = '0.8.0'
-ext.radarSchemasVersion = '0.3.1'
+ext.radarCommonsVersion = '0.8.1'
+ext.radarSchemasVersion = '0.3.2'
 ext.subethamailVersion = '3.1.7'
 ext.jsoupVersion = '1.10.2'
 ext.slf4jVersion = '1.7.25'

gradle/profile.prod.gradle

@@ -2,4 +2,8 @@ task downloadRuntimeDependencies {
     description "Pre-downloads dependencies"
     configurations.compileClasspath.files
     configurations.runtimeClasspath.files
+}
+
+processResources {
+    expand(version: version)
 }

radar.yml

@@ -19,8 +19,10 @@ broker:
 
 #Kafka internal parameters
 stream_properties:
-  auto.commit.interval.ms: 1000
-  session.timeout.ms: 10000
+  auto_commit_interval_ms: 1000
+  max.request.size: 3500042 #Set message.max.bytes for kafka brokers higher than or equal to this value
+  retries: 15
+  session_timeout_ms: 20000
 
 #============================ Kafka Streams ============================#
 #The number of threads that a stream must be run according is priority
@@ -93,9 +95,11 @@ statistics_monitors:
       - android_phone_contacts
       - android_phone_usage_event
       - android_phone_relative_location
-    output_topic: source_statistics_radar_prmt
+    output_topic: source_statistics_android_phone
 
 #========================= Kafka stream ================================#
 stream_masters:
   - org.radarcns.stream.empatica.E4StreamMaster
   - org.radarcns.stream.phone.PhoneStreamMaster
+
+# persistence_path: /var/lib/radar/data

src/main/java/org/radarcns/monitor/SourceStatisticsMonitor.java

@@ -59,7 +59,8 @@ public class SourceStatisticsMonitor extends AbstractKafkaMonitor<GenericRecord,
     public SourceStatisticsMonitor(RadarPropertyHandler radar,
             SourceStatisticsMonitorConfig config) {
         super(radar, config.getTopics(), Objects.requireNonNull(config.getName(),
-                "Source statistics monitor must have a name"), "1",
+                "Source statistics monitor must have a name"), "1-"
+                        + config.getOutputTopic(),
                 new SourceStatisticsState());
 
         if (getStateStore() == null) {

src/main/java/org/radarcns/stream/phone/PlayStoreLookup.java

@@ -38,7 +38,7 @@
 public final class PlayStoreLookup {
     private static final Logger log = LoggerFactory.getLogger(PlayStoreLookup.class);
     private static final String URL_PLAY_STORE_APP_DETAILS = "https://play.google.com/store/apps/details?id=";
-    private static final String CATEGORY_ANCHOR_SELECTOR = "a.document-subtitle.category";
+    private static final String CATEGORY_ANCHOR_SELECTOR = "a[itemprop='genre']";
 
     private final long cacheTimeout;
     private final Cache<String, AppCategory> categoryCache;