In this system we show the solution for detecting anomalies in streaming IoT data with machine learning, using the spark cluster for distributed processing and Azure Databricks for making data science on raw data collected during all this time. We visualize anomalies and aggregations in browser for administrator monitoring. Data is stored in Azure ADLS for analysis.


  1. Overview of the system​
  2. Training ML algorithm to detect anomalies on Databricks
  3. Spark streaming anomaly predictions
  4. Redis stored data​
  5. Data science on ADLS sensor data
  6. Spring web server
  7. UI