Omdena collaboration projects(Part-1): My Omdena experience

Introduction

This article contains a summary of an Omdena project I and other collaborators participated in. It explains the problem, collaboration/solution structure, contribution and knowledge gained.

Project name: Unsupervised Identification of anomalies that lead to data center outages using the Root Cause Analysis (RCA) process.

The Problem:

In Data Centers, once an anomaly is detected, Site Reliability Engineers (SRE) have the challenge is to identify and understand the root cause of that anomaly asap, by analyzing the sequence of events that led to the anomaly. This task of identifying the root cause is done manually, in most cases, and takes significant time and effort.

Sensai’s target is to reduce the human effort required for root cause analysis by automating the whole process of identification and RCA, using state-of-the-art Machine Learning and causal inference mechanisms.

Link to the project details: https://omdena.com/projects/root-cause-analysis/

Dataset description: Simulated and real data from servers

Collaboration/ solution structure:

Divided into three tasks

EDA-data-cleaning: These involved cleaning the dataset provided by sense ai team. EDA and feature reduction were performed.

Modeling: These involve building an ML model to detect anomalies in the dataset. FB prophet was used to detect anomalies. Then causal inference using bayesian networks was performed on it

Pipeline deployment: This involved building a deployment pipeline and displaying with Pareto charts

My contributions

I helped with data cleaning and feature reduction with various methods like correlation. I was part of the modelling team where causal inference was used. We worked on Bayesian networks. I was also involved in task 3 which was deployment with kafka, and building the deployment pipeline

Knowledge gained:

I learnt new technologies like kafka, causal inference, Fb prophet for detecting anomalies.

Conclusion

I have learnt a whole lot from this experience of building an ML model for anomalies and Root cause analysis. Great collaboration and exposure to new technologies in the AI world.

Link to join omdena: https://omdena.com/

WRITER: OLUYEDE SEGUN . A(jr)

linkedin profile: https://www.linkedin.com/in/oluyede-segun-adedeji-jr-a5550b167/

twitter profile: https://twitter.com/oluyedejun1

TAGS: #AI #machinelearning #RCA #Anomalydetection #kafka #paretocharts