A brief look into our AI journey
This week sets a major milestone with the introduction of deep learning in ZEROSPAM’s message filtering chain. Our R&D team lead by Pierre-Luc Vaudry, PhD and assisted by Pr. Alain Tapp of the MILA (Montreal Institute for Learning Algorithms) have worked hard on delivering this for over a year – building datasets, sampling data, cleaning the data, running experiments and of course coding and refactoring code again and again.
So what is IT all about?
I truly believe AI and especially deep learning will have profound impact on the software industry. For us at ZEROSPAM, the deep learning promise is of course better Ham/Spam classification of email messages typically expressed in FPR (False Positive Rate) and TPR (True positive rate or more simply spam catch rate). But it is also a step towards replacing a rule-based system maintained by humans with a system that can learn and apply new rules by itself.
For example, we see a lot of ransomware campaigns with slight variations in language in the message body, something simple for a human to recognize but difficult to express in the form of rules and pattern matching through regular expressions. Machine learning offers the promise of inferring the same generalizations that humans are capable of automatically.
What have we learned?
This project has shown us the importance of data, tons of data - without which no form of Machine Learning would ever be possible. In our case, that meant millions of samples that are both representative of our users traffic and well classified. Our data journey has been a long one but it is at the core of any successful AI or machine learning project.
We’ve also learn that the devil’s in the detail. We did not invent neural networks or stochastic gradient descent, we are merely using existing machine learning algorithms, but the hard part is in tuning them, optimizing the countless parameters and more importantly integrating these new processes into a complex message processing pipeline that does not tolerate delays. With these latests developments ZEROSPAM is setting the stage for the future and pushing towards becoming an AI driven email security company.